UNITED24 - Make a charitable donation in support of Ukraine!

Military

DOT&E Director, Operational Test & Evaluation  
FY98 Annual Report
FY98 Annual Report

SECURE, MOBILE, ANTIJAM, RELIABLE, TACTICAL TERMINAL (SMART-T)


Army ACAT IC Program: Prime Contractor
Total Number of Systems:313Raytheon
Total Program Cost (TY$):$678M 
Average Unit Cost (TY$):$1.3MService Certified Y2K Compliant
Full-rate production (IOC):1QFY99No (Est. December 1998)

SYSTEM DESCRIPTION & CONTRIBUTION TO JOINT VISION 2010

The Secure, Mobile, Antijam, Reliable, Tactical Terminal (SMART-T), a Milstar satellite communications transmit and receive terminal, is a core element of the Joint Service ground terminal segment of the Milstar satellite system. Operating at both the Milstar low (75-2400 bits/second) and medium (up to 1.544 mega-bits/second) data rates, it is designed to provide Army warfighters at corps, division and lower levels with secure, jam resistant, extended range, two-way, point-to-point and network voice, data, and video communications. These characteristics should enable our forces to maintain information superiority throughout all levels of conflict and support the full-dimensional protection operational concept and ensure that warfighters retain freedom of action through continuous, secure communication.

The primary SMART-T mission is multi-channel, near-global extended range connectivity for the Army's Mobile Subscriber Equipment, which is the primary tactical communications equipment for corps and division operations. SMART-T is intended to provide tactical commanders with the communications capacity and flexibility to electronically link networks of dispersed forces in situations beyond line-of-sight. This is accomplished by routing calls and messages from terminal-to-terminal via satellites orbiting 22,000 miles above the earth. The terminal is designed to be rapidly moved and set-up, communicate for short or extended periods of time, and be torn-down and moved again in response to changing tactical situations.

The SMART-T mounts to a High Mobility Multi-Purpose Wheeled Vehicle (HMMWV). This gives it the required high mobility in the tactical environment. The SMART-T consists of a HMMWV, pallet, generator, antenna, radio frequency equipment, associated electronics, remote operating unit, low data rate interface device and cable, AC to DC converter, chemical protection devices and support equipment. The terminal can be operated either mounted onto the HMMWV or as a stand-alone unit. In addition to overcoming the limitations of terrain masking and distance, the SMART-T is designed to operate and survive in severe Electronic Warfare and Nuclear, Biological, and Chemical environments.

The SMART-T operator will be trained to accomplish unit-level maintenance, which includes minor troubleshooting and repair. Satellite terminal maintenance specialists will provide general support to the SMART-T operators. Vehicle, generator, and support equipment repair will also be accomplished at the general support level. Repair tasks beyond the general support level ability will be evacuated to the appropriate specialized activity or depot.


BACKGROUND INFORMATION

The SMART-T acquisition strategy is designed to deliver terminals in advance of the first medium data rate Milstar satellite being placed in orbit. This would allow users immediate access to the first medium data rate capable satellite scheduled for launch in early 1999. The acquisition strategy does not require the terminals to demonstrate all operational effectiveness and suitability requirements during IOT&E and prior to the Milestone III full-rate production decision. Rather, the Milestone III production decision will be based on the SMART-T showing adequate progress toward meeting those requirements. The SMART-T is required to meet operational effectiveness and suitability requirements during FOT&E, which will occur prior to a fielding decision.

The SMART-T entered the engineering and manufacturing development phase of the acquisition process in May 1992 and the low rate initial production in February 1996. IOT&E was conducted from June 1-12 1998. The Milestone III full-rate production decision for the SMART-T is scheduled for 1QFY99. Since the first medium data rate satellite will not be available until 4QFY99, the SMART-T IOT&E was supported by the Lincoln Laboratory low/medium data rate, open-air satellite simulator rather than by an in-orbit satellite. An FOT&E using the in-orbit satellite is scheduled for late 1999. Developmental tests for the SMART-T consist of a First Article Test, Milstar Intersegment Tests, Interoperability Demonstrations, Reliability Growth Tests, Software Tests, and a Production Acceptance Test.


TEST & EVALUATION ACTIVITY

Army, Navy, and Air Force terminals participated in the Milstar Intersegment in 1997 and 1998. In these developmental tests, the Service terminals were connected to the Flight 3 Milstar satellite payloads at the contractor's facility. The SMART-T and the Navy EHF Satellite Communications Program (NESP) terminal participated in both low and medium data rate tests, while the Air Force Command Post Terminal participated in the low data rate tests. These tests examined the compatibility and interoperability of the SMART-T terminal with the Milstar low and medium data rate payloads. Tests included low and medium data rate satellite acquisitions, simultaneous low and medium rate network operations, interoperable low and medium rate network and point-to-point calls, antenna and network control functions, and Year 2000 rollovers.

The Army conducted the SMART-T IOT&E from June 1-12, 1998. The test used eight production representative terminals deployed in simulated corps and division scenarios over two 96-hour periods. Terminals were moved at 12- and 48-hour intervals to simulate tactical deployments on a mobile battlefield. The test networks were designed to replicate the communications requirements of the Army's Mobile Subscriber Equipment networks. Networks included all major components of Mobile Subscriber Equipment network, including Node Center Switches, Large Extension Node, Small Extension Node, and Remote Access Unit. The test was designed to stress the system and soldier/machine operations and demonstrate the operational effectiveness and suitability of the SMART-T. The IOT&E data was supplemented by test data gathered during SMART-T developmental tests, logistics demonstrations, Navy terminal testing, and Milstar Joint Service terminal, control, and space segment tests.

Testing was adequate to determine the operational effectiveness, suitability, and survivability of the SMART-T at the terminal level. However, testing was not adequate to determine SMART-T effectiveness in deployed operations requiring real-time changes to communication networks.

The principal test limitations are as follows:

  • ACMS was not available for IOT&E. ACMS is a stand-alone, computer-based communications planning tool used by in-theater headquarters staff to centrally plan, monitor, control and reconfigure SMART-T networks. ACMS is under separate development by the Milstar Joint Program Office and is not a part of the SMART-T terminal. The Army developed an interim planning tool, which was adequate to accomplish network planning IOT&E, but is not capable of performing the real-time network control, resource monitoring and network reconfiguration required for worldwide SMART-T deployment. Full ACMS capability will not be available until the Follow-On Test and Evaluation (FOT&E) in late 1999. This is consistent with the Milstar acquisition strategy.


  • The SMART-T acquisition strategy required IOT&E occur before a medium data rate capable Milstar satellite was in orbit. IOT&E, therefore, relied on the use of an open-air satellite simulator for medium data rate events. The simulator transmission levels were adjusted to emulate those of a satellite orbiting 22,000 miles above the earth. While the simulator did not duplicate all of the capabilities of an in-orbit satellite, it provided an adequate surrogate for this test. An in-orbit low data rate satellite was used to demonstrate acquisition and tracking of an orbiting satellite, as well as the ability to pass voice and message traffic. Medium data rate capabilities will be further evaluated in 1999 during FOT&E after the first medium data rate Milstar satellite becomes available.


  • SMART-T was not operationally tested under severe tropical or cold weather conditions. DOT&E has directed further testing to demonstrate the full range of operator/machine capabilities under operationally realistic tropical and cold conditions, to include multiple set-up and tear-down demonstrations.

The Milstar Intersegment Test 8000 will be conducted after Milstar Flight 3, the first medium data rate capable Milstar satellite, is launched from Cape Canaveral Air Station, Florida in 1999. After on orbit payload checkout is completed, SMART-T terminals will participate in combined Service testing to demonstrate the compatibility and interoperability of the SMART-T terminal with the in-orbit Milstar Flight 3 satellite. The tests include satellite acquisition, network operations, interoperable network and point-to-point calls with Army, Navy and Air Force terminals, and antenna and network control functions.

Y2K compliance testing consisted of three major events. Raytheon conducted functional performance testing at the Lincoln-Labs, MA, from February 11-14, 1997. Lockheed-Martin conducted Y2K testing in Sunnyvale, CA, as part of MST-4000 from July 1-10, 1997 and MST-6000 from July 20-24, 1998. The MST-series tests included Army and Air Force terminals as well as the Milstar Flight 3 satellite. Y2K issues were not found. The Army expects to complete Y2K compliance certification in December 1998.


TEST & EVALUATION ASSESSMENT

Operational Effectiveness

Even though SMART-T is capable of performing the communications functions for which the terminal was designed, operational effectiveness could not be fully demonstrated because ACMS was not available for IOT&E and there was no medium data rate Milstar satellite in orbit. Operational effectiveness will remain an open issue pending development of a suitable communications planning and management system and operational testing with an in-orbit medium data rate Milstar satellite.

Multi-channel range extension is satisfactory. Communications will be near-global and are possible wherever the terminals can establish a link through a satellite. This should be possible anywhere on the earth between 65 North and 65 South latitude when the Milstar constellation is complete in 2001. The operational requirement for call or message completion is 90 percent success on the first try. Call completion rates for voice were 99.5 percent and 100 percent for low and medium data rates, respectively. Message completion rates for data were 99.6 percent and 95.5 percent for low and medium data rates, respectively. Voice and data quality were determined to be adequate for operational use. The ability to pass video traffic, although not an operational requirement, was also demonstrated in developmental testing.

Deployability is not satisfactory. The SMART-T lacks an orderwire, the rudimentary communications link normally used by the Army when first establishing communications links among dispersed terminals. Although not listed as a terminal requirement, tests confirmed orderwire communications are required to quickly and efficiently establish communications networks. The Program Office is currently adding an orderwire to the SMART-T system.

The SMART-T terminal is a highly mobile tactical system, which can provide network communications for division and corps operations. Soldier operators deployed SMART-Ts, established and joined communications networks, and passed voice and message traffic at the required low and medium data rates.

The deployment requirement for the user to set-up, including establishment of communications, or tear-down the SMART-T within 30 to 45 minutes was not met 90 percent of the time. Requirements vary depending on soldier uniform: battle dress uniform (30 minutes max.), cold weather gear (45 minutes max.), and nuclear/biological/ chemical protective gear (45 minutes max.). Soldiers were able to meet the time requirements in 73 percent of the trials. Test results were quite variable with test times ranging from 9 to 77 minutes. DOT&E has directed these tests be repeated during FOT&E to more fully understand the reasons behind the large time variances.

Communications planning was not demonstrated. In order for SMART-T to be operationally effective, planners at theater headquarters must be able to centrally plan, monitor, control, and reconfigure networks of dispersed, mobile terminals in response to changing combat situations. ACMS, the stand-alone, computer-based communications planning tool still under development, is intended to provide these capabilities. The approved SMART-T acquisition strategy required only the central planning function for IOT&E. The network monitor, control, and reconfigure functions were not available for test. The ability to plan, monitor, control, and reconfigure networks in real-time is required at FOT&E.

Design features and operating characteristics of the SMART-T were adequate to enable soldiers to operate, use, and maintain the system. SMART-T design features and operating characteristics were gauged by observing the operators perform 15 critical tasks. These critical tasks included the complete process of setting-up and configuring the terminal, acquiring the satellite, setting-up networks, establishing communications, and tearing-down the networks and terminals. However, as noted below, some design features prevented the soldiers from performing as efficiently as desired.

Operational Suitability

SMART-T is not operationally suitable. This rating is based on the issues of Training; Soldier Efficiency; Soldier Safety, Health, and Survival; Reliability, Availability, and Maintainability (RAM); and Integrated Logistics Support (ILS). Although Soldier Safety, Health, and Survival are satisfactory, unsatisfactory results in the other categories result in the overall rating of not suitable.

Training provided by the Program Office and the Army's new equipment training team was not satisfactory. Crews committed errors resulting in five operational mission failures in 1507 hours of test, which equates to an operational mission failure every 310 operating hours (point estimate.) The goal is 700 hours or more between operational mission failure. Post-test questionnaires indicate the operators believe they lack the skill level necessary to be fully effective operating the system in combat even though they were able to successfully complete the four-week training course and perform SMART-T operations. Training will be further evaluated in FOT&E after the Army completes development of the final SMART-T training course.

Soldiers were unable to perform efficiently due to minor cuts, scrapes and bruises received during terminal set-up and tear-down. These were the result of man/machine interface, terminal design or manufacturing flaws such as pinch points, sharp edges and protruding parts. Efficiency was further hampered by lack of an orderwire, the rudimentary communications link used by the Army when first establishing communications links among dispersed terminals. Orderwire communications are required to quickly and efficiently establish communications networks. Additionally, troubleshooting procedures and technical manuals often provided erroneous or incomplete instructions.

Soldier Safety, Health, and Survival are satisfactory. Soldiers experienced no major hazards during set-up and tear-down of the SMART-T. Although minor cuts and abrasions occurred, none of the hazards observed were rated higher than marginal. The SMART-T terminal mechanical design and manufacturing processes must be improved to minimize sharp edges and pinch points. OPTEC human factors analysts have also recommended administrative controls requiring all training materials and technical manuals advise soldiers wear work gloves during set-up and tear-down of the terminal until the hazards are removed.

Reliability, availability and maintainability (RAM) are unsatisfactory. The most serious shortfall occurred in achieving the expected operational reliability. Although the developer's reliability growth curve indicated SMART-T Mean Time Between Failure exceeded the 400-hour requirement to enter IOT&E, field test results for Mean Time Between Failure were only 50 hours (point estimate). Also, the observed Mean Time Between Operational Mission Failure was 43 hours (point estimate) in IOT&E. This is substantially below the 700-hour requirement the system must demonstrate during FOT&E in late 1999. Failures were attributed to a wide range of software, hardware, training, procedural issues, and operator errors.

Operational availability during IOT&E was 72 percent and must grow to 98 percent at FOT&E. Maintainability indicators such as Mean Time to Repair also require similar growth if they are to meet FOT&E requirements. SMART-T will not be capable of sustained combat operations without significant RAM improvements.

Another RAM issue deals with performance in severe climate conditions. SMART-T performed well in the extreme hot and cold environmental tests during developmental testing with contractor personnel. However, DOT&E has directed further testing with military users performing the full range of system tasks under extreme operational environments as are found in Alaska and Panama. These tests should further establish SMART-T's ability to meet RAM requirements under all operating conditions.

Integrated logistic support is unsatisfactory due to several design flaws. Poor readability of computer screens in sunlight, lack of storage space, excessive generator noise and heat flowing into the vehicle cab, inflexible fuel hose, inadequate audible alarms for low fuel and loss of communications, and poor placement of generator switches are some of the design flaws observed. There are also additional concerns about the overall durability and quality of construction of some terminal pins, brackets, and other devices.

Operational Survivability

SMART-T is operationally survivable. The performance was evaluated using contractor and independent laboratory tests and analysis, and open-air tests and models.

The Army Research Laboratory and Lincoln Labs conducted SMART-T anti-jam and low probability of transmission interception performance using a combination of laboratory tests and models. Analysis indicates SMART-T meets the requirements for operating against fixed, mobile, and airborne jamming threats in an electronic warfare environment. Additionally, SMART-T transmissions have a low probability of detection and interception, characteristics which contribute greatly to the system's survivability in a combat environment.

Test and analysis indicate SMART-T meets the requirement to withstand high altitude electromagnetic pulse. As an additional interest item, DOT&E has directed the Program Office to investigate SMART-T vulnerability to non-nuclear, high-power microwaves. While SMART-T is not specifically designed to withstand high-power microwaves, DOT&E believes it to be in the best interest of the warfighter to understand the potential impact of this emerging threat.

Although SMART-T vulnerability to electromagnetic environmental effects is generally very low, not all effects have been fully evaluated. Testing revealed SMART-T may corrupt the quality of power busses connected to it by adding conducted interference to the power source in a narrow band near 75 kHz. The SMART-T did pass the contractor's electromagnet interference/connectivity tests, and no electromagnetic problems were seen during IOT&E while working in and among a normal compliment of communications equipment. However, OPTEC was unable to confirm contractor tests included the total system as it is deployed in the field. Lightning analysis indicates the SMART-T has the potential for substantial peak currents during a near strike.

DOT&E also directed SMART-T be tested to determine what effect the loss of the Global Positioning Satellite signal would have on the terminal's ability to operate should that signal be temporarily lost due to jamming. The SMART-T normally uses the highly accurate Global Positioning Satellite timing when acquiring the Milstar satellite signal. The IOT&E, using a variety of timing offsets, confirmed the SMART-T can acquire the Milstar satellite without this timing source.

Tests conducted at Dugway Proving Ground confirmed SMART-T could survive contamination and decontamination and return to normal operational use. Soldiers successfully operated the SMART-T in a simulated contaminated environment while wearing Mission-Oriented Protective Posture IV (MOPP IV) gear during IOT&E.

Conclusions

The following conclusions are based on the IOT&E test results:

  • Operational effectiveness was not demonstrated. Although the SMART-T terminal is capable of supporting communications for the Army's corps and division extended range operations, the SMART-T effectiveness as a deployed combat system cannot be confirmed until the Milstar communication management system is fully developed and operationally tested.


  • The SMART-T is not operationally suitable due to numerous training, soldier efficiency, RAM, and integrated support logistics issues.


  • Operational survivability is satisfactory.

Recommendations

The following recommendations are based on IOT&E results.

  • A fielding decision should not be made until FOT&E confirms SMART-T is both operationally effective and suitable. The production buy should be specifically limited to the first option (91 terminals).


  • The Milstar communications planning and network management system must be operationally tested during SMART-T FOT&E.


  • Set-up and tear-down times in normal, cold weather, and MOPP IV uniforms must be re-tested. Additionally, SMART-T must be field tested in tropical and cold weather conditions. Operators must perform set-up, tear-down and all normal communications operations.


  • Training, troubleshooting procedures and technical manuals must be significantly improved.


  • The numerous user man/machine interface shortcomings must be corrected. Additionally, the SMART-T terminal should be evaluated for overall quality of construction.


  • An orderwire capability should be developed and operationally tested during FOT&E.


  • RAM shortcomings, most notably Mean Time Between Failure and Mean Time Between Operational Mission Failure, must be corrected and verified by FOT&E.


  • Integrated logistics issues such as poor computer screen readability in sunlight, inadequate audible alarms, and poor placement of generator switches must be corrected.


  • DOT&E recommends SMART-T be evaluated for vulnerability to non-nuclear, high-power microwaves to determine its ability to withstand this emerging threat.

DOT&E will continue oversight of SMART-T and will work with the Program Office, the Army, and the operational test community to further refine the FOT&E test requirements and ensure SMART-T is operationally effective and suitable prior to fielding.

Return to Table of Contents



NEWSLETTER
Join the GlobalSecurity.org mailing list