Since initial fielding, the U.S. E-3 AWACS has undergone nearly continuous modification. Early modifications included adding a maritime ship radar detection capability, integrating first generation Class 1 Joint Tactical Information Distribution System (JTIDS) data link terminals, and increasing operator displays from 9 to 14 to support considerably broadened mission tasks and workloads. A significant number of modifications update mission systems, sub-systems, flight controls, and navigation software, and replace selective hardware components with more reliable parts. Block 30/35, the most recent system upgrade, adds a 360-degree-coverage passive Electronic Support Measures (ESM) system to support detection and identification; incorporates a Global Positioning System navigation capability; replaces the Class 1 JTIDS terminal with a Class 2H (High-Power Amplifier, modified) terminal; and increases memory capacity of the central mission computer to support ESM and JTIDS.

The U.S. RSIP IOT&E started with its first sortie on August 3, 1995, and was suspended almost immediately when a mishap damaged the radar components. Investigation showed that RSIP software did not have an automatic safety feature, which existed in the pre-RSIP software, to prevent power from going to the radar when a particular non-RSIP relay failed. The radar was repaired, the software was modified, and IOT&E resumed with the first sortie successfully completed on October 12, 1995. The scheduled six-sortie IOT&E was again suspended because of serious performance deficiencies observed during IOT&E. IOT&E resumed for the third time in August 1996; it was completed in October 1996. RSIP met operational performance requirements at that time; however, suitability issues remained.

Data from U.S. IOT&E were augmented by system performance data gathered during NATO and U.K. tests/exercises, as well as a series of combined developmental/operational test flights. Piggybacking on previously planned activities reduced by approximately 40 percent the number of AWACS and test target sorties necessary to accumulate required data. This approach also reduced test duration by approximately six months.

After the conclusion of IOT&E, the Air Force developed a post-IOT&E action plan to correct the suitability deficiencies highlighted by IOT&E. The plan primarily consisted of software improvements, but also included some hardware improvements. DOT&E monitored the testing of those improvements and analyzed the data. Post-IOT&E results verified significant improvements in RSIP suitability.

DOT&E has been actively involved in identifying test issues and reviewing data during FOT&E. IOT&E involved testing the RSIP-modified AN/APY-2 radar found in the TS-3 test aircraft and one-third of the USAF operational fleet. FOT&E includes the first testing of an RSIP-modified AN/APY-1 radar system. Two-thirds of the U.S. fleet are equipped with the AN/APY-1 radar. Also, RSIP FOT&E provides the opportunity to test a complete production-representative ESM system with RSIP.

DOT&E analyzed the data from both U.S. and NATO IOT&Es and from combined DT/OT, including post-IOT&E testing. Testing confirmed that the RSIP-modified radar is capable of meeting current system-level performance requirements and is substantially more effective than the pre-RSIP radar it is intended to upgrade. RSIP is capable of tracking smaller radar cross-section targets at longer ranges than its predecessor. RSIP is also far more effective when operating against electronic countermeasures.

The RSIP-modified radar provides significant improvements in several areas of suitability. The improvements in man-machine interface are a quantum leap forward from the previous system. Additionally, in-flight repair time, diagnostic effectiveness, fault detection, fault isolation, and built-in-test "cannot duplicate" rates were all system successes. There have been no critical failures of RSIP hardware. However, the issue of software maturity plagued RSIP throughout testing prior to Milestone III. This is most evident in Mean Time Between Failure and Mean Time Between Critical Failure Rates, both of which fell well short of system goals. Despite this, the RSIP-only component break rate met requirements, and the overall radar break rate is comparable to the pre-RSIP operational fleet break rate.

The only negative impact to current system capabilities was to the Beyond-the-Horizon (BTH) radar mode. The U.S. crews indicated that they experienced a degraded ability to use the BTH mode effectively, although NATO crews reported no difference. The apparent cause of the change to BTH operations has been isolated to software. A software change has been made, and U.S. crews are in the process of determining the effectiveness of the fix and their preferences for tuning the BTH radar mode for operational use. There is insufficient test data available to make a final determination as to whether the BTH software change corrected the problem. That determination, and final selection of primary and alternate BTH operational modes, will be made after additional test flight data is gathered.

The initial FOT&E sortie flown in April 1998 provided baseline target detection and radar performance data for the AN/APY-1 equipped E-3. This aircraft was subsequently modified with the RSIP upgrade, and the May 20, 1999 dedicated RSIP FOT&E flight collected data against which to compare the baseline data. Observations of the pre- and post-RSIP sorties indicate that the RSIP-modified radar performs better than the pre-RSIP radar; however, final analysis of the data from the post-RSIP sorties is not yet complete.

Observations of the second dedicated FOT&E flight, conducted at Green Flag, showed that the aircraft was able to perform its mission as effectively as a pre-RSIP aircraft. No crew or computer workload issues were apparent during this sortie.

Software maturity remains the highest risk item in the RSIP program. Data collection continues, and observations during FOT&E sorties indicate that the software maturity and operational suitability of the RSIP-modified E-3 continue to improve.

The E-3 RSIP is operationally effective. Although a final determination has not been made regarding RSIP suitability, RSIP does provide significant improvements to the current system in many areas, and corrective actions are being pursued in the areas where deficiencies have been discovered. In the future, DOT&E will report on RSIP along with other E-3 AWACS improvements and upgrades.

Additional modifications should be made, and FOT&E flights continued, until it is determined that the BTH problem has been corrected to the satisfaction of the U.S. crews.

Software maturity and significant improvement to Mean Time Between Failure and Mean Time Between Critical Failure should be verified during FOT&E prior to the full fielding of RSIP.

Re-hosted radar software, combined with the new RSIP code, resulted in inadequate protection of aircraft radar hardware under certain operating conditions. Improved ground testing and design of new hardware interlocks were needed to ensure aircrew and aircraft safety. New procedures have been implemented to prevent further problems.

Software designed to ensure that RSIP met pulse-Doppler detection requirements inadvertently degraded the long-range detection and tracking performance of the BTH radar. This issue reinforced the need for thorough, ongoing compatibility testing for all system software changes.

Combining developmental and operational tests, as well as gathering test data from NATO and U.K. tests/exercises, significantly reduced test costs and duration. Future tests will use this same approach, leveraging existing activities wherever practical.

Agreements on test concepts and objectives during all phases of test must be thoroughly understood and documented. Key personnel changes in developer, user, and tester organizations resulted in intensive discussions to revalidate previously agreed to post-IOT&E and FOT&E test requirements. Possession of an updated TEMP and B-LRIP report identifying FOT&E issues and the outlining of a test strategy was instrumental in retaining the scope of testing during numerous personnel changes.