• Military Menu                       

• Introduction
• Systems
• Facilities
• Agencies
• Industry
• Operations
• Countries
• Hot Documents
• News
• Reports
• Policy
• Budget
• Congress
• Links

• WMD
• Intelligence
• Homeland Security
• Space

DOT&E FY2000 Annual Report

JOINT ELECTRONIC COMBAT TEST USING SIMULATION
(JECSIM)

Joint Test and Evaluation Program		Lead Service
Authorized Manning:	10	Navy
Total JT&E Budget:	$14.656M
Charter Date:	3QFY96
Completion Date:	4QFY00

Historically, electronic combat has made extensive use of simulation in the development and testing of new systems. It is becoming impractical to address all the needs of testing defensive countermeasures in open-air tests for reasons of complexity, safety, and security, cost, and availability of threat systems. As this trend continues, there is increased need for test and evaluation of the simulations themselves. The Joint Electronic Combat Test Using Simulation (JECSIM) Joint Test and Evaluation (JT&E) was chartered to begin such an assessment. The joint test was to determine the full range of engagement features needed to assess both performance and model accuracy for semi-active missiles that use radio frequency energy reflected from fighter, bomber, and helicopter aircraft being tracked by a powerful illuminating radar. This required the use of laboratory tests, Hardware-in-the-Loop (HITL) facilities, captive carry tests, ground mounted seeker facilities, signature measurement, fuse testing, and full-up open-air tests to address two issues:

1. The degree to which existing Modeling and Simulation (M&S) could be used to predict OT&E and DT&E results from semi-active missile engagements in ECM environments.
2. The sensitivity of probability of kill (P_k) calculations to changes in the end game geometry parameters predicted by M&S.

The tests focused on the SA-6 semi-active missile system and the ALE-50 (towed decoy) and ALQ-165 (Advanced Self-Protect Jammer) ECM suites. The M&S that was tested focused on Defense Intelligence Agency validated threat representations integrated into the Joint Model and Simulation System (JMASS) suite. The JMASS seeker model lacked the capability to simulate responses to the AH-64 onboard Electronic Combat (EC) system (ALQ-211), and the helicopter model further lacked a complex, dynamic signature model. In addition to addressing issues of M&S prediction quality, the measurements had value in their own right for ongoing programs. This JT&E was designed to improve the test and evaluation of ECM systems, which are part of the effort to provide full-dimensional protection to our troops.

This effort grew out of longstanding difficulties with open-air range testing of ECM systems. By 1993, a Flyout Model Working Group was meeting to define a common model set. In 1994, discussions focused on active versus semi-active missiles for consideration. The feasibility study for JECSIM grew out of these efforts, leading to chartering in August 1996. Fuse testing and modeling, P_k sensitivity analysis, and other start-up activities were conducted in 1997.

Activities during 1998 included the first phase of measurements at the HITL facility, preparation of detailed test plans for the captive carry measurements, and the second phase of HITL measurements. A Technical Advisory Group was formed in summer 1998 to address analysis issues. The group performed a technical review of the methodology for using test data to correlate with digital models. In this context "correlate" had a specific meaning-referring to the degree to which a large number of missile flight parameters "correlate" between the test and the simulation.

During 1999, JECSIM completed Ground Mounted Seeker (GMS) testing and the Captive Flight Test (CFT), and documented results from previous testing. In addition, JECSIM conducted simulation runs with JMASS 3.2M and JMASS 98 environments. The GMS test provided seeker interaction with real targets, with and without ECM. Targets of interest included the B-1B with the ALE-50 towed decoy and the F/A-18 with the AN/ALQ-165 ASPJ (Airborne Self-Protection Jammer). The CFT provided the most realistic clutter environment. Targets of interest included the B-1B and F/A-18. JECSIM completed the final reports for SA-6 live fire testing, laboratory testing, HITL testing, and Radar Cross-Section testing. M&S developments-the preparation of JMASS compliant threat models and the conversion to JMASS 98-led to more delays for JECSIM than from the physical measurement program.

With the threat models running in JMASS 98, rapid progress was made. JMASS 98 proved to be much more efficient than earlier versions because it allowed faster turnaround and greater ease of debugging modeling and data problems. The M&S work duplicating the lab and HITL measurements was completed at the end of FY99. HITL measurements and the related M&S work clearly demonstrated the impact of threat system variability (by serial number) on blue system performance. By serial number variability we mean the item-by-item difference in performance for a collection of systems of the same type. AFOTEC had previously demonstrated similar variability for a command guided missile in support of the ECM technique evaluation for the B-1B Defensive Systems Upgrade. These results conclusively established the importance of requiring robust designs of electronic warfare systems rather than point solutions.

The modeling of the Captive Flight Test and the Ground Mounted Seeker Test was completed in 1QFY00. A utility analysis assessed, somewhat subjectively, the utility of M&S in a variety of regimes.

Diagnostic testing with the two CFT seekers and the instrumentation package was conducted in November. This test revealed the source and cause of some CFT and GMS test data problems for the F/A-18 and provided an opportunity to capture the waveform from the onboard EC system.

Six JECSIM Integrated Product Team members and another 26 evaluators from Army, Navy, and Air Force components contributed to the utility evaluations of the simulation. The utility analysis considered functional (what must be represented), fidelity (accuracy of correlation), and operational (user-related issues) aspects for each of the activities of test planning, prediction, evaluation, and extrapolation of results. The simulation proved adequate for prediction of response to the off-board ECM and non-ECM cases for the bomber, but its utility for the fighter was limited. Utility for the helicopter was further limited due to lack of an adequate signature model and inability to simulate response to the onboard technique.

JECSIM developed a method and carried out an "extension analysis." This is a method for "extending" test results to different test conditions using validated M&S. The "extended" results are themselves quantitative predictions, with quantitative confidence measures for those predictions. The methodology is elaborate and its description is beyond the scope of this report. Nevertheless, it appears quite valuable in its intended use. In addition, it offers the possibility of using M&S to treat variability in threat systems (by serial number) to design robust electronic warfare solutions effective over the range of the variability.

JMASS 98 proved to be a much more useful tool in T&E than earlier versions of JMASS. Its use dramatically improved the JECSIM team's ability to execute M&S work.

JECSIM results conclusively demonstrate the need for robust electronic warfare designs based upon an informed assessment of threat system variability. Currently, existing M&S tools can be of great use here. Ideally, validated models based upon exploited systems would be used. However, with careful use, even models developed in the absence of fully exploited systems can support design and test of robust electronic warfare systems.

N-point scattering center models require significant resources to produce, are accurate for either far or near-field applications, and may require very large data sets. They and the software to produce them should be verified and validated.

Developing the extension analysis was an ambitious undertaking that shows promise. First, it provides a methodology for validated simulations of threat systems to be used to make quantitative predictions of blue jammer effectiveness against the actual threat systems, accompanied by a quantitative confidence level. These predictions can be extrapolated outside the region where the simulation was validated, and the confidence level informs the user of the likely accuracy of the predictions under the assumption that the extension outside the measurement region does not sample untested features of either threat or electronic warfare system. This was demonstrated for a few cases, but it remains to be seen how universal the application will be. Extension analysis may impact the area of variability. The extension analysis offers an approach to validating models and making predictions in cases where the serial number variability of the threat system leads to dramatic differences in system performance, especially at end game. Essentially, this methodology allows one to validate a system model against detailed test data when serial number variability leads to dramatic differences in field test outcomes. The validation comes with a confidence measure, which can be computed for any point in the operating space of the system. This is of tremendous potential value for designing robust systems.

Modeling and Simulation can have great value when coupled with physical measurements as part of an integrated T&E program.

The difficulties encountered by JECSIM in comparing modeling and simulation and measurement results show that for optimal value M&S efforts must start early. Ideally, there would have been an M&S infrastructure largely in place at the start of the measurement program.

There should be a standard approach for M&S application during requirements planning, system acquisition, and operation and support phases.

The added value of M&S comes over the life of a program and is not primarily a short-term benefit.

Consider a mix of simulation types, not just detailed, emulative EC capable seeker models for the support of future EC T&E programs.

Evaluate methods to shorten timelines for M&S development and integration with EC T&E programs.

Results of the end game analysis suggest that miss distance Measures Of Effectiveness should be used (instead of P_k) to evaluate EC systems during OT&E. P_k sensitivity analyses should be used to define miss distance criteria that can be safely and reliably tested using accurate instrumentation.

To reduce risk and explore conditions that cannot be tested, the T&E community needs constructive M&S that is reliable, flexible, readily available, and credible with a relevant validation history.

FY00 JOINT ELECTRONIC COMBAT TEST USING SIMULATION (JECSIM)