Tactical Intelligence: Joint Stars Full-Rate Production Decision Was Premature and Risky (Letter Report, 04/25/97, GAO/NSIAD-97-68).
GAO reviewed the Department of Defense's (DOD) recent decision to commit to the full-rate production of the Joint Surveillance Target Attack Radar System (Joint STARS), focusing on whether: (1) the system had demonstrated a level of maturity through testing to justify a full-rate production commitment; (2) DOD considered and resolved important cost and performance issues prior to making its decision; and (3) there are future actions that could reduce program risk. GAO noted that: (1) Joint STARS' performance during its combined development and operational test and the operational evaluation done in Bosnia do not support a decision to commit the system to full-rate production; (2) the system's operational effectiveness and suitability were not demonstrated during the operational testing; (3) DOD's decision to move Joint STARS into full-rate production was premature and raised the program's level of risk; (4) the program could have continued under low-rate initial production (LRIP) until operational effectiveness and suitability for combat were demonstrated and plans to address identified deficiencies and reduce program costs were completed; (5) DOD decided in favor of Joint STARS full-rate production without the benefit of that information; (6) during the period that the full-rate production decision was being considered, the Assistant to the President for National Security was promoting the sale of the system to the North Atlantic Treaty Organization (NATO); (7) in an August 10, 1996, memorandum to the Secretaries of State, Defense, and Commerce and to the Chairman of the Joint Chiefs of Staff, the Assistant to the President stated that: "We have been working through various military, diplomatic, and political channels to secure NATO support for a fall 1996 decision in principle by the Conference of Armament Directors...to designate [Joint STARS] as NATO's common system"; (8) a DOD official informed GAO that in November 1996, the NATO armament directors delayed their decision on Joint STARS for 1 year; (9) before DOD approved the full-rate production of Joint STARS, the Director of Operational Test and Evaluation (DOT&E) provided Congress with a Joint STARS "Beyond LRIP" report; (10) the report clearly indicates that further operational testing is needed, DOT&E could only declare effectiveness for operations other than war, and the system was unsuitable as tested; (11) DOD plans follow-on test and evaluation to address the deficiencies identified during the earlier testing: (12) there is an opportunity not currently under consideration that could reduce the Joint STARS' program cost and result in an improved system; (13) since the Joint STARS was approved for LRIP, the procurement cost objective of the Air Force's share of the Joint STARS has increased by about $1 billion, primarily due to the greater effort and more resources needed to refurbish the 25-30 year ol* --------------------------- Indexing Terms ----------------------------- REPORTNUM: NSIAD-97-68 TITLE: Tactical Intelligence: Joint Stars Full-Rate Production Decision Was Premature and Risky DATE: 04/25/97 SUBJECT: Radar equipment Military procurement Military aircraft Testing Military cost control Product performance evaluation Concurrency Life cycle costs Cost effectiveness analysis IDENTIFIER: E-8 Aircraft DOD Operation Joint Endeavor Bosnia Boeing 707 Aircraft Boeing 767-200 Aircraft Boeing 757 Aircraft Joint Surveillance Target Attack Radar System JSTARS ****************************************************************** ** This file contains an ASCII representation of the text of a ** ** GAO report. Delineations within the text indicating chapter ** ** titles, headings, and bullets are preserved. Major ** ** divisions and subdivisions of the text, such as Chapters, ** ** Sections, and Appendixes, are identified by double and ** ** single lines. The numbers on the right end of these lines ** ** indicate the position of each of the subsections in the ** ** document outline. These numbers do NOT correspond with the ** ** page numbers of the printed product. ** ** ** ** No attempt has been made to display graphic images, although ** ** figure captions are reproduced. Tables are included, but ** ** may not resemble those in the printed version. ** ** ** ** Please see the PDF (Portable Document Format) file, when ** ** available, for a complete electronic file of the printed ** ** document's contents. ** ** ** ** A printed copy of this report may be obtained from the GAO ** ** Document Distribution Center. For further details, please ** ** send an e-mail message to: ** ** ** ** <info@www.gao.gov> ** ** ** ** with the message 'info' in the body. ** ****************************************************************** Cover ================================================================ COVER Report to Congressional Committees April 1997 TACTICAL INTELLIGENCE - JOINT STARS FULL-RATE PRODUCTION DECISION WAS PREMATURE AND RISKY GAO/NSIAD-97-68 Joint STARS Production Decision (702226) Abbreviations =============================================================== ABBREV DOD - Department of Defense AFOTEC - Air Force Operational Test and Evaluation Center JointSTARS - Joint Surveillance Target Attack Radar System LRIP - low-rate initial production DOT&E - Director of Operational Test and Evaluation AWACS - Airborne Warning and Control System NATO - North Atlantic Treaty Organization Letter =============================================================== LETTER B-275858 April 25, 1997 Congressional Committees We have reviewed the Department of Defense's (DOD) recent decision to commit to the full-rate production of the Joint Surveillance Target Attack Radar System (Joint STARS). More specifically, we analyzed whether (1) the system had demonstrated a level of maturity through testing to justify a full-rate production commitment, (2) DOD considered and resolved important cost and performance issues prior to making its decision, and (3) there are future actions that could reduce program risk. This review was performed under our basic legislative responsibility and we are addressing it to you because it falls under your committees' jurisdiction. BACKGROUND ------------------------------------------------------------ Letter :1 Joint STARS is a joint Air Force and Army wide-area surveillance and target attack radar system designed to detect, track, classify, and support the attack of moving and stationary ground targets. This $11 billion major defense acquisition program consists of air and ground segments-- refurbished 707 aircraft (designated the E-8) equipped with radar, operation and control, data processing, and communications subsystems, together with ground stations equipped with communications and data processing subsystems. Low-rate initial production (LRIP)\1 of the Joint STARS aircraft began in fiscal year 1993. In line with 10 U.S.C. 2399, DOD's final decision to proceed beyond LRIP first required the DOD Director of Operational Test and Evaluation (DOT&E) to submit a report to Congress, referred to as the Beyond LRIP report, stating whether (1) the test and evaluation performed was adequate and (2) testing demonstrated that the system is effective and suitable for combat, that is, operationally effective and suitable.\2 The Joint STARS aircraft was scheduled to begin its initial operational test and evaluation--referred to as the Joint STARS multi-service operational test and evaluation\3 --in November 1995. That testing was delayed and then changed because of the deployment of Joint STARS assets to the European theater to support Operation Joint Endeavor in Bosnia. The Air Force Operational Test and Evaluation Center (AFOTEC) and the U.S. Army Operational Test and Evaluation Command conducted a combined development and operational test of Joint STARS from July through September 1995 and an operational evaluation of the system during Operation Joint Endeavor from January through March 1996. Two Air Force Joint STARS aircraft and 13 Army Joint STARS ground station modules were deployed to support Operation Joint Endeavor and operationally evaluated from January through March 1996. After analyzing the data from the combined development and operational test and the operational evaluation performed during Operation Joint Endeavor, AFOTEC issued its Joint STARS multi-service operational test and evaluation final report on June 14, 1996. DOT&E staff analyzed the same and additional data and the Director issued his Beyond LRIP report to Congress on September 20, 1996.\4 On September 25, 1996, the Under Secretary of Defense for Acquisition and Technology signed an acquisition decision memorandum approving the Joint STARS program's entry into full-rate production with a total planned quantity of 19 aircraft. -------------------- \1 Low-rate initial production of systems is to produce the minimum quantity necessary to (1) provide production-configured or representative articles for operational test and evaluation, (2) establish an initial production base for the system, and (3) permit an orderly increase in the production rate for the system sufficient to lead to full-rate production upon the successful completion of operational test and evaluation. \2 Operational effectiveness refers to the ability of a system to accomplish its mission in the planned operational environment. Operational suitability is the degree to which a system can be placed satisfactorily in field use considering such factors as reliability and maintainability. \3 The multi-service operational test and evaluation was to consist of a combined development and operational test and a dedicated operational test. \4 Our analysis focused principally on the details in the "Beyond LRIP" report rather than just the conclusions in the letter transmitting that report to Congress. Those details provide a clearer picture of Joint STARS' performance. For example, while the letter states "as far as suitability is concerned, [the system] did not meet its requirements in [Operation Joint Endeavor], a problem which would be exacerbated in a higher intensity conflict," the report states that Joint STARS "as tested is unsuitable" and provides detailed examples of its suitability problems. RESULTS IN BRIEF ------------------------------------------------------------ Letter :2 The Joint Surveillance Target Attack Radar System's performance during its combined development and operational test and the operational evaluation done in Bosnia do not support a decision to commit the system to full-rate production. The system's operational effectiveness and suitability were not demonstrated during the operational testing. For example, the DOD Director of Operational Test and Evaluation could only state that the system had demonstrated effectiveness for "operations other than war" and found that the system "as tested is unsuitable." He further reported that only 18 (25 percent) of 71 performance criteria tested were demonstrated met by the system and that further testing is required for the remaining 53. DOD's decision to move Joint STARS into full-rate production was premature and raised the program's level of risk. The program could have continued under LRIP until operational effectiveness and suitability for combat were demonstrated and plans to address identified deficiencies and reduce program costs were completed. Instead, DOD decided in favor of Joint STARS full-rate production without the benefit of that information. During the period that the full-rate production decision was being considered, the Assistant to the President for National Security was promoting the sale of the system to the North Atlantic Treaty Organization (NATO). In an August 10, 1996, memorandum to the Secretaries of State, Defense, and Commerce and to the Chairman of the Joint Chiefs of Staff, the Assistant to the President for National Security Affairs stated that: "We have been working through various military, diplomatic, and political channels to secure NATO support for a Fall 1996 decision in principle by the Conference of Armament Directors. . .to designate [Joint STARS] as NATO's common system." A DOD official informed us that in November 1996, the NATO armament directors delayed their decision on Joint STARS for a year. Before DOD approved the full-rate production of Joint STARS, DOT&E provided Congress with a Joint STARS Beyond LRIP report, as required by law. The report clearly indicates that (1) further operational testing is needed, (2) DOT&E could only declare effectiveness for operations other than war, and (3) the system was unsuitable as tested. Having issued this report, DOT&E is under no further obligation to report to Congress at the Beyond LRIP report level of detail on the adequacy of the operational testing or on whether the system has demonstrated effectiveness and suitability for combat. However, DOD plans follow-on test and evaluation to address the deficiencies identified during the earlier testing. There is an opportunity not currently under consideration that could reduce the Joint STARS's program cost and result in an improved system. Since the Joint STARS was approved for LRIP, the procurement cost objective of the Air Force's share of the Joint STARS has increased by about $1 billion. This is primarily due to the fact that it is taking greater effort and more resources to refurbish the 25-30 year old 707 airframes than previously anticipated. The estimated cost of procuring, refurbishing, and modifying each used 707 airframe to receive the system's electronics is now about $110 million. As early as 1992, the Boeing Company proposed putting the system on newer Boeing 767-200 Extended Range aircraft, but this proposal was not accepted at that time as cost-effective. Given the current 707 airframe procurement, refurbishment, and modification cost and a 1996 price for a commercial version Boeing 767-200 Extended Range aircraft of between $82 million and $93 million, it may now be more cost-effective for the Air Force to buy that or some other new, more capable aircraft. Such an aircraft could provide a longer life, greater room for growth, greater flight range, greater fuel efficiency, higher operational availability, and lower program life-cycle costs. TEST RESULTS DO NOT SUPPORT FULL-RATE PRODUCTION ------------------------------------------------------------ Letter :3 LRIP of the Joint STARS aircraft began in fiscal year 1993. By statute, 10 U.S.C. 2399, the "Secretary of Defense shall provide that a major defense acquisition program may not proceed beyond low-rate initial production until initial operational test and evaluation of the program is completed." Operational test and evaluation is the primary means of assessing weapon system performance in a combat-representative environment. It is defined as the (1) field test, conducted under realistic combat conditions, to determine an item's effectiveness and suitability for use in combat by typical military users and (2) evaluation of the results of such a test. If used effectively, operational test and evaluation is a key internal control measure to ensure that decisionmakers have objective information available on a weapon system's performance, thereby minimizing risks of procuring costly and ineffective systems. Joint STARS was moved from low-rate to full-rate production even though (1) it performed poorly during both the combined development and operational test and the operational evaluation in Bosnia, (2) excessive contractor effort was needed to support Operation Joint Endeavor, (3) the suitability and sustainability of the system is questionable since it uses refurbished 25-30 year old airframes, and (4) operational software is considered significantly immature. TEST RESULTS WERE REPORTED AS DISAPPOINTING ---------------------------------------------------------- Letter :3.1 In DOT&E's Beyond LRIP report, the DOT&E stated that Joint STARS had only demonstrated effectiveness for operations other than war. The report indicated that of three critical operational issues\5 to judge effectiveness, only one had been demonstrated as met ". . . with limitations." Those critical operational issues related to (1) performance of the tactical battlefield surveillance mission, that is, surveillance--"met with limitations"; (2) support of the execution of attacks against detected targets, that is, target attack support; and (3) the provision of information to support battlefield management and target selection, that is, battle management. The effectiveness critical operational issues were judged based on seven supporting measures. In its report to Congress, DOT&E listed four of those measures of effectiveness as "not met" during the system's combined development and operational test and did not list any as having been demonstrated during the Operation Joint Endeavor operational evaluation. However, of greater concern, according to DOT&E, is the fact that the system did not meet its overall suitability requirements during Operation Joint Endeavor. In his executive summary, the Director stated that most of DOT&E's Joint STARS concerns relate to operational suitability. He went on to say that "In the current configuration, the [Joint STARS] aircraft has not demonstrated the ability to operate at the required maximum altitude; adequate tactics, techniques, or procedures to integrate [Joint STARS] into operational theaters have not been developed; [Joint STARS] exceeded the break rate and failed the mission reliability rate during [Operation Joint Endeavor]. During [Operation Joint Endeavor], [Joint STARS] did not achieve the effective time-on-station requirement." He concluded that without corrective actions, "[Joint STARS] would not be suitable in higher intensity conflict" and later in the report judged that the system "as tested is unsuitable." Analysis of DOT&E's Beyond LRIP report indicates that not only did Joint STARS have disappointing test results but also that extensive follow-on operational testing of Joint STARS is needed. In its Beyond LRIP report, DOT&E presented a table that reported its findings of the combined development and operational test and Joint STARS Operation Joint Endeavor operational evaluation and indicates where further testing is required. Our analysis of that table indicates that at most only 25 of 71 test criteria could be judged met. DOT&E considers 18 of those 25 to require no further testing, that is, DOT&E judges them clearly met. However, our analysis also indicates that 19 test criteria were clearly not met and that as many as 26 might not have been met. Twenty-seven of the criteria could not be determined in either the combined development and operational test or the Operational Joint Endeavor operational evaluation. Of the 71 Joint STARS operational test and evaluation criteria listed, DOT&E indicates that 53, or about 75 percent, require further testing. In addition to the above, DOT&E also noted that there were several operational features present during Joint STARS Operation Joint Endeavor deployment that were essential to its mission accomplishment but were not included in the recent production decision. It provided two specific examples-- satellite communications and a deployable ground support station. DOT&E believes these features "will be a necessary part of the production decision to achieve a capable [Joint STARS] system." It also noted the need for other features-- moving target indicator clutter suppression, communications improvements, terrain masking tools for ground station module operators, and linkage to operational theater intelligence networks. Since at least two of the features present during Operation Joint Endeavor were "essential" to its mission accomplishment have already been developed, and may be needed "to achieve a capable Joint STARS system," those features should also be tested during the planned Joint STARS follow-on test and evaluation. -------------------- \5 DOD regulation 5000.2-R states that "critical operational issues are the operational effectiveness and operational suitability issues (not parameters, objectives or thresholds) that must be examined in operational test and evaluation to evaluate/assess the system's capability to perform its mission." It also states that "if every critical operational issue is resolved favorably, the system should be operationally effective and operationally suitable when employed in its intended environment by typical users." SIGNIFICANT CONTRACTOR INVOLVEMENT ---------------------------------------------------------- Letter :3.2 The degree of contractor involvement required during the operational evaluation indicates increased program risk and makes the reported Joint STARS performance appear better than it would have otherwise. The multi-service operational test and evaluation plan, in discussing contractor involvement during the testing, stated: "[Multi-service operational test and evaluation] must yield the most credible and objective results possible. All facets of the test effort must operate under the rules that support total objectivity and prevents improper data manipulation." The test plan also states that interim contractor support "will be limited to perform ground maintenance only; no in-flight support." Regarding the Army's ground station modules, it states that "the Army maintenance concept does not call for [contractor involvement] at any level . . . ." However, during Operation Joint Endeavor there was significant contractor support of the 2 aircraft and 13 ground station modules deployed. According to the AFOTEC report, "Approximately 80 contractors were deployed to support the E-8C. However, three or four [contractor] systems engineers flew on each flight to ensure they could provide system stability and troubleshooting expertise during missions. Additionally, three or four [contractor] software engineers were on the ground full time, researching and developing fixes to software problems identified during the deployment." AFOTEC also reported that "Each of the [ground station modules] had one contractor representative on site and on call with additional help available as necessary. Five contractor representatives remained at [Rhein-Main Air Base] and functioned as a depot." The AFOTEC report stated that the "test director agreed to contractor participation in the [Operational Evaluation] to a greater extent [than] permitted under US Public Law, Title 10, Section 2399." [Emphasis added.] When we formally expressed our concerns about the significant contractor involvement in Operation Joint Endeavor,\6 DOD did not directly acknowledge that contractors were utilized beyond the constraints of the law governing operational test and evaluations. It stated that "were this solely an [initial operational test and evaluation], contractors would not have been utilized beyond the constraints of 10 U.S.C. µ2399," and noted that the contractors were involved in the Joint STARS operation to support the mission. It further stated that employing Joint STARS in Operation Joint Endeavor "allowed the system to be operated and tested at a greater operational tempo than the system would have undergone in traditional testing." DOD also stated that "because of the developmental nature of the aircraft, we needed to have more contractor personnel involved than we would otherwise have had." It is understandable that DOD wanted to provide the best support possible in Operation Joint Endeavor. However, such significant contractor use neither supports a conclusion that the system is operationally effective or suitable for combat, nor is it indicative of a level of system maturity that justifies full-rate production. Joint STARS failure to meet its maintainability criteria during an operation less demanding than combat, even with such significant contractor involvement beyond that planned for in combat, also raises the question of the Air Force's ability to develop a cost-effective maintenance plan for the system. This issue is recognized in the Under Secretary's acquisition decision memorandum approving Joint STARS full-rate production. In that memorandum, the Under Secretary called for the Air Force to fully examine Joint STARS affordability, sustainability, and life-cycle costs, including the scope of contractor support. -------------------- \6 Joint STARS Production Decision (GAO/NSIAD-96-242R, Sept. 20, 1996). SYSTEM SUSTAINABILITY AND SUITABILITY IS QUESTIONABLE ---------------------------------------------------------- Letter :3.3 In discussing the sustainability of the Joint STARS system, DOT&E noted in its Beyond LRIP report that "It is not yet known what the operational tempo will be for Joint STARS." It concluded that "If it is determined that the system will be operated at rates similar to AWACS [Airborne Warning And Control System], it is questionable whether the [Joint STARS aircraft] can be sustained over time. Airframe problems have already been experienced on the existing [Joint STARS airframes], including a hydraulics failure and a cracked strut in the fuselage between the wings." In discussing the Joint STARS aircraft engines, DOT&E noted that they "are 1950s technology and may not be reliable" and cited AFOTEC's reporting that engine failures were among the principal reasons that the aircraft failed to meet the break rate criteria\7 during Operation Joint Endeavor. In discussing Joint STARS suitability, DOT&E also noted that the limited power of the engines "made it difficult to reach the aircraft's normal operating altitude of 36,000 feet, much less the 42,000 feet maximum altitude it is required to reach." It further reported that during Operation Joint Endeavor, the aircraft required approximately 11,000 feet of runway when taking off with 140,000 pounds of fuel and concluded that "this may pose a significant challenge to operational commanders because the [North Atlantic Treaty Organization] standard runway length is 8,000 feet." It noted that operational challenges would be great in other theaters as well and cited Korea as an example. It reported that Joint STARS ". . . would face operational challenges taking off from five runways in Korea, each approximately 9,000 feet long. Operations out of Korea would likely require taking off with less fuel and subsequent aerial refueling or shortening the time on station." Another area of Joint STARS suitability concern is the system's growth potential. DOT&E has reported that it is not clear that the remanufactured 707 platforms will be capable of incorporating all of the planned upgrades, noting that the airframe limits the system's growth potential both in weight and volume. It reported that as the current mission equipment already fills much of the fuselage, there is little room for expansion. DOT&E also noted that increasing the payload weight would require longer takeoff runways or taking off with less fuel, thus increasing the aerial refueling requirement or decreasing mission duration. DOT&E also noted that the system's current computers limited its growth potential due to their having very little reserve processor time or memory. It stated that the Air Force requires that no more than 50 percent of central processor unit cycles or memory be utilized by a new system. DOT&E reported that "None of the E-8C computer subsystems meet these requirements." It provided an example of the problem, stating that "the memory reserve of the operator workstations still does not meet the requirement, even after being increased from 128 megabytes to 512 megabytes just prior to [Operation Joint Endeavor]." This assessment is another indicator of the program's elevated risk. As DOT&E noted "Future software enhancements and modifications may require significant hardware upgrades. . . ." -------------------- \7 Break rate is one measure of system suitability. For Joint STARS operational testing it was defined as the percentage of missions flown for a specific period of time in which a previously mission-capable essential subsystem was inoperable once the aircraft landed. OPERATIONAL SOFTWARE RATED SIGNIFICANTLY IMMATURE ---------------------------------------------------------- Letter :3.4 The AFOTEC report specifically pointed to the lack of maturity in Joint STARS software. For example, AFOTEC reported that -- "during Joint STARS [multi-service operational test and evaluation], software deficiencies were noted on every E-8C subsystem;" -- the software "does not adequately support [the] operator in executing the mission;" and -- "Joint STARS software does not show the expected maturity trends of a system at the end of development." In discussing Joint STARS software maturity, DOD advised us that the AFOTEC report judged the system overall operationally effective and suitable. Specifically, in reference to software problems, DOD stated that "the majority of software faults that occurred during Operation Joint Endeavor were resolved while airborne in less than 10 minutes." However, both AFOTEC and DOT&E had some critical concerns regarding how Joint STARS software functioned. For example, according to AFOTEC, the "Joint STARS software is immature and significantly impedes the system's reliability and effectiveness," and according to DOT&E -- "Immature software was clearly a problem during [Operation Joint Endeavor]. . ." -- ". . .the prime contractor had to be called in to assist and correct 69 software-specific problems during the 41 E-8C missions . . . .an average of 1.4 critical failures per flight. . ." -- "Communications control was lost on 69 percent of the flights." -- "The system management and control processor failed and had to be manually reset on half of the flights." DOD has stated that the Air Force "plans several actions to mature the software and provide the required support resources" and that "an interim software release in April 1997 will correct some software deficiencies identified during the operational evaluation." DOD also noted that software updates will be loaded each year thereafter and that software changes are easily incorporated. How easily these software changes are incorporated remains to be seen because much of this software, according to AFOTEC and DOT&E, is poorly documented. For example, AFOTEC has reported that there are 395 deficiency reports open against the Joint STARS program, 318 of which are software related. DOT&E also stated that the more than 750,000 lines of Joint STARS software code are "poorly documented" and later commented that "Software problems with the communications and navigation systems were never fully corrected, even after extensive efforts by the system contractor."\8 These facts in combination with DOD's comments raise the serious question as to which software deficiencies are to be addressed in the planned April software update. -------------------- \8 AFOTEC stated that "Safety of flight is jeopardized due to invalid navigational commands sent from the FMS-800 [flight management system] and conflicting and/or invalid navigational data displayed to the flight crew." ALTERNATIVE AIRCRAFT SHOULD BE CONSIDERED ------------------------------------------------------------ Letter :4 There is an opportunity not currently under consideration that could reduce the Joint STARS program cost and result in an improved system. Since the Joint STARS was approved for LRIP, the procurement cost objective of the Air Force's share of the Joint STARS has increased by about $1 billion. Program costs escalated from approximately $5.2 billion to approximately $6.2 billion in then-year dollars. A DOD official informed us that of the $1 billion cost growth, $760 million is attributed to the increased cost to buy, refurbish, and modify the used 707 airframes to receive the Joint STARS electronics. The remaining cost growth is attributed to other support requirements and growth in required spare parts. At least as early as 1992, the Boeing Company proposed putting Joint STARS on newer Boeing 767-200 Extended Range aircraft, but this proposal was not accepted as cost-effective. According to the 1996 Boeing price list, the commercial version of this aircraft can be bought for between $82 million and $93 million depending on options chosen (this is flyaway cost--the cost of a plane ready to be flown in its intended use). Furthermore, the flyaway cost of a commercial Boeing 757, which a Boeing representative informed us is in many respects more comparable to the 707s being used, is listed at between $61 million to $68 million. The actual cost of procuring either of these aircraft could be lowered by volume discounts and by the cost of the commercial amenities not required. On the other hand, these aircraft would require modifications to receive Joint STARS equipment, which would raise their cost. DOD informed us that the cost of procuring, refurbishing, and modifying the current 707 aircraft to receive Joint STARS equipment is now estimated to be about $110 million per airframe. The cost of procuring and preparing new aircraft might be comparable or even less than the current cost. In addition, the Air Force would acquire a new platform that could have (1) greater room for growth (both volume and weight), (2) take off capability from a shorter runway, (3) greater time-on-station capability, (4) significantly improved fuel efficiency, (5) extended aircraft life over the 707 currently used, and (6) reduced operational and support cost. In commenting on a draft of this report, DOD stated that it considered alternatives to the current air platform, both before LRIP started and at the full-rate production decision point. It also stated that the cost of moving the Joint STARS mission to an alternative platform would outweigh the benefits. We note, however, that at a meeting with DOD and service officials to discuss that draft, we asked about the reported DOD and service analyses. One Air Force official stated that the Air Force's platform choice was not revisited prior to the full-rate production decision. None of the other 13 DOD and service officials present objected to that statement. Furthermore, when we asked for copies of the air platform analyses that were done in support of either the low- rate or the full-rate production decision, DOD was unable to supply those analyses. Finally, DOD officials have informed us that a Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance Mission Assessment\9 has been performed that indicates that the Air Force could acquire a more effective system while saving $3 billion through the year 2010 by moving the Joint STARS mission to either a business jet or an unmanned aerial vehicle following the procurement of the twelfth current version Joint STARS aircraft. -------------------- \9 The results of this mission assessment are being considered in the Quadrennial Defense Review, a process in which DOD is taking a fundamental look at its strategy, force structure, modernization, infrastructure, and readiness to meet future mission requirements. PRODUCTION COMMITMENT UNNECESSARY AND RISKY ------------------------------------------------------------ Letter :5 We have previously informed DOD of our concerns about the decision to move to full-rate production in spite of the numerous testing deficiencies reported by both AFOTEC and DOT&E. DOD responded that in making the decision to move to full-rate production, it "considered the test reports (both the services' and the Director, Operational Test and Evaluation's), the plans to address the deficiencies identified during developmental and operational testing, cost estimates, operational requirements, and other program information." Although DOD believes that "none of the deficiencies identified are of a scope that warrants interrupting production," the production decision memorandum clearly reflects a recognition that this program carries significant risk. In his memorandum, the Under Secretary of Defense for Acquisition and Technology directed (1) an update of the Joint STARS Test and Evaluation Master Plan to "address multi-service [operational test and evaluation] deficiencies (regression testing);" (2) acceleration of the objective and threshold dates for the planned Follow-on Operational Test and Evaluation; and (3) the Air Force to "fully examine [Joint STARS] affordability, sustainability, and life cycle costs\10 including the scope of contractor use for field- level system support." Factors other than system performance may have influenced the decision to move Joint STARS forward into full-rate production. DOD's full-rate production decision for this program occurred during the same time frame the Joint STARS system was actively being promoted as the U.S. government's candidate for meeting NATO's military requirement for a ground surveillance system. For example, in an August 10, 1996, memorandum to the Secretaries of State, Defense, and Commerce and to the Chairman of the Joint Chiefs of Staff, the Assistant to the President for National Security Affairs stated that "We have been working through various military, diplomatic and political channels to secure NATO support for a Fall 1996 decision in principle by the Conference of Armament Directors . . . to designate [Joint STARS] as NATO's common system." He went on to state that "I am writing to be sure you know that the President is personally committed to [Joint STARS], has engaged Chancellor Kohl on this issue and will continue his personal involvement with key allies to ensure our goal is achieved. I would ask that you underscore your personal support for our collective efforts on behalf of [Joint STARS] when you meet with your NATO and European counterparts." Notwithstanding DOD's September 1996 commitment to full-rate Joint STARS production, a DOD official informed us that the NATO armament directors in their November 1996 meeting delayed for 1 year any decision on designating Joint STARS as NATO's common system or pursuing an alternate system to be developed. -------------------- \10 We believe this analysis should include examining the use of new airframes. FOLLOW-ON OPERATIONAL TEST AND EVALUATION PLANNED ------------------------------------------------------------ Letter :6 In the process of moving the Joint STARS program forward into full- rate production, DOD produced a Beyond LRIP report for Congress and thus moved past a key congressional reporting requirement that serves as an important risk management mechanism. The Beyond LRIP report to Congress that is required before major defense acquisition programs can proceed into full-rate production serves to inform Congress of the adequacy of the operational testing done on the system and to provide it with a determination of whether the system has demonstrated effectiveness and suitability. Having issued this report, DOT&E is under no further obligation to report to Congress at the Beyond LRIP report level of detail on the adequacy of the operational testing or on whether the system has demonstrated effectiveness and suitability for combat. However, DOD plans follow-on test and evaluation of the system to address the deficiencies identified during the system's earlier testing. On September 20, 1996, DOT&E sent to Congress a Joint STARS "Beyond LRIP" report that (1) clearly indicates that further operational testing is needed, (2) could only declare effectiveness for operations other than war, and (3) stated that Joint STARS is unsuitable as tested. On September 25, 1996, DOD approved the full-rate production of Joint STARS. In the acquisition memorandum approving Joint STARS full-rate production, the Under Secretary of Defense for Acquisition and Technology called for an accelerated follow-on operational test and evaluation of Joint STARS that is to address the deficiencies identified in the initial operational test and evaluation DOT&E reported on in the Beyond LRIP report to Congress. The planned follow-on operational test and evaluation will provide an opportunity to judge the Joint STARS program's progress in resolving the issues identified in earlier testing. CONCLUSIONS ------------------------------------------------------------ Letter :7 Notwithstanding any concurrent efforts to have Joint STARS designated as a NATO common system, Joint STARS test performance and the clearly unresolved questions about its operational suitability and affordability should have, in our opinion, caused DOD to delay the full-rate production decision until (1) the system had, through the planned follow-on operational test and evaluation, demonstrated operational effectiveness and suitability; (2) the Air Force had completed an updated analysis of alternatives for the Joint STARS to address the identified aircraft suitability and cost issues; and (3) the Air Force had developed an analysis to determine whether a cost-effective maintenance concept could be designed for the system. Furthermore, as they were judged "essential" to mission accomplishment and needed "to achieve a capable Joint STARS system," the satellite communications and deployable ground support station features (present, but untested, during Operation Joint Endeavor) should also be tested during the planned Joint STARS follow-on operational test and evaluation. Concerns of the magnitude discussed in this report are not indicative of a system ready for full-rate production. The program should have continued under LRIP until the issues identified by AFOTEC and DOT&E were resolved and the system was shown to be effective and suitable for combat. Furthermore, the recent cost growth related to refurbishing and modifying the old airframes being used for Joint STARS and questions regarding the suitability of those platforms indicate an opportunity to reduce the program's cost and improve the systems acquired. We believe, therefore, that an updated study of the cost effectiveness of placing Joint STARS on new, more capable aircraft is warranted. RECOMMENDATION ------------------------------------------------------------ Letter :8 We recommend that the Secretary of Defense direct the Air Force to perform an analysis of possible alternatives to the current Joint STARS air platform, to include placing this system on a new airframe. MATTERS FOR CONGRESSIONAL CONSIDERATION ------------------------------------------------------------ Letter :9 Because of (1) DOD's decision to commit to full-rate production in the face of the test results discussed in this report and (2) its subsequent decision to do additional tests while in production to address previous test deficiencies, we are convinced that DOD plans to proceed with the program. However, if Congress agrees that there is unnecessarily high risk in this program and believes the risk should be reduced, it may wish to require that: -- The Air Force obtain DOT&E approval of a revised test and evaluation master plan (and all plans for the tests called for in that master plan) for follow-on operational testing to include adequate coverage of gaps left by prior testing and include testing of any added features considered part of the standard production configuration and that DOT&E considers key system components. -- DOT&E provide a follow-on test and evaluation report to Congress evaluating the adequacy of all testing performed to judge operational effectiveness and suitability for combat and a definitive statement stating whether the system has demonstrated operational effectiveness and suitability. -- DOD develop and provide Congress an analysis of alternatives report on the Joint STARS air platform that considers the suitability of the current platform and other cost-effective alternatives, and the life-cycle costs of the current platform and best alternatives. AGENCY COMMENTS AND OUR EVALUATION ----------------------------------------------------------- Letter :10 In commenting on a draft of this report, DOD disagreed with our recommendation that the Air Force be directed to perform an analysis of possible alternatives to the current Joint STARS air platform. It also disagreed with our suggestion that Congress may wish to require DOD to develop and provide Congress a report on that analysis. DOD stated that alternative platforms were considered prior to both the start of LRIP and the full-rate production decision. DOD stated that based on (1) the fact that over half the fleet is already in the remanufacturing process or delivered to the user; (2) the large nonrecurring costs that would be associated with moving the Joint STARS mission to a different platform; (3) the additional cost to operate and maintain a split fleet of Joint STARS airframes; and (4) the expected 4-year gap in deliveries, such a strategy would force the costs of moving the Joint STARS mission to a different platform outweigh the benefits. DOD's comment about having previously considered alternative platforms is inconsistent with the information we developed during our review and with Air Force comments provided at our exit conference. In an effort to reconcile this inconsistency, we requested copies of the prior analyses of alternative platforms, but DOD was not able to provide them. DOD's statement that the costs of moving the Joint STARS mission to another platform would outweigh the benefits contradicts Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance Mission Assessment briefings given the Quadrennial Defense Review. Those briefings recommend (1) limiting Joint STARS production to 12 aircraft, (2) moving the Joint STARS mission to either corporate jets or Unmanned Aerial Vehicles, and (3) phasing out Joint STARS 707 variants as quickly as the new platform acquisitions will allow. According to those briefings, implementation of this recommendation could result in a more effective system and save over $3 billion through fiscal year 2010. We believe that the issue clearly warrants further consideration. Furthermore, given DOD's resistance to the concept, we are more convinced of the merits of our suggestion that Congress might wish to require a report on such an analysis. In commenting on our draft report, DOD also indicated that congressional direction was unneeded on our suggestions that Congress might wish to require (1) DOT&E approval of a revised test and evaluation master plan for the planned Joint STARS follow-on operational test and evaluation and (2) DOT&E to provide Congress with a follow-on operational test and evaluation report on the adequacy of Joint STARS testing and stating whether Joint STARS has demonstrated operational effectiveness and suitability. DOD stated that congressional direction on the first point was unneeded because the Joint STARS full-rate production decision memorandum required that the test and evaluation master plan be updated for Office of the Secretary of Defense approval and current DOD policy is that DOT&E will review, approve, and report on oversight systems in follow-on operational test and evaluation. DOD also stated that congressional direction on the second point is unneeded because DOT&E has retained Joint STARS on its list of programs for oversight and is to report on the system in its annual report to Congress as appropriate. DOD's response did not directly address our point since, as DOD pointed out, the acquisition decision memorandum that approved full-rate production required Office of the Secretary of Defense approval, not DOT&E approval, of the follow-on operational test and evaluation master plan. During the course of our review, DOD officials informed us that there was significant disagreement between the Air Force and DOT&E as to what follow-on testing was needed. It was indicated that the issue would probably have to be resolved at higher levels within the department, an indication of greater flexibility than DOD implies. Furthermore, while DOD stated there were some improvements and enhancements "that could benefit the warfighter" and acknowledged that those features were not tested, it did not respond to our comments that DOT&E judged those features "essential" to mission accomplishment or commit to their operational test and evaluation. Given these facts, we have not only maintained our suggestion that Congress may wish to require the Air Force to obtain DOT&E approval of a revised test and evaluation master plan, but also strengthened it to include DOT&E approval of supporting test plans. In its response to our suggestion that Congress may wish to require that DOT&E provide it a detailed, follow-on test and evaluation report, DOD states congressional direction is unnecessary as DOT&E will report on the system, among many others, in its annual report to Congress. DOD's comment fails to recognize, however, that we are suggesting that, given the already reported test results, Congress may wish a more detailed report outlining the adequacy of and the system's performance during follow-on operational testing to help in its oversight and provide it assurance that the system's problems have been substantially resolved. Given that (1) Congress felt such reporting to be beneficial enough to require it before a system can proceed beyond LRIP and (2) the fact that DOT&E, in the required report provided for Joint STARS, could not certify effectiveness for war and found the system unsuitable as tested, we continue to believe that Congress may wish to require a similar report based on the follow-on operational test and evaluation planned. DOD's comments are reprinted in their entirety in appendix I, along with our evaluation. SCOPE AND METHODOLOGY ----------------------------------------------------------- Letter :11 To determine whether Joint STARS test performance indicates a maturity justifying full-rate production, we interviewed officials and reviewed documents in Washington, D.C., from the DOD Office of the Director of Operational Test and Evaluation and the Joint STARS Integrated Product Team. We reviewed the Air Force Operational Test and Evaluation Center's multi-service operational test and evaluation plan and its final report on that testing and the DOD Director of Operational Test and Evaluation's Beyond LRIP report. To determine whether DOD considered and resolved important cost and performance issues prior to making its full-rate production decision, we reviewed Joint STARS program budget documents and program-related memoranda issued by the Under Secretary of Defense for Acquisition and Technology. To determine whether it is possible that a more useful operational test and evaluation report can be provided Congress, we reviewed the statute governing operational testing and evaluation, examined DOT&E's Beyond LRIP report, and considered other relevant program information. We considered and incorporated where appropriate DOD's response to our September 20, 1996, letter of inquiry\11 and its response to a draft of this report. We conducted this review from October 1996 through April 1997 in accordance with generally accepted government auditing standards. -------------------- \11 Joint STARS Production Decision (GAO/NSIAD-96-242R, Sept. 20, 1996). --------------------------------------------------------- Letter :11.1 We are sending copies of this letter to other appropriate congressional committees; the Director, Office of Management and Budget; and the Secretaries of Defense, the Army, and the Air Force. Copies will also be made available to others upon request. If you or your staff have any questions, please contact me, Mr. Charles F. Rey, Assistant Director, or Mr. Bruce Thomas, Evaluator-in-Charge, at (202) 512-4841. Thomas J. Schulz Associate Director, Defense Acquisitions Issues List of Congressional Committees The Honorable Strom Thurmond Chairman The Honorable Carl Levin Ranking Minority Member Committee on Armed Services United States Senate The Honorable Ted Stevens Chairman The Honorable Daniel K. Inouye Ranking Minority Member Subcommittee on Defense Committee on Appropriations United States Senate The Honorable Floyd D. Spence Chairman The Honorable Ronald V. Dellums Ranking Minority Member Committee on National Security House of Representatives The Honorable C.W. Bill Young Chairman The Honorable John P. Murtha Ranking Minority Member Subcommittee on National Security Committee on Appropriations House of Representatives The Honorable Richard C. Shelby Chairman The Honorable J. Robert Kerrey Vice Chairman Select Committee on Intelligence United States Senate The Honorable Porter J. Goss Chairman The Honorable Norman D. Dicks Ranking Minority Member Select Committee on Intelligence House of Representatives (See figure in printed edition.)APPENDIX I COMMENTS FROM THE DEPARTMENT OF DEFENSE ============================================================== Letter (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) (See figure in printed edition.) The following are GAO's comments on the Department of Defense's (DOD) letter dated March 31, 1997. GAO COMMENTS ----------------------------------------------------------- Letter :12 1. We have not suggested or recommended that Joint STARS production be interrupted. We have, however, suggested actions that we believe (1) will help reduce the program's risk; (2) could result in the acquisition of a more effective, less costly system; and (3) could help decisionmakers ensure that the Joint STARS program continues to make progress. 2. The report has been modified in light of DOD's comments. 3. DOD's indication that other factors were considered in deciding to proceed to full-rate production is a signal that DOD and the Air Force are willing to accept a high level of risk even when the Director, Operation, Test, and Evaluation (DOT&E) has concluded that the system was unsuitable as tested and operational effectiveness for war remains to be demonstrated. We believe, given the system's test performance as reported by both the Air Force Operational Test and Evaluation Command (AFOTEC) and DOT&E and the program's procurement cost growth of $1 billion between the low-rate and full-rate production decision points, that an informed full-rate production decision required the following information: (1) an approved test and evaluation master plan for follow-on operational testing and specific plans for the tests called for in that master plan, (2) the results of the already ongoing study of ways to reduce the program's cost, and (3) an analysis of alternatives to the current platform. DOD did not have these items in hand when it made its decision. We must also note that DOD implies that our recommendations would require a break in production. This is inaccurate. As we stated in the body of our report, the program could have continued under low-rate initial production (LRIP) until operational effectiveness and suitability for combat were demonstrated and plans to address identified deficiencies and reduce program costs were completed. 4. In its report on the Joint STARS multi-service operational test and evaluation, AFOTEC stated that "Joint STARS software is immature and significantly impedes the system's reliability and effectiveness." We do not believe that, given the software intensive nature of the system, this statement supports a conclusion that the system could be judged operationally effective. 5. We must note that follow-on operational test and evaluation of the system was planned before the full-rate production decision. The full-rate production decision called for acceleration of that testing and for that testing to address deficiencies identified in the earlier tests. Joint STARS could have continued under LRIP pending a demonstration of operational effectiveness and suitability. 6. This speaks to the number of aircraft missions planned and the number for which an aircraft was provided. It does not address the quality or quantity of the support provided during those missions. Furthermore, DOD's comment refers to the same--operation that is reported on in both the Air Force and DOT&E reports and in this report. 7. U.S.-based contractor support was utilized during the first Operation Joint Endeavor deployment. It is also our understanding that during the second Operation Joint Endeavor deployment the Air Force may have utilized a "reach-back" maintenance concept in which U.S. stationed contractor staff were providing field support through satellite communications. Moreover, DOD and Air Force officials told us that at least at the beginning of the second Operation Joint Endeavor deployment, contractor staff were flying on the deployed aircraft. This clearly raises the question of what the overall level of contractor support was for both the first and second deployments. More importantly, a decrease in the level of contractor support between the two Operation Joint Endeavor deployments does not speak to (1) the poor test results during the first deployment with, and in spite of, the level of contractor support or (2) the quality of the system's performance during the second deployment; that is, there was no independent--DOT&E-- measurement or observation of how the system performed against its operational requirements. 8. Given that (1) the procurement cost growth of $760 million for 19 Joint STARS aircraft since the low-rate production decision; (2) a current 707 airframe purchase, refurbishment, and modification cost of about $110 million; (3) the age of the current airframes--25 to 30 years; and (4) the $7 billion estimated operations and maintenance life-cycle cost of those aircraft, we continue to believe that an analysis of alternatives to the current air platform should be performed, a belief bolstered by DOD's inability to provide copies of its reported analyses and by the recommendations of the Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance Mission Assessment, as discussed on pages 10, 11, 15, and 16 of this report. We believe that the issue clearly warrants further study. Furthermore, given DOD's resistance to the concept, we are more convinced of the merits of our suggestion that Congress might wish to require a report on such an analysis. 9. DOD stated that there were some improvements and enhancements "that could benefit the warfighter" but acknowledges that those features were not tested. It did not respond, however, to our comments that DOT&E judged some of those features "essential" to mission accomplishment. Furthermore, it did not state whether those features would be subjected to operational testing and evaluation. As we stated in the body of our report, since at least two of the features present during Operation Joint Endeavor were "essential" to its mission accomplishment, have already been developed, and may be needed "to achieve a capable Joint STARS system," those features should also be tested during the planned Joint STARS follow-on test and evaluation. 10. Given the level of contractor support during the multi-service operational test and evaluation, we are unable to understand how that support could not have impacted upon the system's test results. As we stated in our report, AFOTEC reported that 80 contractors were deployed, "three or four [contractor] systems engineers flew on each flight to ensure they could provide system stability and troubleshooting expertise during missions," and "three or four . . . were on the ground full time, researching and developing fixes to software problems . . . ." Furthermore, in its report DOT&E states that "While these [contractor] needs show that [Joint STARS] requires sophisticated support, they may also mask certain deficiencies." It also reported that "As already discussed, extensive efforts by the system contractor were required to achieve the demonstrated availability for the E-8C aircraft. Even with those efforts the system was not able to meet the user criteria for several measures directly related to the maintenance concept in place during [Operation Joint Endeavor]--a concept that involved considerably more contractor support than previously envisioned." 11. As we noted in the body of our report, Joint STARS failed to meet test criteria during an operation less demanding than combat, even with such significant contractor involvement beyond that planned for in combat. In discussing operational tempo in its Beyond LRIP report, DOT&E stated that if the system is operated at rates similar to the Airborne Warning and Control System, "it is questionable whether the [Joint STARS aircraft] can be sustained over time." DOD commented that an unbiased assessment of the measure of Joint STARS' ability to maintain the required tempo could not be made and would be tested during the follow-on operational test and evaluation. We believe that an informed full-rate production decision requires knowledge of a system's ability to satisfy the operational tempo expected of it. DOD made its Joint STARS full-rate production decision without this knowledge. 12. We understand that Joint STARS, like most systems, has limitations that need to be planned around. At issue here is a question of how great those limitations are and whether they are acceptable. DOD states that "the user is satisfied that the system meets requirements." However, we must note that the Air Force's own Operational Test and Evaluation Center reported that the "two critical suitability [measures of performance, sortie generation rate and mission reliability rate], were affected by [Operation Joint Endeavor] contingency requirements and system stability problems." AFOTEC stated that the sortie generation rate performance was undetermined and judged the other critical suitability measure of performance--mission reliability rate--as not being met. In discussing the later critical measure of performance AFOTEC reported that "The high failure rate of aging aircraft components affected [mission reliability rates] as critical failures were statistically determined to affect over 30 percent of the sorties flown. Analysis revealed the elevated critical failure rate was steady and showed no potential for improvement. Technical data and software immaturity affected the maintainability of the aircraft, and contractor involvement further compromised clear insight into the Air Force technicians' ability to repair the system." AFOTEC also reported on Joint STARS performance relative to 15 supporting suitability criteria. It stated "Eight did not meet users' criteria. One was not tested. Only one . . . met the users' criteria. The remaining five are reported using narrative results." 13. DOD discusses only the weight growth of funded activities, leaving open the question of whether there are future, but currently unfunded, improvements planned that will add weight growth. Air Force officials told us that the Airborne Warning and Control System had experienced weight growth over the life of its program. That growth was attributed to the system's being given added tasks over time. We believe it reasonable to expect that the Joint STARS program experience might track that of the Airborne Warning and Control Systems program, that is, be given added tasks and face weight growth as a result. Also, regarding Joint STARS room for growth, DOD previously advised us that Joint STARS currently has about 455,000 cubic inches of space available. We must note that this equates to a volume of under 7 feet cubed and that in commenting on the system's space limitation, DOT&E stated "There is little room available for additional people or operator workstations." 14. As we stated in the body of our report, how easily these software changes are incorporated remains to be seen. 15. We requested and DOD provided additional information on this point. DOD's subsequent response indicates that this DOD comment was in error. In its subsequent response, DOD stated that the follow-on test and evaluation was accelerated "to reflect [Office of the Secretary of Defense] desire for earlier [operational test and evaluation] to evaluate fixes to [multi-service operational test and evaluation] deficiencies." We believe this statement reflects a recognition of increased program risk. 16. The acquisition decision memorandum approving Joint STARS production clearly indicates that the Skantze study\1 mentioned was not completed at that time. We believe that the full-rate production decision should have been made with the Skantze study in hand. Furthermore, we do not understand why DOD felt the need to direct the Air Force to fund and implement a plan that is to save it money, but felt no need to direct the Air Force to examine alternative platforms that at least one other DOD panel had stated would not only save $3 billion but also provide greater effectiveness. 17. We believe that not only should DOT&E approval of the Joint STARS Test and Evaluation Master Plan be required, but also of all supporting test plans. We have changed the language of this matter for congressional consideration accordingly. 18. We are suggesting that Congress may wish to request a more detailed report, one at the Beyond LRIP report level of detail, a level of detail not provided in DOT&E's annual report. Given that DOT&E could only state effectiveness for operations other than war--could not state a belief as to whether the system would be effective in two of the three critical operational roles it is expected to perform in war--and found the system unsuitable as tested, we believe that such report would help Congress maintain program oversight. DOD's comment of "other reports as appropriate" leaves the matter in DOD's hand to decide if Congress would benefit from such a report. -------------------- \1 This study was an affordability review of Joint STARS that examined the affordability, sustainability, and life-cycle costs to thoroughly evaluate and control program costs. *** End of document. ***
Join the GlobalSecurity.org mailing list
