Defense Acquisition: Comanche Program Objectives Need to Be Revised to More Achievable Levels (07-JUN-01, GAO-01-450)
GAO found that the Army's Comanche helicopter program faced
significant risks related to cost overruns, scheduling delays,
and degraded performance in August 1999, and that proceeding to
the next development phase with high levels of uncertainty was
not in accordance with best practices followed by successful
commercial firms. This report evaluates changes, since GAO's last
review in 1999, in the Comanche's status on cost, schedule, and
performance, and assesses whether the Army will have the
knowledge it needs to proceed with its current production plans.
First, GAO found that the Comanche program's total development
and production cost estimate has increased by almost $4.8
billion. However, areas of high technical risks and unfunded
requirements could further increase the program's costs. The
program office does not plan to update its April 2000 current
estimate to reflect these increases until January 2003. Second,
the Comanche's December 2006 full rate production decision date
has not changed even though the risks of not meeting this date
have increased. Third, the Army continues to face the risk that
critical performance requirements may not be met--at least for
the helicopters it will initially produce. The Department of
Defense recently provided $84 million in additional development
funding to help reduce some of these high-risk areas.
Additionally, the Army is not likely to have the knowledge it
should have to begin production when scheduled. It is also not
likely to know whether certain technologies being developed--such
as those used for the mission equipment package--will work on the
helicopter and function as expected and whether the helicopter
can be produced within current cost estimates.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-450
ACCNO: A01163
TITLE: Defense Acquisition: Comanche Program Objectives Need to
Be Revised to More Achievable Levels
DATE: 06/07/2001
SUBJECT: Army procurement
Helicopters
Military aircraft
Developmental testing
Operational testing
Defense capabilities
Schedule slippages
Cost overruns
Concurrency
Army Comanche Helicopter Program
RAH-66 Helicopter
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-450
Report to the Honorable Peter A. DeFazio, House of Representatives
United States General Accounting Office
GAO
June 2001 DEFENSE ACQUISITION Comanche Program Objectives Need to Be Revised
to More Achievable Levels
GAO- 01- 450
Page i GAO- 01- 450 Defense Acquisition Letter 1
Results in Brief 2 Background 4 The Comanche Program Continues to Experience
Cost Increases,
Schedule Delays, and Performance Shortfalls 6 The Army Plans to Begin
Comanche Production Despite Numerous
Uncertainties 15 Conclusion 17 Recommendations for Executive Action 17
Agency Comments and Our Evaluation 17 Scope and Methodology 18
Appendix I Comments From the Department of Defense 20
Appendix II GAO Contacts and Acknowledgments 22
Tables
Table 1: Timeline of Comanche?s Acquisition History and Schedule. 5 Table 2:
Comparison of Total Acquisition Cost By Category 6
Figures
Figure 1: Comparison of Comanche Pre- Production Aircraft Deliveries 9
Figure 2: Comanche Concurrent Flight Testing and Production 10 Contents
Page 1 GAO- 01- 450 Defense Acquisition
June 7, 2001 The Honorable Peter A. DeFazio House of Representatives
Dear Mr. DeFazio: Since 1983, the Army has been working to build its next-
generation helicopter, the Comanche, with the intention of significantly
expanding the Army?s capability to conduct attack and reconnaissance
operations in all battlefield environments, day or night and during adverse
weather conditions. With a projected total acquisition cost of about $48
billion, Comanche is the Army?s largest aviation acquisition program. In
June 2000, the Comanche program awarded a six- year engineering and
manufacturing development contract to Boeing- Sikorsky. The program office
plans to begin low- rate initial production in June 2005 and full- rate
production in December 2006. Current Army plans call for the acquisition of
1, 213 Comanches through fiscal year 2026. Success in meeting the Comanche?s
performance requirements largely depends on the Army?s ability to meet the
helicopter?s weight requirements and to develop and integrate advanced
technologies such as the critical mission equipment package, integrated
satellite communication capabilities, and on- board capability to detect and
isolate equipment problems.
Since its inception, the Army has restructured the Comanche program five
times, significantly delayed the development schedule, and reduced planned
quantities. In August 1999, 1 we reported that the program faced significant
risks related to cost overruns, scheduling delays, and degraded performance.
The Army faced these risks primarily because it decided to (1) begin
engineering and manufacturing development before key technologies had
matured, (2) compress the flight- test schedule, which increases concurrency
between developmental and operational testing, and (3) begin initial
production before completing operational testing. We also reported that by
proceeding to the next development phase with high levels of uncertainty,
the program?s actions were not in accordance with best practices followed by
successful commercial firms.
1 Defense Acquisition: Comanche Program Cost, Schedule, and Performance
Status (GAO/ NSIAD- 99- 149, August 24, 1999).
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 01- 450 Defense Acquisition
In response to the findings of our 1999 report, you requested that we (1)
evaluate changes in the Comanche?s status with regard to cost, schedule, and
performance and (2) assess whether the Army will have the knowledge it needs
on the helicopter?s performance and costs to proceed with its current
production plans.
The Comanche program continues to face significant cost, scheduling, and
performance risks.
First, since our last review, the program?s total development and
production cost estimate has increased by almost $4. 8 billion- from $43.3
billion to $48.1 billion. Development cost increased $75 million- from about
$8. 178 to $8. 253 billion and production cost increased by about $4. 8
billion. However, areas of high technical risks and unfunded requirements
could further increase the program?s costs. The program office does not plan
to update its April 2000 current estimate to reflect these increases until
January 2003.
Second, the Comanche?s December 2006 full rate production decision date
has not changed even though the risks of not meeting this date have
increased. In particular, the development and testing schedule has become
more compressed with many critical development and test events coming close
together or concurrently in the late stages of development. This, in turn,
has left the Army with very little time to correct deficiencies found during
testing. Failure to do so during development could result in costly
retrofits and repairs to aircraft already produced. These costs could be
substantial because the Army is planning to buy a significant number of pre-
production and low- rateinitial production aircraft before design and
testing are completed. The Army plans to use what it considers production-
representative aircraft produced during development for operational flight-
testing. As these aircraft are being tested, the Army plans to produce 84
low- rate initial production aircraft to equip the helicopter?s first
operational units.
Third, the Army continues to face the risk that critical performance
requirements may not be met- at least for the helicopters it will initially
produce. Specifically, the program is at risk of not (1) achieving the rate
of vertical climb requirement; (2) completing development and integration of
its mission equipment package, which is needed to support a range of
important functions including early warning, target acquisition, piloting,
navigation, and communications; (3) completing development of the system for
detecting equipment problems; and (4) achieving the ?beyond- line- of-
sight? communications Results in Brief
Page 3 GAO- 01- 450 Defense Acquisition
capability needed to perform its mission. The Department of Defense (DOD)
recently provided $84 million in additional development funding to help
reduce some of these high- risk areas.
Additionally, we found that the Army is not likely to have the knowledge it
should have to begin production when scheduled. The Army currently plans to
begin low- rate initial production of the Comanche in June 2005. Before
entering this stage, our work has shown that successful commercial firms
already know that (1) technologies match customer requirements; that is,
they can fit onto a product and function as expected, (2) the product?s
design meets performance requirements, and (3) the product can be produced
within cost, schedule, and quality targets. It is unlikely that the Army
will have this level of knowledge about Comanche by the June 2005 scheduled
low- rate initial production decision date. Specifically, the Army does not
plan to freeze Comanche?s design configuration until January 2006, or six
months after the low- rate initial production decision point. In addition,
the Army is not likely to know whether certain technologies being developed-
such as those used for the mission equipment package- will work on the
helicopter and function as expected and whether the helicopter can be
produced within current cost estimates. That level of knowledge will not be
obtained until much later when the results of operational flight- testing
are available and the contractor has more experience and data on producing
the fully developed Comanche helicopter.
In light of the current status and the significant challenges ahead, the
potential for undesirable outcomes for the Comanche program are high- higher
than expected costs, longer than expected schedules, and uncertain
performance. DOD and Army officials acknowledge that the current program
cost and schedule objectives are not achievable and should be changed to
reflect more realistic objectives. Yet they believe that the planned January
2003 review for the Comanche program is the appropriate time to address the
changes. Such a delay in revising the program?s cost and schedule estimate
limits the visibility and knowledge that Army and DOD management as well as
the Congress needs to (1) provide program oversight and direction; (2) make
effective cost, schedule, and performance trade- off decisions; and (3)
assess affordability and annual funding requirements. To improve management
oversight and direction and achieve more favorable program outcomes, this
report recommends that the Secretary of the Army reassess the program?s
cost, schedule, and performance objectives, and revise those objectives to
more achievable levels prior to submitting its next fiscal year budget.
Page 4 GAO- 01- 450 Defense Acquisition
In commenting on this report, DOD partially concurred with our
recommendation. DOD stated that it agreed with some of the report?s concerns
and recognizes there are some risks in the currently planned Comanche
engineering and manufacturing development program. DOD stated that it is
currently examining whether any of Comanche's requirements should be
deferred, in order to reduce the risk of not meeting cost and schedule
objectives. DOD disagreed with a reference to our previous Comanche report
stating that current program risks were caused by, among other things, the
program being allowed to enter engineering and manufacturing development
prior to maturation of key technologies. DOD maintains that the Comanche
program successfully demonstrated its exit criteria prior to entering
engineering and manufacturing development. However, the exit criteria did
not require that the technologies used in Comanche be at or above specific
levels of demonstrated readiness. As we noted in our 1999 report, the Army's
own assessments clearly indicated that several key areas of technology were
not at those levels called for in commercial best practices guidelines. 2
The Comanche helicopter program began in 1983 to provide a family of high
technology, low- cost aircraft that would replace the Army?s light
helicopter fleet, which includes the AH- 1 Cobra, OH- 58 Kiowa, OH- 6
Cayuse, and the UH- 1 Iroquois (Huey). The Army subsequently decided to
develop only a single Comanche aircraft capable of conducting either armed
reconnaissance or attack missions. The Army intends for the Comanche to be
part of its future or ?objective? force. 3
The Comanche is designed to have improved speed, agility, aircrew
visibility, reliability, availability, and maintainability over current
reconnaissance and attack helicopters. The helicopter is also designed for
low observability (stealth) and is expected to be capable of deploying over
long ranges without refueling. Lastly, the Comanche is being designed to
provide enemy information to force commanders at all levels.
2 Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes (GAO/ NSIAD- 99- 162, July 30, 1999). 3 On October
12, 1999, the Chief of Staff of the Army announced plans to radically
transform the Army. The transformation strategy is designed to ensure that
the Army can respond to a broad range of operations- from peacekeeping, to
regional conflicts, to major theater wars. This strategy centers on
developing a new combat force that is expected to be lighter, but just as
powerful and survivable as today?s heavy force. The new force is planned
around a common unit design and a family of combat vehicles that can be
transported on an Air Force C- 130- type of transport aircraft. The Comanche
will be the Army?s objective force reconnaissance aircraft. Background
Page 5 GAO- 01- 450 Defense Acquisition
Critical to achieving the Comanche?s desired capabilities is the successful
development and integration of advanced technologies, especially for the
mission equipment package. The mission equipment package includes an
integrated communication system, piloting system, target acquisition system,
navigation system, helmet- mounted display, survivability and early warning
equipment, mission computer, and weapon management system.
The Comanche program started in 1983 and is currently projected to continue
through fiscal year 2028. A timeline of the Comanche?s acquisition history
and schedule is provided below.
Table 1: Timeline of Comanche?s Acquisition History and Schedule. Dates
Program Phase Highlights
1983 to April 2000
Demonstration and Validation a Program restructured 5 times. Last
restructuring extended development of the Comanche from 1996 to 2006 and
reduced planned quantities from 2, 096 to 1, 292. Subsequently, quantities
were reduced to 1, 213. Army decreased the number of developmental aircraft
planned, accelerated the development of the fire control radar by 5 years,
and extended the production schedule. April 2000 to Dec. 2006 Engineering
and
Manufacturing Development In June 2000, a 6- year engineering and
manufacturing development contract was awarded to Boeing- Sikorsky.
First pre- production aircraft were originally scheduled to be delivered in
April 2002. Subsequently, that delivery has been delayed until January 2004.
Developmental flight- testing is scheduled to start in mid- 2004 and
continue through December 2006. Program reviews are scheduled for January
2003 and January 2005. June 2005 to Oct. 2007 Low- Rate Initial Production
Low- rate initial production of 84 aircraft scheduled to begin in June 2005.
Initial operational test and evaluation of pre- production aircraft is
scheduled for June 2006 through October 2006. Initial Operational Capability
is scheduled for December 2006 (also using aircraft built in engineering and
manufacturing development). Delivery of the first low- rate production
helicopter is planned for October 2007. Dec. 2006 to Fiscal Year 2028
Full- Rate Production Full- rate production decision is scheduled for
December 2006. Last production is planned for 2026, with the last unit to be
equipped occurring in 2028. a Now known as the program definition and risk
reduction phase.
Page 6 GAO- 01- 450 Defense Acquisition
Since our August 1999 review, the Comanche program?s estimated cost has
increased significantly- from $43.3 billion to $48. 1 billion- and costs are
expected to increase further. In addition, the Comanche continues to
experience scheduling delays and performance risks. These problems are due
to a range of factors, such as understated acquisition program cost
estimates; ambitious flight test schedules with substantial concurrency in
test events; delays in another DOD program which had been counted on to
develop a critical component of the aircraft; inadequate facilities to fully
test and integrate system hardware and software; and considerable growth in
aircraft weight. The Army has not updated the Comanche?s cost or schedule
estimates since April 2000 and does not plan such an update until its in-
progress program review in January 2003.
The Comanche program?s latest cost estimate, in April 2000, shows estimated
costs have increased by almost $4. 8 billion- from $43. 3 billion to $48. 1
billion- since our last report. Table 2 identifies where the cost estimate
has changed.
Table 2: Comparison of Total Acquisition Cost By Category
(then year dollars in millions)
Research, Development, Test & Evaluation Procurement Military
Construction Total
Current Program Costs Estimate $8,253. 8 4 $39,358. 2 $522.3 $48,134. 3
Prior Program Costs Estimate $8,178. 5 $34,581 $589.8 $43,339. 3 Cost
Estimate Increases $ 75. 3 $ 4,777 -$ 67. 5 $ 4,795
The $75. 3 million increase in research, development, testing, and
evaluation resulted from added testing for the Comanche program. During the
Milestone II 5 decision process, the Defense Acquisition Executive directed
that the Comanche testing program be expanded by adding more
4 The total research, development, testing, and evaluation cost estimate
includes about $4. 2 billion spent in previous years. 5 A milestone decision
point is when a recommendation is made and approval sought
regarding starting or continuing an acquisition program. The Comanche
Program Continues to Experience Cost Increases, Schedule Delays, and
Performance Shortfalls
Cost Estimate Increases
Page 7 GAO- 01- 450 Defense Acquisition
testing to fully demonstrate the aircraft?s reliability before completion of
its engineering and manufacturing development phase.
The $4. 777 billion increase in estimated production cost was to address DOD
concerns about the long- term affordability and stability of the Comanche
program. Specifically, DOD directed the Army to add 10 percent to Comanche?s
production unit cost estimate in order to ensure that annual planned
procurement funding would be sufficient to cover planned procurement
quantities. To reduce the annual funding increase resulting from this
directive, the Army reduced Comanche?s peak annual production rate from 72
aircraft per year to 62 per year, which extended the planned delivery
schedule by 3 years.
The $67. 5 million reduction in estimated military construction costs
reflects changes in anticipated needs for operating and maintenance
facilities.
In January 2001, DOD added about $504 million in funding to the Comanche
program over the next few years. About $84 million of the additional funds
are earmarked for research, development, test, and evaluation, and the
remaining $420 million for production. These additional funds have not yet
been reflected in the program?s official cost estimates. The program office
plans to use the additional development funding to at least partially
address what had been unfunded requirements in three areas considered to be
high risk: (1) developing and integrating the mission equipment package; (2)
developing the technology to detect and isolate equipment problems
(automatic fault isolation); and (3) developing and integrating satellite
communication capabilities. The section on performance discusses these areas
in more detail.
The Comanche?s most recent cost estimate was made in April 2000, when DOD
approved the program for entry into the engineering and manufacturing
development phase. At that time, DOD?s Cost Analysis Improvement Group
estimated that the Comanche program would need an additional $180 million
for its engineering and manufacturing development phase. However, the higher
costs estimated by the Cost Analysis Group were not included in the cost
estimate when the program office established a new baseline 6 for the
Comanche program in April 2000. The
6 A baseline is a detailed estimate of acquisition and ownership costs
normally required for high level decisions. This estimate is performed early
in the program and serves as the base point for all subsequent tracking and
auditing purposes. Additional Funds Recently
Provided by DOD Additional Cost Growth Likely
Page 8 GAO- 01- 450 Defense Acquisition
Comanche program is scheduled for an in- progress program review in January
2003 to review, among other things, its cost estimate. DOD believes that
this January 2003 review, along with other major program reviews and
oversight processes will permit successful management of program risks. The
Deputy Program Manager acknowledged that the Army?s cost estimate for the
Comanche may need to be revised at this point.
The Comanche program office also maintains a list of unfunded requirements.
The additional funds recently added to the program have reduced these
funding requirements, but the revised list still has unfunded requirements
in the amount of $68 million. The program office acknowledges that, unless
additional funds are obtained, some yet- to- bedetermined program
performance requirements could be impacted.
We have reported that when development work and low- rate initial production
are done concurrently, significant schedule delays that cause cost increases
and other problems are not uncommon in early production. Also, production
processes are often not able to consistently yield output of high quality
when full- rate production begins. 7 DOD?s guidance also states that
programs in which development work and low- rate initial production are done
concurrently typically have a higher risk of production items having to be
retrofitted to make them work properly and of system design not being
thoroughly tested. We have also reported that the discovery of problems in
testing conducted late in development is a fairly common occurrence on DOD
programs, as is the attendant ?late
cycle churn?, that is, the unanticipated effort that must be invested to
overcome such problems. 8 Further, these problems could be exacerbated if
the program plans to produce a significant number of systems during the low-
rate initial production period, before design and testing are completed.
In August 1999, we reported that the Army would experience a 19- month delay
in testing because the first pre- production aircraft for testing were
expected to be delivered 19 months later than planned. We noted that, by
retaining the December 2006 initial operating capability date, the delay in
7 Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions (GAO/ T- NSIAD- 00- 137, April 26, 2000). 8 Best Practices:
A More Constructive Test Approach Is Key to Better Weapon System Outcomes
(GAO/ NSIAD- 00- 199, July 31, 2000). Highly Compressed and
Concurrent Schedule May Lead To Additional Schedule Slippage and Higher
Costs
Page 9 GAO- 01- 450 Defense Acquisition
acquiring test aircraft would compress the majority of Comanche?s flighttest
schedule into the last 3 years of development. The compressed flighttest
schedule would, in turn, shorten the available time for completing all test
events and taking necessary corrective actions before the full- rate
production decision.
Since our last report, the first pre- production aircraft to be used for
development testing is now scheduled for delivery in January 2004, adding an
additional 3- month delay to the 19- month delay we reported in August 1999.
As shown in figure 1 below, the delivery of pre- production Comanche
aircraft has been delayed and, because the Army has retained the December
2006 full- rate production decision, the time available for testing,
assessing, and correcting problems has been reduced.
Figure 1: Comparison of Comanche Pre- Production Aircraft Deliveries
Many critical test events are now scheduled late in the development stages-
during the low- rate initial production phase of the program- and, as shown
in figure 2, many developmental and operational test events are scheduled to
be conducted concurrently.
Page 10 GAO- 01- 450 Defense Acquisition
Figure 2: Comanche Concurrent Flight Testing and Production
The combination of compressing the development schedule and undertaking
developmental and operational testing activities concurrently leaves the
Army with little room to accommodate any delays that may result from
assessing, correcting, and retesting problems found during testing. In
Comanche?s case, several critical subsystems- to be included in the mission
equipment package- may not be available until the development flight-
testing is well underway. These subsystems are very complex, state- of- the-
art systems that have not been demonstrated on a helicopter platform like
Comanche. As testing proceeds, any problems found will need to be analyzed,
fixed, and retested. However, with the ambitious test schedule, there may
not be time available between test events to correct problems and prepare
properly for the next event.
Further, the Army?s schedule for developing and testing software for the
Comanche may not be completed prior to the full- rate production decision.
The contractor is experiencing a shortage of software engineers available to
work on the Comanche contract. In addition, only about 1. 4 million of the
projected 1.9 million lines of computer code for the
Page 11 GAO- 01- 450 Defense Acquisition
Comanche?s mission equipment package will be completed by the time the
package is to be tested on the initial pre- production aircraft. Additional
segments of computer code for the mission equipment package will be
introduced as developmental testing is underway. At this point, it is
uncertain if all of the computer code for the full mission equipment package
will be completed by the time the Army is scheduled to make a full- rate
production decision for Comanche in late 2006.
Finally, the Army plans to use pre- production aircraft that it considers
production- representative for operational flight- testing. Before this
testing is complete, the Army plans to begin producing a total of 84 low-
rate initial production aircraft. These aircraft are to be used to equip
Army helicopter units and to ramp- up production. To produce that many
aircraft during low- rate initial production, the Army will have to ramp- up
its production capabilities rapidly and at a time when the aircraft design
is still evolving as new subsystems are introduced and test results are
evaluated. Specifically, the Army does not plan to freeze Comanche?s design
configuration until January 2006, or six months after the low- rate initial
production decision point. Making design changes and retrofits to a large
number of aircraft already produced could be costly.
In our last report, we noted that the Army was making modifications to the
Comanche that would adversely impact some of the Comanche?s planned
performance capabilities; for example, some modifications have added weight
and drag to the aircraft. While their exact impacts are still unknown, these
changes increase the risk that the Comanche?s planned performance goals may
not be achieved. The Comanche continues to have several areas of high
technical risk that jeopardize the achievement of several critical
performance requirements. Performance Degradation
Is Still A Risk
Page 12 GAO- 01- 450 Defense Acquisition
The Comanche?s ability to climb at a rate of 500 feet per minute is a key
performance requirement for the aircraft. 9 Since we last reported on the
Comanche program, the aircraft?s projected empty weight 10 has increased by
653 pounds- from 8, 822 pounds to 9, 475 pounds. At the current projected
design weight of 9, 475 pounds, the Comanche program office has acknowledged
that the helicopter cannot achieve the required vertical rate of climb of
500 feet per minute without increasing the horsepower of the current engine.
Consequently, the program office has assessed its achievement of the weight
requirement as high risk.
The Army offered its prime contractor for Comanche?s development, Boeing-
Sirkosky, an award fee of $1. 4 million to reduce its projected weight to 9,
250 pounds. However, the contractor did not achieve the first iteration of
weight reduction in December 2000. The program office is considering
increasing the incentive fee to $5 million for the contractor to reduce the
projected weight to 9,300 pounds in December 2001.
The program office believes that it can achieve its vertical rate of climb,
even with the increase in Comanche?s weight, by increasing the horsepower of
Comanche?s T- 801 engine from its current horsepower rating of 1131 to 1201.
The program office estimates that the increase in the engine?s power can be
obtained at a cost of about $13 million, and this approach will be less
costly than other weight reduction efforts. However, an increase in engine
performance could adversely affect the expected life of the engine since it
will have to perform about 47 degrees hotter than is normally required.
According to the program office, this increased performance may not have an
appreciable impact on the engine?s life.
As noted earlier in this report, the successful development and integration
of the mission equipment package is critical to meeting Comanche?s
performance requirements. This package includes an integrated communications
system, piloting system, target acquisition system, navigation system,
helmet- mounted display, survivability and early
9 A key performance requirement is a capability or characteristic that DOD
believes is so central to the Comanche?s performance that failing to meet
its threshold can be cause for the concept or system selection to be
reevaluated or the program to be reassessed or terminated. Other key
performance parameters include (1) night target acquisition range, (2) radar
cross signature, (3) infrared engine exhaust signature, and (4) digitally
communications with joint and combined armed forces. 10 Comanche?s empty
weight is the weight of the aircraft and it component parts. It does not
include the pilot?s weight, gear, fuel, radar kits, or expendable munitions.
Weight Growth Could
Adversely Impact Vertical Rate of Climb
Scheduled Integration of Critical Mission Equipment Package Still High Risk
Page 13 GAO- 01- 450 Defense Acquisition
warning equipment, mission computer, and weapons management system. The
program office has assessed the achievement of this portion of its
development effort as high risk. 11
In order to reduce this risk, the Army had planned to develop a mobile
integration laboratory, called a hotbench, which simulates Comanche?s
hardware, to integrate and test mission equipment package software before
installing the software on the flight test aircraft. However, due to a
shortage of development funds, the Army had listed the hotbench as an
unfunded requirement. DOD recently provided additional funding to the
Comanche program, which the program office plans to use to fully fund the
hotbench. Despite the additional funding for the hotbench, the program
office continues to acknowledge that integration of Comanche?s mission
equipment package as an area of high technical risk.
A critical Comanche requirement is an on- board fault detection system that
can rapidly and accurately provide information about equipment problems.
With an on- board fault isolation system, the Army would be able to promptly
identify and correct potential problems in advance, according to the
Comanche?s operational requirements document. Additionally, without the
system, the time and cost of maintaining the aircraft will likely increase.
According to the Army, this system needs to be 75 to 95 percent accurate- 75
percent for mechanical and electrical equipment and 95 percent for avionics
and electronics equipment. The Comanche program office has concluded that
this requirement will be difficult to achieve within the current cost,
weight, and packaging constraints, and does not expect to achieve a mature
fault detection and fault isolation capability until 2 years after initial
fielding. 12 According to the program office, this system depends, in part,
on a database built on flight data and equipment failure experience;
therefore, the system becomes better with additional flight hours. The
program office anticipates that after 2 years of flight testing, the system
should meet the full level of predictability required. Although some of the
recently provided development funding will be used by the Army in this area,
the Comanche program has identified an
11 Comanche system integration involves the integration of its weapon
systems and battlefield information into a total weapon system that is
expected to provide maximum effectiveness with minimum crew workload. 12
This approach would entail using previously obtained developmental
diagnostic analyses and models to support the initial operational testing
and evaluation test events. Afterward, the Army plans to define system
changes that are needed to obtain required system performance. Fully Capable
On- Board
Fault Isolation Requirement May Not Be Achieved Until 2 Years After Initial
Fielding
Page 14 GAO- 01- 450 Defense Acquisition
additional $20 million unfunded requirement for the fault isolation
capability.
In some battle situations, the Army plans to use Comanche as a deep
reconnaissance aircraft to provide critical information and situational
awareness to joint forces. Satellite communication technology is necessary
for the helicopter to be able to achieve the ?beyond- line- ofsight?
capability needed to carry out this function, according to the Comanche
operational requirement document. To meet this need, the Army was planning
to rely on satellite communication technology being developed and
miniaturized as part of the Joint Strike Fighter program, which is being
developed jointly by the Air Force, Navy, and Marines. However, in May 2000,
Congress provided that the Joint Strike Fighter program could not enter into
the engineering and manufacturing development phase until the Secretary of
Defense certified the technological maturity of its critical technologies.
13 This has delayed the Joint Strike Fighter program?s schedule for
beginning its engineering and manufacturing development phase.
When assessing the risk of its dependency on the Joint Strike Fighter?s
program, the Comanche program office concluded that the helicopters in low-
rate initial production would not have the beyond- line- of- sight
communication capability if the Joint Strike Fighter program was delayed.
The program office now believes that it must develop its own satellite
communication capability. However, the development schedule remains high-
risk for the timely inclusion of this capability on the initially fielded
Comanche helicopters. The Army has estimated that it will require about $58
million to develop this capability and plans to fully fund this effort with
additional funds recently provided by DOD.
13 Floyd D. Spence National Defense Authorization Act for Fiscal Year 2001,
section 212, PL 106- 239, October 12, 2000. Critical Comanche
Beyond- Line- of- Sight Communications Requirements May Not Be Achievable
Page 15 GAO- 01- 450 Defense Acquisition
Our work on best practices has found that product development in successful
commercial firms is a clearly defined undertaking for which firms insist on
having in hand the technology that meets customers? needs before starting.
The firms demand- and receive- specific knowledge about a new product before
production begins. And, they do not go forward unless a strong business case
on which the program was originally justified continues to hold true. Such a
knowledge- based process is essential to commercial firms getting better
cost, schedule, and performance outcomes. It enables decision- makers to be
reasonably certain about critical facets of the product under development
when they need it.
At the point of going into production, successful firms will already know
that (1) technologies match customer requirements, that is, they can fit
onto a product and function as expected, (2) the product?s design meets
performance requirements, and (3) the product can be produced within cost,
schedule, and quality targets. The Comanche program does not yet have this
knowledge and is not likely to have this knowledge when it plans to begin
low- rate initial production in June 2005.
First, the Army does not yet know and it will not know until well after its
low- rate initial production decision whether certain technologies being
developed will fit on the helicopter and function as expected. Our report 14
on incorporating new technologies into programs indicated that demonstrating
a high level of maturity before new technologies are incorporated into
product development programs puts those programs into a better position to
succeed. Further, technologies that were included in a product development
before they were mature later contributed to cost increases and schedule
delays to those products. While the Comanche program has made progress in
the technology readiness level of its critical components, integration of
those components into subsystems, such as the mission equipment package, and
the helicopter as a whole remains high- risk. In addition, the integration,
development, and configuration of key satellite communication technology for
inclusion in the integrated communication, navigation, and identification
avionics has also been assessed as high risk. Finally, some of the
technologies have not been developed to meet Comanche?s specific
configuration requirements. For instance, the Comanche?s second generation
forward- looking infrared
14 Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes (GAO/ NSIAD- 99- 162, July 30, 1999). The Army Plans
to
Begin Comanche Production Despite Numerous Uncertainties
Page 16 GAO- 01- 450 Defense Acquisition
sensor has been tested and proven on the Black Hawk helicopter by the Army?s
night vision laboratory but not on the Comanche itself. Such testing needs
to be done to ensure that the system can work together with other unique
systems being developed for the Comanche, including the piloting, target
acquisition, and navigation systems, which work as one unit. Comanche?s
contractor has maintained that its mission equipment package technology is
challenging because some key components have not been developed and
configured in the required manner for the helicopter?s intended mission.
Second, as discussed earlier, the Army does not yet know and may not know
until well after the start of low- rate initial production, whether
performance requirements can be met- including vertical rate of climb, on-
board fault isolation, and beyond- line- of- sight communication
requirements. The Army plans to conduct a limited user test before it begins
low- rate initial production but it is a rudimentary test and not a complete
operational test that fully demonstrates the aircraft?s capabilities. By
compressing many key events late in the development schedule and conducting
developmental and operational testing activities concurrently, the Army is
running the risk of not fully demonstrating many of its critical
capabilities before its planned full- rate production decision. Under
current plans, for example, the Army will not complete a full demonstration
of its integrated mission equipment package until December 2006- over a full
year after its low- rate initial production decision and within the same
month that the Army plans to make its decision on Comanche full- rate
production.
Third, as noted earlier, it is still uncertain whether the Comanche can be
developed within cost and scheduling estimates. Although additional costs
have been identified for the Comanche since it was last restructured, the
full development cost will not be known until critical technology is fully
developed, integrated, and tested. This will not occur until well after a
lowrate initial production decision has been made in June 2005. The program
office believes that it will know the cost of the initial production
aircraft, which will have been negotiated prior to the low- rate initial
production decision. However, at that time, the program office and the
contractor will have limited experience and data relative to producing the
fully developed Comanche helicopter. Until more experience and data is
available, there is not a high level of confidence in the Army?s production
cost estimate.
Further, the Director of Operational Test and Evaluation in assessing the
results of the Comanche milestone II test data indicated that it is highly
unlikely that the Army can deliver the expected system performance
Page 17 GAO- 01- 450 Defense Acquisition
within the current budget and schedule. The Director's assessment revealed
that, without an operational assessment of an integrated system, it is
difficult to predict with any degree of confidence whether (1) the
individual subsystems can be successfully integrated, (2) the subsystems
will function properly in an operational environment, or (3) the subsystems,
in concert, will provide the anticipated benefits in operational
performance.
In 1999, we reported that the Army started the Comanche?s program
development too early in terms of technology readiness, which is contrary to
best commercial practices. Further, in approving the program for engineering
and manufacturing development, the Army accelerated the development of some
components, reduced the number of test aircraft, and compressed the test
schedule. Two years later, the program is confronted with rising development
costs, a compressed development schedule, and several major areas of high
technical risk. The Army plans to proceed to low- rate initial production in
June 2005 and full- rate production in December 2006, both of which could be
well in advance of attaining sufficient knowledge of the helicopter?s
technical maturity, demonstrated performance capabilities, and production
costs. With such a scenario, the potential for adverse program outcomes is
high- higher than expected costs, longer than expected schedules, and
uncertain performance. DOD and Army officials acknowledge that the current
program cost and schedule objectives are not achievable and should be
changed to reflect more realistic objectives, but they believe that the
planned January 2003 review for the Comanche program is the appropriate time
to address such changes. Such a delay in revising the program?s cost and
schedule estimate limits the visibility and knowledge that Army and DOD
management as well as the Congress needs to (1) provide program oversight
and direction; (2) make effective cost, schedule, and performance trade- off
decisions; and (3) assess affordability and annual funding requirements.
To improve management oversight and direction and achieve more favorable
program outcomes, this report recommends that the Secretary of the Army
reassess the program?s cost, schedule, and performance objectives, and
revise those objectives to more achievable levels prior to submitting its
next fiscal year budget.
In commenting on a draft of this report, DOD partially concurred with our
recommendation. DOD noted that it agrees with some of our concerns and
recognizes there are risks in the currently planned Comanche engineering and
manufacturing development program. DOD noted that Conclusion
Recommendations for Executive Action
Agency Comments and Our Evaluation
Page 18 GAO- 01- 450 Defense Acquisition
these risks were understood during the Comanche milestone II review. At that
time, the Defense Acquisition Executive directed that the program proceed as
planned, but that interim decision reviews be conducted in January 2003 and
June 2005 to review program status. DOD stated that these reviews, along
with other major program review and oversight processes, will permit
successful management of program risks. Nevertheless, DOD stated that it is
currently examining whether any of Comanche's requirements should be
deferred, in order to reduce the risk of not meeting cost and schedule
objectives. DOD's examination of Comanche's requirements is consistent with
our recommendation. We continue to believe that DOD should report on the
results of this examination and any revisions to the program?s objectives to
the defense committees of the Congress with its next budget request.
DOD disagreed with a reference to our previous Comanche report stating that
current program risks are caused by, among other things, the program being
allowed to enter engineering and manufacturing development prior to
maturation of key technologies. DOD maintains that the Comanche program
successfully demonstrated its exit criteria prior to entering engineering
and manufacturing development. However, the Comanche program?s demonstration
of its exit criteria was not sufficient as a basis to move forward in the
acquisition process. For example, the exit criteria did not require that the
technologies used in Comanche be at or above specific levels of demonstrated
readiness. As we previously reported, the Army's own assessments clearly
indicated that several key areas of technology were not at those levels
called for in commercial best practices guidelines.
DOD's comments are reprinted in appendix I. Other comments provided by DOD
were incorporated in the report as appropriate.
To evaluate changes in the Comanche?s status with regard to cost, schedule,
and performance and assess whether the Army has the certainty it needs to
proceed with beginning production, we examined and compared program
schedules, pertinent cost documents, and acquisition strategies, and
discussed potential changes and causative factors with cognizant Comanche
program officials. We analyzed flight- test plans, schedules, and reports
and discussed significant issues with program officials. We reviewed program
documents related to risk and analyzed program risks and development
problems by comparing them with various test schedules and plans. To assess
performance capabilities before beginning with production, we analyzed
required and projected performance and compared it with the Comanche?s
operational requirements. We relied on previous GAO best practices work to
examine Comanche?s technological readiness levels for key program
technologies. Scope and
Methodology
Page 19 GAO- 01- 450 Defense Acquisition
Our analyses focused on the impact of Comanche?s cost, schedule, and
performance on the Army?s ability to field a Comanche helicopter that would
meet its requirements and incorporate technological upgrades in its
helicopter fleet.
In performing our work, we obtained pertinent program documents and
interviewed officials from the offices of the Secretary of Defense and the
Army, Washington, D. C.; the Program Executive Office- Aviation and Comanche
Program Office, Redstone Arsenal, Alabama; the U. S. Army Training and
Doctrine Command, Fort Rucker, Alabama; the Comanche Joint Project Office,
Huntsville, Alabama; and the Aviation Test and Evaluation Command,
Alexandria, Virginia. We conducted our review from September 2000 through
March 2001 in accordance with generally accepted government auditing
standards.
As agreed with your office, unless you publicly announce the contents of
this report earlier, we will not distribute this report until 5 days from
its date. At that time, we will send copies of this report to the Honorable
Donald H. Rumsfeld, Secretary of Defense; the Honorable Thomas White,
Secretary of the Army; Director, Office of Management and Budget; and other
interested congressional committees and parties. We will also make copies
available to others upon request.
If you have any questions regarding this report, please contact me on (202)
512- 4530. GAO contacts and major contributors to this report are listed in
appendix II.
Sincerely, James F. Wiggins Director, Acquisition and Sourcing Management
Team
Appendix I: Comments From the Department of Defense
Page 20 GAO- 01- 450 Defense Acquisition
Appendix I: Comments From the Department of Defense
Appendix I: Comments From the Department of Defense
Page 21 GAO- 01- 450 Defense Acquisition
Appendix II: GAO Contacts and Acknowledgments
Page 22 GAO- 01- 450 Defense Acquisition
James F. Wiggins (202) 512- 4841 William Graveline (256) 650- 1414
In addition to those named above, Leon S. Gill, Wendy Smythe, Gary
Middleton, and Cristina Chaplain made key contributions to this report.
Appendix II: GAO Contacts and
Acknowledgments GAO Contacts Acknowledgments
(707564)
The first copy of each GAO report is free. Additional copies of reports are
$2 each. A check or money order should be made out to the Superintendent of
Documents. VISA and MasterCard credit cards are also accepted.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail:
U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
Orders by visiting:
Room 1100 700 4 th St., NW (corner of 4 th and G Sts. NW) Washington, DC
20013
Orders by phone:
(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet
For information on how to access GAO reports on the Internet, send an email
message with ?info? in the body to:
Info@ www. gao. gov or visit GAO?s World Wide Web home page at: http:// www.
gao. gov
Contact one:
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
E- mail: fraudnet@ gao. gov
1- 800- 424- 5454 (automated answering system) Ordering Information
To Report Fraud, Waste, and Abuse in Federal Programs
*** End of document. ***
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
