Joint Strike Fighter Acquisition: Development Schedule Should Be Changed to Reduce Risks (Testimony, 03/16/2000, GAO/T-NSIAD-00-132)
Pursuant to a congressional request, GAO discussed the results of its
review of the Joint Strike Fighter (JSF) aircraft program, focusing on
whether the acquisition strategy: (1) is designed to demonstrate to a
low level of technical risk those critical technologies, processes, and
system characteristics necessary to produce an affordable family of
strike aircraft that meets all participants' needs before entering
engineering and manufacturing development; and (2) is being implemented
in a manner that will ensure that this objective will be achieved.
GAO noted that: (1) the Joint Strike Fighter acquisition strategy is
designed to meet affordability goals by reducing program risk before
proceeding into the engineering and manufacturing development phase; (2)
the acquisition strategy is designed to ensure a better match between
the maturity of key technologies and the aircraft's requirements; (3)
matching the requirements and the maturity of technology when a program
enters engineering and manufacturing development (development phase) is
a critical determinant of a program's success; (4) once the development
phase begins, a large, fixed investment in the form of human capital,
facilities, and materials is sunk into the program and any significant
changes will have a large, rippling effect on schedule and cost; (5)
beginning the development phase when critical technologies are at a low
level of maturity serves to significantly increase program risk and the
likelihood of schedule delays, which in turn result in increased program
costs; (6) while GAO was encouraged by the design of the Joint Strike
Fighter acquisition strategy, GAO has some concerns about its
implementation; (7) GAO's biggest concern is that critical technologies
are projected to be at low levels of technical maturity when the
engineering and manufacturing development contract is scheduled to be
awarded; (8) in addition, when the competing contractors experienced
design problems and cost overruns, the Department of Defense (DOD)
restructured the program in a manner that will provide less information
than originally planned prior to selecting between the two competing
contractor proposals; and (9) GAO believes that to demonstrate DOD's
commitment to acquisition reform, follow best commercial practices, and
reduce the risk of future cost growth, the Joint Strike Fighter program
office should continue to focus on risk-reduction efforts by maturing
critical technologies prior to entering engineering and manufacturing
development, and be allowed to do so without the penalty of withdrawal
of funding support.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: T-NSIAD-00-132
TITLE: Joint Strike Fighter Acquisition: Development Schedule
Should Be Changed to Reduce Risks
DATE: 03/16/2000
SUBJECT: Weapons research and development
Fighter aircraft
Federal procurement policy
Defense procurement
Defense cost control
Defense capabilities
Operational testing
IDENTIFIER: Joint Strike Fighter
DOD Joint Advanced Strike Aircraft Technology Program
Joint Direct Attack Munition
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO/T-NSIAD-00-132
Testimony
Before the Subcommittees on Military Procurement and on Military Research
and Development, Committee on Armed Services, House of Representatives
For Release on Delivery
Expected at
1:00 p.m.
Thursday, March 16, 2000
JOINT STRIKE
FIGHTER ACQUISITION
Development Schedule
Should Be Changed to
Reduce Risks
Statement of Louis J. Rodrigues, Director, Defense Acquisitions Issues,
National Security and International Affairs Division
GAO/NSIAD-00-132
Mr. Chairman and Members of the Subcommittees:
I am pleased to be here today to discuss the results of our review of the
Joint Strike Fighter (JSF) aircraft program. My testimony today summarizes
our draft report on the JSF, which was prepared at the request of the
Chairman, Subcommittee on National Security, Veterans Affairs, and
International Relations, House Committee on Government Reform. The
Department of Defense (DOD) is scheduled to provide its comments on our
draft by the end of next week. Therefore, my remarks today do not include
the Department's views on our report.
DOD has designated the Joint Strike Fighter program as a flagship program
for acquisition reform. According to the Joint Strike Fighter Single
Acquisition Management Plan, a principal objective of the current concept
demonstration phase is to demonstrate to a low level of technical risk those
critical technologies, processes, and system characteristics necessary to
produce an affordable family of strike aircraft that meets all participants'
needs before entering engineering and manufacturing development. Our review
focused on whether the acquisition strategy (1) is designed to accomplish
this objective and (2) is being implemented in a manner that will ensure
that the objective will be achieved.
SUMMARY
The Joint Strike Fighter acquisition strategy is designed to meet
affordability goals by reducing program risk before proceeding into the
engineering and manufacturing development phase. To that end, the
acquisition strategy is designed to ensure a better match between the
maturity of key technologies and the aircraft's requirements. Matching the
requirements and the maturity of technology when a program enters
engineering and manufacturing development (development phase) is a critical
determinant of a program's success. Once the development phase begins, a
large, fixed investment in the form of human capital, facilities, and
materials is sunk into the program and any significant changes will have a
large, rippling effect on schedule and cost. Beginning the development phase
when critical technologies are at a low level of maturity serves to
significantly increase program risk and the likelihood of schedule delays,
which in turn result in increased program costs.
While we were encouraged by the design of the Joint Strike Fighter
acquisition strategy, we have some concerns about its implementation. Our
biggest concern is that critical technologies are projected to be at low
levels of technical maturity when the engineering and manufacturing
development contract is scheduled to be awarded. In addition, when the
competing contractors experienced design problems and cost overruns, DOD
restructured the program in a manner that will provide less information than
originally planned prior to selecting between the two competing contractor
proposals.
We believe that to demonstrate DOD's commitment to acquisition reform,
follow best commercial practices, and reduce the risk of future cost growth,
the Joint Strike Fighter program office should continue to focus on
risk-reduction efforts by maturing critical technologies prior to entering
engineering and manufacturing development, and be allowed to do so without
the penalty of withdrawal of funding support.
I would like now to further discuss the factors that led us to our
recommendation that the JSF program office adjust its currently planned
engineering and manufacturing development decision date of March 2001.
BACKGROUND
As currently planned, the JSF program will cost about $200 billion to
develop and procure over 3,000 aircraft and related support equipment for
the U.S. Air Force, Marine Corps, and Navy and for Great Britain. To date,
the program has awarded contracts totaling over $2 billion to Boeing and
Lockheed Martin for the current concept demonstration phase. DOD is
scheduled to award the contract for engineering and manufacturing
development to either Boeing or Lockheed Martin in March 2001. During
engineering and manufacturing development, the Joint Strike Fighter will be
fully developed, engineered, designed, fabricated, tested, and evaluated to
demonstrate that the production aircraft will meet stated requirements.
The Air Force expects its JSF variant will cost about $28 million per unit;
the Navy variant is estimated to cost between $31 million and $38 million;
and the Marine Corps variant will cost between $30 million and $35 million.
Other independent estimates are not so optimistic. For example, in
congressional hearings held in March 1999, the Congressional Budget Office
estimated that the unit cost of the Joint Strike Fighter could be as much as
47 to 51 percent higher than expected, depending on which variant was being
procured. DOD and the Congressional Budget Office estimates vary as a result
of differing estimating techniques, including estimating the cost of
incorporating stealth technologies into the Joint Strike Fighter design.
JSF ACQUISITION STRATEGY DESIGNED TO
REDUCE TECHNICAL RISK
The Joint Strike Fighter acquisition strategy is designed to lower technical
risk through aircraft flight demonstrations and advanced technology
development prior to awarding the engineering and manufacturing development
contract.
During the current concept demonstration phase, DOD requires each contractor
to design and build two aircraft--one aircraft for conventional takeoff and
landing and one for short takeoff and vertical landing--to demonstrate the
following:
* commonality/modularity to validate the contractors' ability to produce
three aircraft variants on the same production line;
* the aircrafts' ability to do a short takeoff and vertical landing,
hover, and transition to forward flight; and
* satisfactory low airspeed, carrier approach flying and handling
qualities.
Each contractor will also be required to submit a Preferred Weapon System
Concept, which outlines its final design concept for developing an
affordable JSF aircraft to meet the goals specified in the final
requirements document. The Preferred Weapon System Concept will include
results from the flight and ground demonstrations and will ultimately be
used by DOD to select the winning aircraft design and to award the
engineering and manufacturing development contract.
We are encouraged by the design of the acquisition strategy and its focus on
risk reduction by maturing critical technologies before entering the
engineering and manufacturing development phase. Once in a development
environment, external pressures to keep the program moving become dominant,
such as preserving cost and schedule estimates to secure budget approval.
For example, DOD policies require that a program be funded in the current
year and that funds be made available over the next 6 years in the DOD
planning cycle. If a program manager decided that an additional year was
needed to reach the desired level of technical maturity during the risk
reduction/concept demonstration phase, the planned start of the engineering
and manufacturing development phase could be delayed. This delay could
jeopardize the funding for the development phase, thus risking the funding
support for the entire program. Consequently, the program manager may be
more likely to accept the risk of moving forward with a lower level of
technology rather than risk losing the program. That decision would raise
cost/benefit issues because cost increases and performance compromises would
likely occur.
IMPLEMENTATION OF JSF ACQUISITION STRATEGY WILL
NOT ENSURE THAT PROGRAM OBJECTIVE IS ACHIEVED
Contrary to a principal objective of its acquisition strategy, the Joint
Strike Fighter program will not enter the engineering and manufacturing
development phase with low technical risk. In addition, when the competing
contractors experienced design problems and cost overruns, DOD restructured
the program in a manner that is moving away from best commercial practices
that were evident in the original strategy toward traditional practices that
have caused problems on other programs.
Critical Technologies Not Developed
to Acceptable Levels
The aircraft being produced during the concept demonstration phase are not
intended to demonstrate many of the technologies considered critical for
achieving JSF program cost and performance requirements, such as those for
integrated avionics. Instead, many of these technologies will be
demonstrated only in laboratory or ground testing environments and therefore
will have low levels of technical maturity when the engineering and
manufacturing development contract is scheduled to be awarded.
To determine the maturity of Joint Strike Fighter technologies, we requested
the program office to identify the technology areas they considered critical
to meeting JSF cost and/or requirement objectives. The program office
identified eight critical technology areas encompassing avionics; flight
systems; manufacturing and producibility; propulsion; supportability, and
weapons delivery system. We then requested the program office and the two
competing contractors to assign maturity levels for these critical
technologies using a tool referred to as technology readiness levels (TRLs).
Using this tool, technology maturity levels are measured on a scale of one
to nine: studies of the basic concept have a readiness level of one,
laboratory demonstrations have a readiness level between three and six, and
technologies that have been proven through integration and operation on the
intended product have a readiness level of nine. Without going into the
details of each level, let me note that a level four equates to a laboratory
demonstration of a technology that is not in its usable form. Imagine, if
you will, an advanced radio technology that can be demonstrated with
components that take up a table top. A level seven is the demonstration of a
technology that approximates its final form and occurs in an environment
outside the laboratory. The same radio at level seven would be installed and
demonstrated in an aircraft similar to the JSF.
The lower the level of maturity when a technology is included in a
development program, the higher the risk that it will cause problems.
According to the people in DOD that use the TRLs in rating the maturity
levels of technologies, level seven enables a technology to be included in a
development program with acceptable risk. With this measurement tool, the
program office can gauge the likely consequences of placing various
technologies at a given maturity level into a development program and make
informed choices and trade-offs if necessary to meet program goals.
The TRLs were pioneered by the National Aeronautics and Space Administration
and adopted by the Air Force Research Laboratory to determine the readiness
of technologies to be incorporated into a weapon or other type of system.
The Joint Advanced Strike Technology program--from which the JSF program
evolved--made extensive use of TRLs to assess early maturity levels for many
of the current JSF technologies. The program also identified TRL 7 as the
acceptable readiness level for low-risk transition into the engineering and
manufacturing development phase.
We also used TRLs in our prior work when, at the request of the Chairman of
the Senate Armed Services Committee's Subcommittee on Readiness and
Management Support, we assessed the impact of technology maturity on product
outcomes. During that work, we reviewed commercial and DOD experiences in
incorporating 23 different technologies into new product and weapon system
designs. Table 1 shows that cost and schedule problems arose when programs
started when technologies were at low readiness levels and it conversely
shows that programs met product objectives when the technologies were at
higher levels of readiness when the programs were started.
Table 1: Cost and Schedule Experiences on Product Developments
Product Development
TRL at
Product and associated technologies Cost Growth Schedule
program Slippage
start
Comanche helicopter
5
Engine
5
Rotor
3 101 percent 120 percent
Forward looking infrared
3
Helmet mounted display
3
Integrated avionics
Brilliant Anti-Armor Submunition
2
Acoustic sensor
3
Infrared seeker
3 88 percent 62 percent
Warhead
3
Inertial measurement unit
3
Data processors
Hughes HS-702 satellite
6 None None
Solar cell array
Ford Jaguar
8
Adaptive cruise control None None
8
Voice activated controls
Note: The Comanche has experienced a great deal of cost growth and schedule
slippage for many reasons, of which technology immaturity is only one. Other
factors, such as changing the scope, funding, and pace of the program for
affordability reasons, have also contributed to cost increases.
Unlike the Comanche and Brilliant Anti-Armor Submunition programs discussed
in table 1, other DOD programs have more attentively matched customer
requirements with technological capabilities. For example, the Joint Direct
Attack Munition program used modified variants of proven product lines for
its guidance component and global positioning system. It also used mature,
existing components from other proven manufacturing processes for its own
system for controlling tail fin movements. The designs for the battery and
the tail housing both used mature technology and were built using mostly
existing tooling and processes. The Deputy Under Secretary of Defense
(Acquisition and Reform) attested to the success of the Joint Direct Attack
Munition program during a hearing today before the Government Reform
Committee. The Deputy Under Secretary testified that the Joint Direct Attack
Munition performed flawlessly in Kosovo and was purchased for less than half
of its expected unit cost.
Regarding the Joint Strike Fighter, in conjunction with the program office
and the two competing contractors, we determined the readiness levels of
critical technologies at two points in time: when the JSF program was
started in 1996 and when the program is scheduled to enter engineering and
manufacturing development in March 2001. Those assessments showed that when
the JSF program was started, most of the critical technologies were well
below TRL 6, which is the level considered acceptable risk to begin a
program by the Air Force Research Laboratory. In terms of engineering and
manufacturing development, none of the critical technology areas are
projected to be at readiness level 7, which the Air Force Research
Laboratory considers acceptable risk for entry into engineering and
manufacturing development. The technology readiness levels of the eight
critical JSF technology areas are shown in table 2.
Table 2: JSF Critical Technology Readiness Levels at Program Start and as
Projected for Entry into Engineering and Manufacturing Development
EMBED PowerPoint.Slide.8 As shown in table 2, all of the critical technology areas are expected to be
at maturity levels lower than considered acceptable risk for entry into
engineering and manufacturing development (readiness level 7) and six of the
technologies will still be below the level of maturity (readiness level 6)
which is considered acceptable risk for program start, which occurred over 3
years ago for the JSF program.
Should any of these technologies be delayed or, worse still, not available
for incorporation into the final JSF design, the impact on the program would
be dramatic. For example, if one of the above critical technologies needed
to be replaced with its planned backup, DOD could expect an increase of
several billion dollars in production and operation and support costs. The
backup technology would also significantly increase aircraft weight, which
can negatively impact aircraft performance. This technology is expected to
be TRL 5 at the beginning of the engineering and manufacturing development
phase, which indicates that substantial technology development must still
occur during this phase.
DOD RESPONSE TO CONTRACTOR COST AND SCHEDULE PROBLEMS IS
INCONSISTENT WITH ACQUISITION STRATEGY GOALS
As a result of cost growth and schedule concerns related to both competing
contractors, DOD restructured the JSF program in May 1999. Under the program
restructuring, the contractors will provide DOD with less information than
originally planned prior to submission of their proposals. For example, the
JSF flight test program has been decreased, which will reduce the data
available for final proposal evaluation. JSF program officials stated that
with these flight test reductions, only the minimum acceptable flight
quality demonstrations are expected. The number of preferred weapon system
design updates has also been reduced, which means that information submitted
for evaluation will be less representative of the final JSF design. Finally,
DOD has eliminated risk-reduction efforts and delayed other technology
demonstrations, which will increase the program's technical risk as it
transitions into engineering and manufacturing development.
CONCLUSIONS
A key objective of the Joint Strike Fighter acquisition strategy --entering
into engineering and manufacturing development with low technical risk--will
not be achieved due to the manner in which the JSF program office is
implementing the acquisition strategy. On its current schedule, the program
will enter the engineering and manufacturing development phase without
having reduced to an acceptable level the technical risk of technologies
that the program office has identified as critical to meeting the program's
cost and performance objectives. This approach is not consistent with best
commercial practices in which technologies are more fully developed before
proceeding into product development. It is also not consistent with DOD's
originally planned approach for developing the Joint Strike Fighter.
Instead, the program office's revised approach is consistent with DOD's
traditional approach in weapons system programs of concurrently developing
technologies and products. This traditional approach has often raised
cost/benefit issues as a result of cost increases, schedule delays, and
compromised performance as problems arose in completing technology
development.
RECOMMENDATION
The Joint Strike Fighter program is at an early development stage and,
therefore, DOD still has the opportunity to both demonstrate its commitment
to acquisition reform and chart a course to avoid the problems that often
befall major weapon systems. A decision to allow the JSF to proceed as
planned, without mature critical technologies, would compromise DOD's
position on acquisition reform, not follow best commercial practices, and
would perpetuate conditions that have led to cost growth and schedule delays
in many prior DOD weapon system acquisition programs. Accordingly, in our
draft report we recommend that the Secretary of Defense direct the JSF
program office to adjust the currently planned March 2001 engineering and
manufacturing development decision date, without the penalty of withdrawal
of funding support, to allow adequate time to mature critical technologies
to acceptable maturity levels, thereby closing the gap between technology
and requirements, before awarding the engineering and manufacturing
development contract.
- - - -
Mr. Chairman, that concludes my statement. I will be happy to respond to any
questions you or other Members of the Subcommittees might have.
(707495)
ORDERS BY INTERNET
For information on how to access GAO reports on the INTERNET, send an e-mail
message with "info" in the body to
HYPERLINK mailto:Info@www.gao.gov Info@www.gao.gov
or visit GAO's World Wide Web Home Page at
HYPERLINK http://www.gao.gov http://www.gao.gov
TO REPORT FRAUD, WASTE, AND ABUSE
IN FEDERAL PROGRAMS
Contact one:
* website: HYPERLINK http://www.gao.gov/fraudnet/fraudnet.htm
http://www.gao.gov/fraudnet/fraudnet.htm
* e-mail: HYPERLINK mailto:fraudnet@gao.gov fraudnet@gao.gov
* 1-800-424-5454 (automated answering system)
*** End of document. ***
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
