[Senate Hearing 111-149]
[From the U.S. Government Printing Office]
S. Hrg. 111-149
ACQUISITION OF MAJOR WEAPONS SYSTEMS BY THE DEPARTMENT OF DEFENSE AND
S. 454, THE WEAPON SYSTEMS ACQUISITION REFORM ACT OF 2009
=======================================================================
HEARING
before the
COMMITTEE ON ARMED SERVICES
UNITED STATES SENATE
ONE HUNDRED ELEVENTH CONGRESS
FIRST SESSION
__________
MARCH 3, 2009
__________
Printed for the use of the Committee on Armed Services
U.S. GOVERNMENT PRINTING OFFICE
53-267 WASHINGTON : 2009
-----------------------------------------------------------------------
For Sale by the Superintendent of Documents, U.S. Government Printing Office
Internet: bookstore.gpo.gov Phone: toll free (866) 512-1800; (202) 512�091800
Fax: (202) 512�092104 Mail: Stop IDCC, Washington, DC 20402�090001
COMMITTEE ON ARMED SERVICES
CARL LEVIN, Michigan, Chairman
EDWARD M. KENNEDY, Massachusetts JOHN McCAIN, Arizona
ROBERT C. BYRD, West Virginia JAMES M. INHOFE, Oklahoma
JOSEPH I. LIEBERMAN, Connecticut JEFF SESSIONS, Alabama
JACK REED, Rhode Island SAXBY CHAMBLISS, Georgia
DANIEL K. AKAKA, Hawaii LINDSEY GRAHAM, South Carolina
BILL NELSON, Florida JOHN THUNE, South Dakota
E. BENJAMIN NELSON, Nebraska MEL MARTINEZ, Florida
EVAN BAYH, Indiana ROGER F. WICKER, Mississippi
JIM WEBB, Virginia RICHARD BURR, North Carolina
CLAIRE McCASKILL, Missouri DAVID VITTER, Louisiana
MARK UDALL, Colorado SUSAN M. COLLINS, Maine
KAY R. HAGAN, North Carolina
MARK BEGICH, Alaska
ROLAND W. BURRIS, Illinois
Richard D. DeBobes, Staff Director
Joseph W. Bowab, Republican Staff Director
(ii)
?
C O N T E N T S
__________
CHRONOLOGICAL LIST OF WITNESSES
Acquisition of Major Weapons Systems by the Department of Defense and
S. 454, the Weapon Systems Acquisition Reform Act of 2009
march 3, 2009
Page
Sullivan, Michael J., Director, Acquisition and Sourcing
Management, Government Accountability Office................... 7
Gansler, Hon. Jacques S., Chairman, Defense Science Board Task
Force on Industrial Structure for Transformation............... 19
Kaminski, Hon. Paul G., Chair, Committee on Pre-Milestone A
Systems Engineering, Air Force Studies Board, National Research
Council........................................................ 25
Adolph, Charles E. (Pete), Chairman, Defense Science Board Task
Force on Developmental Test and Evaluation..................... 40
(iii)
ACQUISITION OF MAJOR WEAPONS SYSTEMS BY THE DEPARTMENT OF DEFENSE
AND S. 454, THE WEAPON SYSTEMS ACQUISITION REFORM ACT OF 2009
----------
TUESDAY, MARCH 3, 2009
U.S. Senate,
Committee on Armed Services,
Washington, DC.
The committee met, pursuant to notice, at 9:36 a.m. in room
SH-216, Hart Senate Office Building, Senator Carl Levin
(chairman) presiding.
Committee members present: Senators Levin, Lieberman,
Akaka, Bill Nelson, Webb, McCaskill, Udall, Hagan, Begich,
Burris, McCain, Chambliss, Thune, Martinez, Burr, and Collins.
Committee staff members present: Richard D. DeBobes, Staff
Director, and Leah C. Brewer, Nominations and Hearings Clerk.
Majority staff members present: Creighton Greene,
professional staff member; Peter K. Levine, general counsel;
John H. Quirk V, professional staff member; Arun A. Seraphin,
professional staff member; and William K. Sutey, professional
staff member.
Minority staff members present: Joseph W. Bowab, Republican
staff director; Daniel A. Lerner, professional staff member;
David M. Morriss, minority counsel; Lucian L. Niemeyer,
professional staff member; and Christopher J. Paul,
professional staff member.
Staff assistants present: Jessica L. Kingston, Brian F.
Sebold, and Breon N. Wells.
Committee members' assistants present: Jay Maroney,
assistant to Senator Kennedy; Bonni Berge, assistant to Senator
Akaka; Christopher Caple, assistant to Senator Bill Nelson; Jon
Davey, assistant to Senator Bayh; Gordon I. Peterson, assistant
to Senator Webb; Stephen C. Hedger, assistant to Senator
McCaskill; Jennifer Barrett, assistant to Senator Udall;
Michael Harney, assistant to Senator Hagan; David Ramseur,
assistant to Senator Begich; Brady King, assistant to Senator
Burris; Sandra Luff, assistant to Senator Sessions; Clyde A.
Taylor IV, assistant to Senator Chambliss; Jason Van Beek,
assistant to Senator Thune; Erskine W. Wells III, assistant to
Senator Wicker; and Kevin Kane, assistant to Senator Burr.
OPENING STATEMENT OF SENATOR CARL LEVIN, CHAIRMAN
Chairman Levin. Good morning, everybody. The committee
meets today to consider the performance of the Department of
Defense's (DOD) acquisition programs at a time when cost growth
on these programs has reached levels that we cannot afford,
including consideration of our bill, S. 454, the Weapon Systems
Acquisition Reform Act of 2009, which Senator McCain and I
recently introduced.
Since the beginning of 2006, nearly half of the DOD's
largest acquisition programs have exceeded the so-called Nunn-
McCurdy cost-growth standards established by Congress to
identify seriously troubled programs.
As Secretary Gates pointed out in his testimony before our
committee last month, the list of big-ticket weapon systems
that have experienced contract or program performance problems
spans the Services and includes the Air Force tanker, the CSAR-
X, the VH-71, the Osprey, the Future Combat Systems (FCS), the
Armed Reconnaissance Helicopter, the Littoral Combat Ship
(LCS), the Joint Strike Fighter (JSF), and so on.
Overall, DOD's 95 defense acquisition programs, known as
Major Defense Acquisition Programs (MDAPs), have exceeded their
research and develop budgets by an average of 40 percent, seen
their acquisition costs grow by an average of 26 percent, and
experienced an average schedule delay of almost 2 years. Last
summer, the Government Accountability Office (GAO) reported
that cost overruns on DOD's MDAPs now total $295 billion over
the original program estimates. That's true, even though we've
cut unit quantities and reduced performance expectations on
many programs, in an effort to hold costs down.
These cost overruns happen because of fundamental flaws
that are endemic to our acquisition system. We have a pretty
good idea of what those flaws are. Major acquisition programs
fail because DOD: one, continues to rely on unreasonable cost
and schedule estimates; two, establishes unrealistic
performance expectations; three, insists on the use of immature
technologies; and four, adopts costly changes to program
requirements, production quantities, and funding levels in the
middle of ongoing programs.
Earlier this year, Under Secretary of Defense for
Acquisition John T. Young, Jr., wrote a memo in which he sought
to explain the cost growth on some of DOD's largest programs.
This is what his memorandum said: ``A number of programs had a
poor foundation and Milestone B, the starting point for major
development and manufacturing design. . . Fundamentally, these
programs moved past Milestone B with inadequate foundations
built on artificially low cost estimates, optimistic schedules
and assumptions, immature design or technology, fluid
requirements, and other issues.''
Mr. Young then went on to list the flaws of each MDAP. The
JSF: too little understanding of the design; the FCS: fluid
program strategy; the V-22: immature technology and Congress
reversed DOD termination; the C-17: development issues and
underfunding; the Army's Family of Medium Tactical Vehicles:
design flaws; the CH-47F: low estimates and invalid
remanufacture assumptions; the advanced EHF Satellite:
optimistic schedule; the LPD-17: flawed lead ship design
process and knowledge base; and the F-22A: immature, exquisite
technology.
Now, the first two of these programs, JSF and FCS, account
for almost $80 billion in cost overruns, with average unit
costs that have already increased by roughly 40 percent each
over original program estimates, and are likely to rise
further. According to GAO, both programs were initiated with
insufficiently mature technologies and overly optimistic
assumptions about system performance.
With regard to the JSF, GAO reports that initial estimates
assumed that commonality between the three variants of the
aircraft could cut development costs by about 40 percent;
however, this level of commonality has proven impossible to
achieve. Twelve years after the program started, three of the
JSF's eight critical technologies are still not mature, its
production processes are not mature, and its designs are still
not fully proven and tested.
With regard to FCS, GAO reports that the estimated lines of
code needed to support FCS's software and development are
almost three times the original assumptions, leading to an
increase in software development costs that now approaches $8
billion. Eight years after the program started, only 3 of the
FCS's 44 critical technologies are fully mature. GAO tells us
that the Army has not advanced the maturity of 11 critical
technologies since 2003, and that 2 other technologies, which
are central to the Army's plans, are now rated less mature than
when the program began.
This is the price that we have paid for our failure to
complete needed systems engineering tasks, perform appropriate
developmental testing, and build prototypes. Particularly at
this time, when the Federal budget is under immense strain as a
result of the economic crisis, we cannot continue this kind of
waste and inefficiency.
That is why Senator McCain and I have introduced the Weapon
Systems Acquisition Reform Act of 2009. This bill is designed
to help put MDAPs on a sound footing from the outset by
addressing program shortcomings in the early phases of the
acquisition process.
In particular, our bill would address unreasonable
performance requirements by requiring DOD to rebuild its
systems engineering capability, reestablish the position of
director of developmental testing, and use the Joint
Requirements Oversight Council (JROC) to make early tradeoffs
between cost schedule and performance requirements.
Our bill will address unreasonable cost and schedule
estimates by establishing a new director of independent cost
assessment to ensure that cost estimates for MDAPs are fair,
reliable, and unbiased.
Our bill will reduce the use of immature technologies by
requiring DOD to make greater use of prototypes, including
competitive prototypes, and requiring the Director of Defense
Research and Engineering to periodically review and provide
independent assessments of the maturity of critical
technologies on major weapon systems.
Our bill, finally, addresses costly changes in the middle
of programs by ensuring, through preliminary design review,
that requirements are well understood before a program receives
Milestone B approval, by providing an incentive for contractors
to improve performance in ongoing programs by developing
mechanisms to maintain competitive pressure through the program
cycle, and by tightening the so-called Nunn-McCurdy
requirements for underperforming programs by providing for the
termination of any program that cannot be justified after a
complete reexamination and revalidation.
Today, we will hear from two distinguished former Under
Secretaries of Defense for Acquisition, Paul Kaminski and
Jacques Gansler. We will also hear from Pete Adolph, a former
DOD Director of Developmental Testing, and Mike Sullivan, the
GAO Director of Acquisition and Sourcing Management. Each of
our witnesses has great experience in the area of weapon
systems acquisition; and, in the course of the last year, each
has completed a major report recommending significant
improvements and reforms. We all look forward to their
testimony on these issues.
I now call on Senator McCain.
STATEMENT OF SENATOR JOHN McCAIN
Senator McCain. Thank you, Mr. Chairman, for today's
hearing, and more importantly, your leadership on the bill that
is the subject of today's hearing, the Weapons Acquisition
Reform Act of 2009. I join you in welcoming our expert
witnesses today.
Let me set the overall context of today's discussion, and
I'll do so very simply. A train wreck is coming. Look at the
President's 10-year budget and you'll see an overall decrease
in defense spending. Unless difficult decisions are made and
serious reform measures are undertaken, our ability to provide
for our national security will be, over time, fundamentally
compromised.
Clearly, the endless cycle of runaway costs, prolonged
delivery schedules, and poor performance in the acquisition of
major weapons has, in my view, us mired in a form of unilateral
disarmament. Since scrutinizing the tanker lease scandal years
ago, I'm not sure that things are any better. For example, how
could DOD award a multibillion-dollar contract based on a
proposal it later found was fundamentally unexecutable? That's
exactly what happened on the Navy's VH-71 program, the program
to replace the President's own helicopters.
Just over the last few years, the VH-71 program has
doubled, with an additional cost of $6 billion for 28 aircraft
that will likely cost taxpayers well over $400 million each.
How could DOD laden a multibillion-dollar shipbuilding program
with so many requirements that the program more than doubled in
cost, with DOD basically asleep at the switch? That happened
with the Navy's LCS program. At times, the program saw change
orders averaging 75 per week.
How could a multibillion-dollar program for next-generation
fighter jets produce planes that are operating below
satisfactory readiness rates and could end up being too
expensive to operate? That happened with the Air Force's F-22
Raptor program.
How could DOD spend billions for the Army's biggest
transformational program, valued at almost $200 billion, only
for it to be, in many respects, closer to the beginning of
development than it is to the end? That's the FCS program. At
this point, it's not been clear when, or even if, the
information network at the heart of the FCS concept can be
built.
On our military satellite program, how could a design flaw
recently emerge that will take at least 1 year, and up to $1
billion to fix? That's the Air Force's Space-Based Infrared
System (SBIRS), high satellite program. More cost and schedule
increases are likely there.
But, to understand the depth and breadth of our acquisition
problems, one needs to go no further than to look at the status
of particular programs. Across all the Services, the top 75
programs have unfunded cost overruns of at least $295 billion.
From 2000, the number of MDAPs has increased from 75 to 95, and
the cost of those programs has doubled from $790 billion to
$1.6 trillion, leaving unfunded acquisition commitments equal
to more than 10 years' worth of major weapons procurement
funding.
In other words, in the current fiscal environment we find
ourselves, the DOD acquisition plan is unaffordable. In my
view, meaningful reform is only going to happen if DOD itself
decides to change, develops an overarching management
philosophy, sets up clear lines of authority and
accountability, brings discipline and control over the
requirement process, shuts the revolving door, and restores the
corps of qualified and experienced acquisition and contracting
professionals. That's what this legislation helps to do.
In this bill, the chairman and I built on previous reform
initiatives by focusing on costs and risk. The bill reflects
that a key to managing defense procurement programs effectively
is starting them right by requiring key program reviews up
front to catch costly design flaws and technology risks before
we actually buy them.
Probably the most aggressive feature of the bill gives DOD
a big stick, bigger than anything available under current law,
to wield against the very worst-performing programs. It does so
by giving DOD additional tools to enforce fair, reliable, and
unbiased independent cost estimates with the creation of a new
director. Unlike merely promulgated DOD instructions, which
apply only to new programs, that provision will capture
chronically-poor performers that are in the development
pipeline now.
Mr. Chairman, I don't want to go all over the features of
the bill; I want to hear from our witnesses. But, for truly
meaningful reform to endure, the commitment to reform must
begin with the fiduciaries of the taxpayers dollars within the
DOD itself.
I thank you, Mr. Chairman. I welcome the witnesses.
[The prepared statement of Senator McCain follows:]
Prepared Statement by Senator John McCain
Thank you, Mr. Chairman, for today's hearing and, more importantly,
your leadership on the bill that is the subject of today's hearing,
S.454, ``The Weapons Systems Acquisition Reform Act of 2009.''
I appreciate the opportunity to serve as an original co-sponsor on
it with you, and I too welcome our expert witnesses today.
Before I turn to the bill, let me briefly set the overall context
of today's discussion, and I'll do so simply.
A train wreck is coming, and unless hard decisions are made and
serious reform measures undertaken, our ability to see to our national
security interests will be over time fundamentally compromised.
Let me be clear. The endless cycle of runaway costs, prolonged
delivery schedules and poor performance we have seen in major weapons
has us, in my view, mired in a form of unilateral disarmament.
Since we closely scrutinized the tanker lease scandal years ago,
I'm not sure that things have gotten much better. For example:
How could the DOD have awarded a multibillion contract
based on a proposal it later found was ``fundamentally
unexecutable.'' That's exactly what happened on the Navy's VH-
71 program--the program to replace the President's own
helicopters. Just over the last few years, the program has
increased by about $6 billion--for aircraft that will likely
cost taxpayers well over $400 million each.
How could the DOD have loaded up a multibillion shipbuilding
program with so many requirements that the program doubled in cost (by
about $400 million) without the DOD really noticing until it was too
late? That happened--on the Navy's Littoral Combat Ship (LCS) program.
At times, that program saw change orders averaging 75 per week.
How could a multibillion dollar program for next
generation fighter jets have produced planes that may end up
being too expensive to operate? That too happened--on the F-22
Raptor program. While being the Nation's most expensive fighter
aircraft, those jets continue to operate below satisfactory
reliability rates.
How could the DOD have spent billions for the Army's
biggest transformational program, valued at about $200 billion,
only for it to be (in many respects) closer to the beginning of
development than it is to the end? That's the Future Combat
Systems (FCS) program. At this point, it's not even clear when
(or even if) the information network at the heart of the FCS
concept can be built.
On a military satellite program, how could a design
flaw have recently emerged that will take at least 1 year and
up to $1 billion to fix? That's the Air Force's Space-Based
Infrared System High satellite program. More cost and schedule
increases are likely there.
We're supposed to have laws in the books that are supposed to
prevent these sorts of things from happening. Why didn't they work? To
the person--who is responsible?
The fact that we're asking those questions (with billions of
dollars of taxpayer money at stake) and that we have no answers, lays
out what we're dealing with far better than I can possibly describe
using facts and figures.
So, I don't have to mention that the top 75 programs across all of
the Services have unfunded cost overruns of at least $295 billion. Or,
that (since fiscal year 2000) the number of major defense acquisition
programs has increased from 75 to 95. Or, that within that period the
cost of those programs doubled from $790 billion to $1.6 trillion. Or,
that this left unfunded acquisition commitments equal to more than 10
years worth of major weapons procurement funding.
I don't have to talk about how risky developing most of those
programs are; or the likelihood that they too will balloon in costs; or
how much other government-wide priorities will constrain defense
spending going forward. I don't have to go into all that to make the
point that the DOD's acquisition plan--as it currently stands--is
itself likely unaffordable.
However one looks at it, the honeymoon is over.
In my view, meaningful reform is only going to happen if DOD itself
decides to change. DOD has to:
Develop an overarching management philosophy that
dictates an overall approach to ensuring the timely delivery of
major weapons that satisfy the needs of the joint warfighter at
the most reasonable cost to the taxpayer.
Set up clear lines of authority and accountability for
managing procurement programs.
Bring discipline and control over the requirements
process and get out of the business of gold-plating programs.
Shut the revolving door. While a leavening of
experienced DOD procurement officials working for defense
contractors (and vice-versa) is healthy, the lack of meaningful
controls on this revolving door is creating an unhealthy
tolerance of conflict-of-interest.
Restore the corps of qualified and experienced
acquisition and contracting professionals that DOD had in the
1980s before it gave its functions over to contractor/lead
systems integrators, thereby letting the fox guard the hen
house.
Until and unless administration and DOD leadership do these sorts
of things--things that set, if you will, a ``command climate'' that's
conducive to investing the taxpayers' money responsibly--I fear that
reform efforts may amount to only rearranging the deck chairs on the
Titanic. That's something that the chairman and I have no interest in
doing.
At the end of the day, we in Congress can only give DOD tools that
it can use to pursue truly lasting solutions that ensure desirable
cost/scheduling and performance outcomes in our most complicated, most
expensive weapons systems. That's what this bill helps do.
In this bill, the chairman and I build on previous reform
initiatives by focusing on costs and risk. The bill reflects that a key
to managing defense procurement programs effectively is starting them
right--by requiring key program review upfront to catch costly design
flaws and technology risk before we actually buy the system.
In so doing, we continue our efforts to move DOD closer towards
fixed price-type procurement contracting by requiring that technology
and integration risk can be meted out of a program early. So, by the
time a program heads into procurement, if a contractor isn't ready to
sign a good fixed price-type contract, the government shouldn't be
signing a contract with that company to buy that system at all. There's
still too much risk.
Probably the most aggressive feature of the bill gives DOD a big
stick--bigger than anything available under current law--to wield
against the very worst performing programs. With that provision we
intend DOD to, if you will, ``enforce'' fair, reliable, and unbiased
cost estimates verified by a new director of independent cost
estimates, also created by this bill. Unlike rules that DOD recently
put in place, that provision doesn't apply to just new programs. It
will capture chronically poor performers that are in the development
pipeline now.
Another important provision requires DOD to consider a broad range
of cost-effective measures to help maximize competition throughout the
life of a weapons program. The industry consolidation that occurred
after the end of the Cold War went too far. Some 50 prime contractors
merged into only 6. That's far too few to support a competitive base
for our current and future needs. It's resulted in a serious decline in
innovation.
Other provisions in the bill elaborate in the ``starting programs
right'' theme by:
Renewing focus on systems engineering early;
Requiring the completion of preliminary design reviews
before a program can move into the development phase; and
Strengthening DOD's developmental testing and
evaluation capability.
Other helpful provisions include those that:
Require DOD budget, requirements, and acquisition
officials to consult each other and make trade-offs between
cost, schedule, and performance early in the process; and
Crack down on the frequent changes to programs, which
tend to cause many cost increases.
While this bill is not intended as a panacea to cure all that ails
the defense procurement process, it is an important next step in
Congress' continuing efforts to help DOD culturally reform the system.
As I said a moment ago, for meaningful reform to truly endure, the
commitment to reform must begin with the fiduciaries of the taxpayers'
dollars within the Department itself.
Thank you, Mr. Chairman. I look forward to hearing from our
witnesses today.
Chairman Levin. Senator McCain, we thank you.
Now we'll call on our witnesses. First, we'll call on
Michael Sullivan from the GAO.
Would you please proceed?
STATEMENT OF MICHAEL J. SULLIVAN, DIRECTOR, ACQUISITION AND
SOURCING MANAGEMENT, GOVERNMENT ACCOUNTABILITY OFFICE
Mr. Sullivan. Chairman Levin, Senator McCain, members of
the committee, I'm pleased to be here today to discuss DOD's
acquisition outcomes and the legislation proposed by this
committee to improve them. I'll make a brief oral statement and
ask that my written testimony be placed in the record.
Chairman Levin. It will be.
Mr. Sullivan. We've been reporting for years on poor cost
and schedule outcomes on DOD's major weapon system
acquisitions. As the chairman noted, most recently we reported
that 95 programs in DOD's current portfolio have grown in cost
by $295 billion and are, on average, delivered about 21 months
late. We believe there are problems at the strategic and at the
program levels that cause these outcomes.
At the strategic level, DOD's three systemic processes for
building its investment strategy are fragmented and broken. The
requirement-setting process, known as the Joint Capabilities
and Integration Development System (JCIDS) is stovepiped, it
does not consider resources, and it approves nearly every
proposal that it reviews. The funding process accepts programs
with unrealistic cost estimates, and does not fully fund their
development costs. These two processes are poorly integrated,
and this poor communication leads to unhealthy competition,
where too many programs are chasing too few dollars.
Finally, at the program level, the acquisition process
initiates programs with unreliable cost estimates and without
knowledge from proper systems engineering analysis to
understand each weapon system's requirements and the resources
that will be needed to achieve them. These programs move
forward with too much technology, design, and manufacturing
risk as a result.
DOD understands this and has recently revised its policies
to address some of these problems. Its new acquisition policy,
for instance, encourages more systems engineering activity
earlier in programs, competitive prototyping to gain knowledge
more quickly and to maintain competition, earlier milestone
reviews, and steering boards to protect programs against the
desire to add more requirements once they've started.
Recent decisions by DOD on some programs have been
encouraging, and some of the newer programs appear to have
undergone more disciplined reviews.
For many years, there's been a broad consensus that weapon
system acquisition problems are serious and their resolution is
overdue. With the Federal budget under increasing strain from
the economic crisis facing our Nation, the time to change is
now.
In testimony before this committee last month, Secretary of
Defense Gates identified many of the systemic problems
associated with acquisitions and indicated that efforts are
underway to address them.
We believe that the legislation this committee has proposed
will help address the toughest problems, and we
enthusiastically support it. We believe it precisely targets
key problem areas, provides much-needed oversight, and provides
increased authority and independence to the critical functions
of cost estimating and development testing by requiring them to
report to the Secretary and to Congress.
Among other things, its provision to require a full
inventory of DOD's current systems engineering skills is an
excellent beginning to rebuilding that sorely-needed
capability. Its addition of a termination criterion for Nunn-
McCurdy breaches sends a strong signal to programs to have
realistic cost estimates when they start.
It is important to state that there is also a need for
changes to the overall acquisition culture and the incentives
it provides. The culture should begin to change by resisting
the urge to achieve the revolutionary, but unachievable,
capability in one step by allowing technologies to mature in
the technology base rather than forcing them on the acquisition
programs too early, by ensuring that urgent requirements are
well-defined and quickly achievable, and by instituting
shorter, more predictable development cycles.
These changes will not be easy to make. Tough decisions
must be made about DOD's overall portfolio of weapon programs
and about specific programs; and stakeholders from DOD, the
military services industry, and Congress will have to play a
constructive role in this decisionmaking. We see the proposed
legislation discussed here today as a very healthy step in that
direction.
Mr. Chairman, this concludes my statement. I'll wait to
answer questions.
[The prepared statement of Mr. Sullivan follows:]
Prepared Statement by Michael J. Sullivan
Mr. Chairman and members of the committee: I am pleased to be here
today to discuss the Department of Defense's (DOD) management of its
major weapon system acquisitions--an area that has been on the
Government Accountability Office's (GAO) high-risk list since 1990.
Prior to and since that time, Congress and DOD have continually
explored ways to improve acquisition outcomes without significant
results. While the technological sophistication of DOD weapon systems
is unparalleled, major weapon programs continue to cost more, take
longer to complete, and deliver fewer quantities and capabilities than
originally planned. Last year we reported that the cumulative cost
growth in DOD's portfolio of 95 major defense acquisition programs was
$295 billion from first estimates and the average delay in delivering
promised capabilities to the warfighter was 21 months. Clearly, some
problems are to be expected in developing weapon systems given the
technical risks and complexities involved. However, all too often we
have found that cost and schedule problems are rooted in poor planning,
execution, and oversight.
Investment in weapon systems is now at its highest level in two
decades, and DOD plans to spend more than $357 billion over the next 5
years on major defense acquisition programs. Effective management of
this substantial investment is critical as competition for funding has
increased dramatically within the department and across the government.
DOD faces a number of fiscal pressures: the ongoing military campaigns
in Afghanistan and Iraq, rising personnel costs, and the rebuilding and
modernization of the force. In addition, the economic and fiscal crises
now facing the Nation have required unprecedented spending by the
Federal Government, and budget deficits are projected to remain high
for many years to come. At a time when the Federal budget is strained
by spending needs for a growing number of national priorities, it is
imperative that DOD get the best value for every dollar it invests in
weapon system programs. Every dollar wasted during the development and
acquisition of weapon systems is money not available for other
priorities within DOD and elsewhere in the government.
Today, I will discuss: (1) the systemic problems that have
contributed to cost, schedule, and performance problems in DOD's
acquisition of major weapon systems; (2) recent actions the department
has taken to address these problems; (3) our observations on the
committee's proposed acquisition reform legislation; and (4) steps that
Congress and the department need to take to improve the future
performance of acquisition programs. The statement includes findings
from our July 2008 report on a knowledge-based funding approach and
February 2009 report on potential changes to DOD's acquisition
management framework.\1\ It also draws from our extensive body of work
on DOD's acquisition of weapon systems. A list of our key products is
provided at the end of this statement. This work was conducted in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
---------------------------------------------------------------------------
\1\ GAO, Defense Acquisitions: A Knowledge-Based Funding Approach
Could Improve Major Weapon System Program Outcomes, GAO-08-619
(Washington, DC: July 2, 2008), and Defense Acquisitions: Perspectives
on Potential Changes to DOD's Acquisition Management Framework, GAO-09-
295R (Washington, DC: Feb. 27, 2009).
---------------------------------------------------------------------------
FRAGMENTED INVESTMENT DECISIONMAKING, UNEXECUTABLE PROGRAMS, AND LACK
OF ACCOUNTABILITY UNDERLIE POOR ACQUISITION OUTCOMES
Over the past several years our work has highlighted a number of
underlying systemic causes for cost growth and schedule delays at both
the strategic and program levels. At the strategic level, DOD's
processes for identifying warfighter needs, allocating resources, and
developing and procuring weapon systems--which together define DOD's
overall weapon system investment strategy--are fragmented. As a result,
DOD fails to effectively address joint warfighting needs and commits to
more programs than it has resources for, thus creating unhealthy
competition for funding. At the program level, a military Service
typically establishes and DOD approves a business case containing
requirements that are not fully understood and cost and schedule
estimates that are based on overly optimistic assumptions rather than
on sufficient knowledge. Once a program begins, it too often moves
forward with inadequate technology, design, testing, and manufacturing
knowledge, making it impossible to successfully execute the program
within established cost, schedule, and performance targets.
Furthermore, DOD officials are rarely held accountable for poor
decisions or poor program outcomes.
DOD Lacks an Integrated Approach to Balance Weapon System Investments
At the strategic level, DOD largely continues to define warfighting
needs and make investment decisions on a Service-by-Service and
individual platform basis, using fragmented decisionmaking processes.
This approach makes it difficult for the department to achieve a
balanced mix of weapon systems that are affordable and feasible and
that provide the best military value to the joint warfighter. In
contrast, we have found that successful commercial enterprises use an
integrated portfolio management approach to focus early investment
decisions on products collectively at the enterprise level and ensure
that there is a sound basis to justify the commitment of resources.\2\
By following a disciplined, integrated process--during which the
relative pros and cons of competing product proposals are assessed
based on strategic objectives, customer needs, and available resources,
and where tough decisions about which investments to pursue and not to
pursue are made--companies minimize duplication between business units,
move away from organizational stovepipes, and effectively support each
new development program they commit to. To be effective, integrated
portfolio management must have strong, committed leadership; empowered
portfolio managers; and accountability at all levels of the
organization.
---------------------------------------------------------------------------
\2\ GAO, Best Practices: An Integrated Portfolio Management
Approach to Weapon System Investments Could Improve DOD's Acquisition
Outcomes, GAO-07-388 (Washington, DC: Mar. 30, 2007).
---------------------------------------------------------------------------
DOD determines its capability needs through the Joint Capabilities
and Integration Development System (JCIDS). While JCIDS provides a
framework for reviewing and validating needs, it does not adequately
prioritize those needs from a joint, departmentwide perspective; lacks
the agility to meet changing warfighter demands; and validates almost
all of the capability proposals that are submitted. We recently
reviewed JCIDS documentation related to new capability proposals and
found that most--almost 70 percent--were sponsored by the military
Services with little involvement from the joint community, including
the combatant commands, which are responsible for planning and carrying
out military operations.\3\ Because DOD also lacks an analytic approach
to determining the relative importance of the capabilities needed for
joint warfighting, all proposals appear to be treated as equal
priorities within the JCIDS process. By continuing to rely on
capability needs defined primarily by the Services, DOD may be losing
opportunities for improving joint warfighting capabilities and reducing
the duplication of capabilities in some areas. The JCIDS process has
also proven to be lengthy and cumbersome--taking on average up to 10
months to validate a need--thus undermining the department's efforts to
effectively respond to the needs of the warfighter, especially those
needs that are near term. Furthermore, the vast majority of capability
proposals that enter the JCIDS process are validated or approved
without accounting for the resources or technologies that will be
needed to acquire the desired capabilities. Ultimately, the process
produces more demand for new weapon system programs than available
resources can support.
---------------------------------------------------------------------------
\3\ GAO, Defense Acquisitions: DOD's Requirements Determination
Process Has Not Been Effective in Prioritizing Joint Capabilities, GAO-
08-1060 (Washington, DC: Sept. 25, 2008).
---------------------------------------------------------------------------
The funding of proposed programs takes place through a separate
process, the department's Planning, Programming, Budgeting, and
Execution (PPBE) system, which is not synchronized with JCIDS. While
JCIDS is a continuous, need-driven process that unfolds in response to
capability proposals as they are submitted by sponsors, PPBE is a
calendar-driven process comprising phases occurring over a 2-year
cycle, which can lead to resource decisions for proposed programs that
may occur several years later. In addition, because PPBE is structured
by military Service and defense programs and not by the joint
capability areas being used in JCIDS, it is difficult to link resources
to capabilities. The PPBE process also largely allocates resources
based on historical trends rather than on a strategic basis. Service
shares of the overall budget have remained relatively static for
decades, even though DOD's strategic environment and warfighting needs
have changed dramatically in recent years. Because DOD's programming
and budgeting reviews occur at the back end of the PPBE process--after
the Services have developed their budgets--it is difficult and
disruptive to make changes, such as terminating programs to pay for
new, higher-priority programs.
We recently reviewed the impact of the PPBE process on major
defense acquisition programs and found that the process does not
produce an accurate picture of the department's resource needs for
weapon system programs, in large part because it allows too many
programs to go forward with unreliable cost estimates and without a
commitment to fully fund them.\4\ The cost of many of the programs we
reviewed exceeded the funding levels planned for and reflected in the
Future Years Defense Program (FYDP)--the department's long-term
investment strategy (see fig. 1). DOD's failure to balance its needs
with available resources promotes an unhealthy competition for funding
that encourages sponsors of weapon system programs to pursue overly
ambitious capabilities and underestimate costs to appear affordable.
Rather than limit the number and size of programs or adjust
requirements, DOD opts to push the real costs of programs to the
future. With too many programs underway for the available resources and
high cost growth occurring in many programs, the department must make
up for funding shortfalls by shifting funds from one program to pay for
another, reducing system capabilities, cutting procurement quantities,
or in rare cases terminating programs. Such actions not only create
instability in DOD's weapon system portfolio, they further obscure the
true future costs of current commitments, making it difficult to make
informed investment decisions.
---------------------------------------------------------------------------
\4\ GAO-08-619.
---------------------------------------------------------------------------
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Initiating Programs with Inadequate Knowledge of Requirements and
Resources Often Results in Poor Outcomes
At the program level, the key cause of poor outcomes is the
approval of programs with business cases that contain inadequate
knowledge about requirements and the resources--funding, time,
technologies, and people--needed to execute them. Our work in best
practices has found that an executable business case for a program
demonstrated evidence that: (1) the identified needs are real and
necessary and that they can best be met with the chosen concept; and
(2) the chosen concept can be developed and produced within existing
resources. Over the past several years, we have found no evidence of
the widespread adoption of such an approach for major acquisition
programs in the department. Our annual assessments of major weapon
systems have consistently found that the vast majority of programs
began system development without mature technologies and moved into
system demonstration without design stability.
The chief reason for these problems is the encouragement within the
acquisition environment of overly ambitious and lengthy product
developments--sometimes referred to as revolutionary or big bang
acquisition programs--that embody too many technical unknowns and not
enough knowledge about the performance and production risks they
entail. The knowledge gaps are largely the result of a lack of early
and disciplined systems engineering analysis of a weapon system's
requirements prior to beginning system development. Systems engineering
translates customer needs into specific product requirements for which
requisite technological, software, engineering, and production
capabilities can be identified through requirements analysis, design,
and testing. Early systems engineering provides the knowledge a product
developer needs to identify and resolve performance and resource gaps
before product development begins by either reducing requirements,
deferring them to the future, or increasing the estimated cost for the
weapon system's development. Because the government often does not
perform the proper upfront requirements analysis to determine whether
the program will meet its needs, significant contract cost increases
can and do occur as the scope of the requirements changes or becomes
better understood by the government and contractor. Not only does DOD
not conduct disciplined systems engineering prior to the beginning of
system development, it has allowed new requirements to be added well
into the acquisition cycle. We have reported on the negative impact
that poor systems engineering practices have had on several
programs, such as the Global Hawk Unmanned Aircraft System, F-
22A, Expeditionary Fighting Vehicle, and Joint Air-to-Surface Standoff
Missile.\5\
---------------------------------------------------------------------------
\5\ GAO, Best Practices: Increased Focus on Requirements and
Oversight Needed to Improve DOD's Acquisition Environment and Weapon
System Quality, GAO-08-294 (Washington, DC: Feb. 1, 2008).
---------------------------------------------------------------------------
With high levels of uncertainty about requirements, technologies,
and design, program cost estimates and related funding needs are often
understated, effectively setting programs up for cost and schedule
growth. We recently assessed the service and independent cost estimates
for 20 major weapon system programs and found that while the
independent estimates were somewhat higher, both estimates were too low
in most cases.\6\ In some of the programs we reviewed, cost estimates
have been off by billions of dollars. For example, the initial service
estimate for the development of the Marines' Expeditionary Fighting
Vehicle was about $1.1 billion. The department's Cost Analysis and
Improvement Group (CAIG) estimated the development cost of the program
to be $1.4 billion, but development costs for the program are now
expected to be close to $3.6 billion. In the case of the Future Combat
System (FCS), the Army's initial estimate for the development cost was
about $20 billion, while CAIG's estimate was $27 billion. The
department began the program using the program office's estimate of $20
billion, but development costs for the FCS are now estimated to be $28
billion and the program is still dealing with significant technical
risk. Estimates this far off the mark do not provide the necessary
foundation for sufficient funding commitments and realistic long-term
planning.
---------------------------------------------------------------------------
\6\ GAO-08-619.
---------------------------------------------------------------------------
The programs we reviewed frequently lacked the knowledge needed to
develop realistic cost estimates. For example, program Cost Analysis
Requirements Description documents--used to build the program cost
estimate--often lack sufficient detail about planned program content
for developing sound cost estimates. Without this knowledge, cost
estimators must rely heavily on parametric analysis and assumptions
about system requirements, technologies, design maturity, and the time
and funding needed. A cost estimate is then usually presented to
decisionmakers as a single, or point, estimate that is expected to
represent the most likely cost of the program but provides no
information about the range of risk and uncertainty or level of
confidence associated with the estimate.
Lack of Accountability for Making Weapon System Decisions Hinders
Achieving Successful Outcomes
DOD's requirements, resource allocation, and acquisition processes
are led by different organizations, thus making it difficult to hold
any one person or organization accountable for saying no to a proposed
program or for ensuring that the department's portfolio of programs is
balanced. DOD's 2006 Defense Acquisition Performance Assessment study
observed that these processes are not connected organizationally at any
level below the Deputy Secretary of Defense and concluded that this
weak structure induces instability and inhibits accountability.
Furthermore, a former Under Secretary of Defense for Acquisitions,
Technology, and Logistics has stated that weapon system investment
decisions are a shared responsibility in the department and, therefore,
no one individual is accountable for these decisions. Frequent turnover
in leadership positions in the department exacerbates the problem. The
average tenure, for example, of the Under Secretary of Defense for
Acquisition, Technology, and Logistics over the past 22 years has been
only about 20 months.\7\
---------------------------------------------------------------------------
\7\ The position of Under Secretary of Defense for Acquisition was
established in 1986 and the title was subsequently changed to the Under
Secretary of Defense for Acquisition, Technology, and Logistics in
2000. Since 1986, there have been 11 Under Secretaries.
---------------------------------------------------------------------------
When DOD's strategic processes fail to balance needs with resources
and allow unsound, unexecutable programs to move forward, program
managers cannot be held accountable when the programs they are handed
already have a low probability of success. Program managers are also
not empowered to make go or no-go decisions, have little control over
funding, cannot veto new requirements, and have little authority over
staffing. At the same time, program managers frequently change during a
program's development, making it difficult to hold them accountable for
the business cases that they are entrusted to manage and deliver.
The government's lack of control over and accountability for
decisionmaking is further complicated by DOD's growing reliance on
technical, business, and procurement expertise supplied by contractors.
This reliance may reach the point where the foundation upon which
decisions are based may be largely crafted by individuals who are not
employed by the government, who are not bound by the same rules
governing their conduct, and who are not required to disclose any
financial or other personal interests they may have that conflict with
the responsibilities they have performing contract tasks for DOD. For
example, while the total planned commitments to major acquisition
programs have doubled over recent years, the size of the department's
systems engineering workforce has remained relatively stable, leading
program offices to rely more on contractors for systems engineering
support. Further, in systems development, DOD typically uses cost-
reimbursement contracts in which it generally pays the reasonable,
allocable, and allowable costs incurred for the contractor's best
efforts, to the extent provided by the contract. The use of these
contracts may contribute to the perpetuation of an acquisition
environment that lacks incentives for contractors to follow best
practices and keep costs and schedules in check.
RECENT DOD POLICY CHANGES COULD IMPROVE FUTURE PERFORMANCE OF WEAPON
SYSTEM PROGRAMS
The department understands many of the problems that affect
acquisition programs and has recently taken steps to remedy them. It
has revised its acquisition policy and introduced several initiatives
based in part on direction from Congress and recommendations from GAO
that could provide a foundation for establishing sound, knowledge-based
business cases for individual acquisition programs. However, to improve
outcomes, DOD must ensure that its policy changes are consistently
implemented and reflected in decisions on individual programs--not only
new program starts but also ongoing programs as well. In the past,
inconsistent implementation of existing policy has hindered DOD's
efforts to execute acquisition programs effectively. Moreover, while
policy improvements are necessary, they may be insufficient unless the
broader strategic issues associated with the department's fragmented
approach to managing its portfolio of weapon system investments are
also addressed.
In December 2008, DOD revised its policy governing major defense
acquisition programs in ways intended to provide key department leaders
with the knowledge needed to make informed decisions before a program
starts and to maintain disciplined development once it begins. The
revised policy recommends the completion of key systems engineering
activities before the start of development, includes a requirement for
early prototyping, establishes review boards to identify and mitigate
technical risks and evaluate the impact of potential requirements
changes on ongoing programs, and incorporates program manager
agreements to increase leadership stability and management
accountability. The policy also establishes early milestone reviews for
programs going through the pre--systems acquisition phase. In the past,
DOD's acquisition policy may have encouraged programs to rush into
systems development without sufficient knowledge, in part, because no
formal milestone reviews were required before system development. If
implemented, these policy changes could help programs replace risk with
knowledge, thereby increasing the chances of developing weapon systems
within cost and schedule targets while meeting user needs. Some aspects
of the policy were first pilot-tested on selected programs, such as the
Joint Light Tactical Vehicle program, and indications are that these
programs are in the process of acquiring the requisite knowledge before
the start of systems development. Some key elements of the department's
new acquisition policy include:
a new materiel development decision as a starting
point for all programs regardless of where they are intended to
enter the acquisition process,
a more robust Analysis of Alternatives (AOA) to assess
potential materiel solutions that address a capability need
validated through JCIDS,
a cost estimate for the proposed solution identified
by the AOA,
early program support reviews by systems engineering
teams,
competitive prototyping of the proposed system or key
system elements as part of the technology development phase,
certifications for entry into the technology
development and system development phases (as required by
congressional legislation),
preliminary design review (PDR) that may be conducted
before the start of systems development, and
configuration steering boards to review all
requirements and technical changes that have potential to
affect cost and schedule.
As part of its strategy for enhancing the roles of program managers
in major weapon system acquisitions, the department has established a
policy that requires formal agreements among program managers, their
acquisition executives, and the user community setting forth common
program goals. These agreements are intended to be binding and to
detail the progress the program is expected to make during the year and
the resources the program will be provided to reach these goals. DOD
also requires program managers to sign tenure agreements so that their
tenure will correspond to the next major milestone review closest to 4
years. The department acknowledges that any actions taken to improve
accountability must be based on a foundation whereby program managers
can launch and manage programs toward successful performance, rather
than focusing on maintaining support and funding for individual
programs. DOD acquisition leaders have also stated that any
improvements to program managers' performance depend on the
department's ability to promote requirements and resource stability
over weapon system investments.
Over the past few years, DOD has also been testing portfolio
management approaches in selected capability areas--command and
control, net-centric operations, battlespace awareness, and logistics--
to facilitate more strategic choices for resource allocation across
programs. The department recently formalized the concept of capability
portfolio management, issuing a directive in 2008 that established
policy and assigned responsibilities for portfolio management. The
directive established nine joint capability area portfolios, each to be
managed by civilian and military co-leads. While the portfolios have no
independent decisionmaking authority over requirements determination
and resource allocation, according to some DOD officials, they provided
key input and recommendations in this year's budget process. However,
without portfolios in which managers have authority and control over
resources, the department is at risk of continuing to develop and
acquire systems in a stovepiped manner and of not knowing if its
systems are being developed within available resources.
OBSERVATIONS ON PROPOSED ACQUISITION REFORM LEGISLATION
Overall, we believe that the legislative initiatives being proposed
by the committee have the potential, if implemented, to lead to
significant improvements in DOD's management of weapon system programs.
Several of the initiatives--including the increased emphasis on systems
engineering and developmental testing, the requirement for earlier
PDRs, and the strengthening of independent cost estimates and
technology readiness assessments--could instill more discipline into
the front end of the acquisition process when it is critical for
programs to gain knowledge. Establishing a termination criterion for
Nunn-McCurdy cost breaches could help prevent the acceptance of
unrealistic cost estimates as a foundation for starting programs.\8\
Having greater involvement by the combatant commands in determining
requirements and requiring greater consultation between the
requirements, budget, and acquisition processes could help improve the
department's efforts to balance its portfolio of weapon system
programs. In addition, several of the proposals as currently drafted
will codify what DOD policy already calls for, but are not being
implemented consistently in weapon programs.
---------------------------------------------------------------------------
\8\ 10 U.S.C. Sec. 2433(a)(5) requires the Secretary of Defense to
report to Congress when a program's acquisition unit cost increases by
at least 25 percent over the current baseline estimate or increases by
at least 50 percent over the original baseline estimate.
---------------------------------------------------------------------------
Section 101: Systems Engineering Capabilities
Requires DOD to: (1) assess the extent to which the department has
in place the systems engineering capabilities needed to ensure that key
acquisition decisions are supported by a rigorous systems analysis and
systems engineering process; and (2) establish organizations and
develop skilled employees to fill any gaps in such capabilities.
The lack of disciplined systems engineering analysis conducted
prior to starting system development has been a key factor contributing
to poor acquisition outcomes. Systems engineering activities--
requirements analysis, design, and testing--are needed to ensure that a
weapon system program's requirements are achievable and designable
given available resources, such as technologies. In recent years, DOD
has taken steps to improve its systems engineering capabilities by
establishing a Systems and Software Engineering Center of Excellence
and publishing guidance to assist the acquisition workforce in the
development of systems engineering plans, education, and training.
However, as the National Research Council recently reported, DOD's
systems engineering capabilities have declined over time and shifted
increasingly to outside contractors.\9\ A comprehensive assessment to
determine what systems engineering capabilities are in place and what
capabilities are needed, as recommended in the proposed legislation, is
a critical first step in enhancing the function of systems engineering
in DOD acquisitions. At the same time, it will be important for DOD to
implement steps to ensure systems engineering is applied in the right
way and at the right time.
---------------------------------------------------------------------------
\9\ National Research Council, Pre-Milestone A and Early-Phase
Systems Engineering: A Retrospective Review and Benefits for Future Air
Force Systems Acquisition (Washington, DC: February 2008).
---------------------------------------------------------------------------
Section 102: Developmental Testing
Requires DOD to reestablish the position of Director of
Developmental Test and Evaluation and requires the Services to assess
and address any shortcomings in their developmental testing
organizations and personnel.
Robust developmental testing efforts are an integral part of the
systems development process. They help to identify, evaluate, and
reduce technical risks, and indicate whether the design solution is on
track to satisfy the desired capabilities. As the Defense Science Board
reported in 2008, developmental testing in weapon system programs needs
to be improved.\10\ We believe that developmental testing would be
strengthened by a formal elevation of its role in the acquisition
process and the reestablishment of a Director of Developmental Test and
Evaluation position. Furthermore, requiring the Director to prepare an
annual report for Congress summarizing DOD's developmental test and
evaluation activities would provide more accountability. We also agree
that the military Services should be required to assess their
respective developmental testing entities and address any shortcomings.
This action would help ensure that the Services have the knowledge and
capacity for effective developmental test efforts.
---------------------------------------------------------------------------
\10\ Defense Science Board, Report on Developmental Test &
Evaluation (Washington, DC: May 2008).
---------------------------------------------------------------------------
Section 103: Technical Maturity Assessments
Makes it the responsibility of the Director of Defense Research and
Engineering (DDR&E) to periodically review and assess the technological
maturity of critical technologies used in major defense acquisition
programs.
Ensuring that programs have mature technology before starting
systems development is critical to avoiding cost and schedule problems,
yet for many years we have reported that a majority of programs go
forward with immature technologies and experience significant cost
growth. Legislation enacted by Congress in 2006, requiring DOD to
certify that the technology in a program has been demonstrated in a
relevant environment before it receives approval to start system
development, has begun to help address this problem. Since the
legislation was enacted, DOD has asked the DDR&E to conduct independent
reviews of technology readiness assessments for system development
milestone decisions. Although DDR&E reviews are advisory in nature, we
have seen reviews that have pushed programs to do more to demonstrate
technology maturity. The improvements that this proposed legislation,
as currently written, is intended to bring about may already be
occurring in DOD. Congress, however, may wish to consider requiring the
DDR&E to conduct technology readiness reviews not just periodically,
but for all major defense acquisition programs, and whether or not
DDR&E has the capacity and resources to effectively conduct technology
assessments.
Section 104: Independent Cost Assessment
Establish a Director of Independent Cost Assessment to ensure that
cost estimates for major defense acquisition programs are fair,
reliable, and unbiased.
Within DOD, the CAIG is the organization responsible for conducting
independent costs estimates for major defense acquisition programs. The
CAIG reports to the department's Director of Program Analysis and
Evaluation, but its principal customer is the Under Secretary of
Defense for Acquisition, Technology, and Logistics. We believe that
establishing an independent assessment office that reports directly to
the Secretary or Deputy Secretary of Defense and to Congress--similar
to the Office of the Director of Operation Test and Evaluation--would
more fully integrate cost estimating with the acquisition management
framework and provide an increased level of accountability. We see no
reason why CAIG should not form the basis of the proposed organization.
Congress may also wish to consider appointing the Director for a time-
certain term and making the Director responsible for prescribing cost-
estimating policy and guidance and for preparing an annual report
summarizing cost estimates for major acquisition programs. Ultimately,
however, improved cost estimating will only occur if there is a better
foundation for planning and acquiring weapon system programs--one that
promotes well-defined requirements, is knowledge-based and informed by
disciplined systems engineering, requires mature technology, and
adheres to shorter development cycle times.
Section 105: Role of Combatant Commanders
Requires the Joint Requirements Oversight Council (JROC) to seek
and consider input from the commanders of the combatant commands in
identifying joint military requirements.
Requirements determination in DOD, particularly for major weapon
system programs, continues to be driven largely by the military
Services. Studies by the Defense Science Board, Center for Strategic
and International Studies, and others have revealed that although the
combatant commands--which are responsible for planning and executing
military missions--are the principal joint warfighting customer in DOD,
they have played a limited role in determining requirements. Currently,
the JROC is doing more to seek out and consider input from the
combatant commands through regular trips and meetings to discuss
capability needs and resourcing issues. However, many of the combatant
commands do not believe that their needs, which are reflected through
the Integrated Priority List process, are sufficiently addressed
through the department's JCIDS process. For the combatant commands to
meet this proposed legislative mandate and have more influence in
establishing requirements, DOD should consider providing the combatant
commands with additional resources to establish robust analytical
capabilities for identifying and assessing their capability needs.
Ultimately, the department must better prioritize and balance the needs
of the military Services, combatant commands, and other defense
components, and be more agile in responding to near-term capability
needs.
Section 201: Trade-offs of Cost, Schedule, and Performance
Requires consultation between the budget, requirements, and
acquisition processes to ensure the consideration of trade-offs between
cost, schedule, and performance early in the process of developing
major weapon systems.
As currently structured, DOD's budget, requirements, and
acquisition processes do not operate in an integrated manner. The
function and timing of the processes are not sufficiently synchronized,
and the decisionmakers for each process are motivated by different
incentives. These weaknesses have contributed to the development of a
portfolio with more programs than available resources can support and
programs that launch into system development without executable
business cases. We have recommended that the department establish an
enterprisewide portfolio management approach to weapon system
investment decisions that integrates the determination of joint
warfighting needs with the allocation of resources, and cuts across the
Services by functional or capability area.\11\ To ensure the success of
such an approach, we believe that the department should establish a
single point of accountability with the authority, responsibility, and
tools to implement portfolio management effectively.
---------------------------------------------------------------------------
\11\ GAO-07-388.
---------------------------------------------------------------------------
Section 202: Preliminary Design Review
Require the completion of a PDR and a formal post-PDR assessment
before a major defense acquisition program receives approval to start
system development.
We have found that a key deliverable in a knowledge-based
acquisition process is the preliminary design of the proposed solution
based on a robust systems engineering assessment prior to making a
large financial commitment to system development. Early systems
engineering provides the knowledge needed by a developer to identify
and resolve gaps, such as overly optimistic requirements that cannot be
met with current resources, before product development begins.
Consequently, DOD would have more confidence that a particular system
could successfully proceed into a detailed system development phase and
meet stated performance requirements within cost, schedule, risk, and
other relevant constraints. The recently revised DOD acquisition policy
places an increased emphasis on programs planning for PDR prior to the
start of system development but does not go as far as making it a
requirement to do so. We support any effort to add controls to the
acquisition process to ensure that timely and robust systems
engineering is conducted before major investment decisions, such as the
approval to start system development, are made.
Section 203: Life-Cycle Competition
Require DOD to adopt measures recommended by the 2008 Defense
Science Board Task Force on Defense Industrial Structure for
Transformation--such as competitive prototyping, dual sourcing, open
architectures, periodic competitions for subsystem upgrades, and
licensing of additional suppliers--to maximize competition throughout
the life of a program.
We have reported in the past on the problem of diminishing
competition and the potential benefits of more competition.\12\ In
discussing the environment that leads to poor acquisition outcomes, we
have noted that changes within the defense supplier base have added
pressure to this environment. We noted that in 2006, a DOD-commissioned
study found that the number of fully competent prime contractors
competing for programs had fallen from more than 20 in 1985 to only 6,
and that this has limited DOD's ability to maximize competition in
order to reduce costs and encourage innovation. However, avenues exist
for reducing costs through competition. For example, we reported that
although continuing an alternate engine program for the Joint Strike
Fighter would cost significantly more in development costs than a sole-
source program, it could, in the long run, reduce overall life-cycle
costs and bring other benefits.
---------------------------------------------------------------------------
\12\ GAO, Defense Acquisitions: Better Weapon Program Outcomes
Require Discipline, Accountability, and Fundamental Changes in the
Acquisition Environment, GAO-08-782T (Washington, DC: June 3, 2008).
---------------------------------------------------------------------------
Section 204: Nunn-McCurdy Breaches
Requires that a major defense acquisition program that experiences
a critical Nunn-McCurdy cost breach be terminated unless the Secretary
of Defense certifies that: (1) continuing the program is essential to
national security and the program can be modified to proceed in a cost-
effective manner; and (2) the program receives a new milestone approval
prior to the award of any new or modified contract extending the scope
of the program.
In order for DOD to improve its program outcomes, realistic cost
estimates must be required when programs are approved for development
initiation. DOD often underestimates costs in large part because of a
lack of knowledge and overly optimistic assumptions about requirements
and critical technologies. This underestimation is also influenced by
DOD's continuing failure to balance its needs with available resources,
which promotes unhealthy competition among programs and encourages
programs to overpromise on performance capabilities and underestimate
cost. This false optimism is reinforced by an acquisition environment
in which there are few ramifications for cost growth and delays. Only
in very rare instances have programs been terminated for poor
performance. When DOD consistently allows unsound, unexecutable
programs to begin with few negative ramifications for poor outcomes,
accountability suffers. As section 204 proposes, the strengthening of
the Nunn-McCurdy provision--by including the potential termination of
programs that experience critical cost growth--could facilitate a
change in DOD's behavior by preventing the acceptance of unrealistic
cost estimates as a foundation for program initiation and placing more
accountability on senior DOD leadership for justifying program
continuation. Programs may thus be forced to be more candid and upfront
about potential costs, risks, and funding needs, and the likelihood of
delivering a successful capability to the warfighter at the cost and in
the time promised may grow.
Section 205: Organizational Conflicts of Interest
Prohibits systems engineering contractors from participating in the
development or construction of major weapon systems on which they are
advising DOD, and requires tightened oversight of organizational
conflicts of interest by contractors in the acquisition of major weapon
systems.
The defense industry has undergone significant consolidation in
recent years which has resulted in a few large, vertically integrated
prime contractors. This consolidation creates the potential for
organizational conflicts of interest where, for example, one business
unit of a large company may be asked to provide systems engineering
work on a system being produced by another unit of the same company. As
the Defense Science Board has recognized, these conflicts of interest
may lead to impaired objectivity, which may not be mitigated
effectively through techniques such as erecting a firewall between the
employees of the two units. While the Federal Acquisition Regulation
currently covers some cases of potential organizational conflicts of
interest involving the systems engineering function, there may be a
need for additional coverage in this area. In general, we would support
efforts to enhance the oversight of potential organizational conflicts
of interest, particularly in the current environment of a heavily
consolidated defense industry.
Section 206: Acquisition Excellence
Establishes an annual awards program to recognize individuals and
teams that make significant contributions to the improved cost,
schedule, and performance of defense acquisition programs.
We support the creation of an annual awards program to recognize
individuals and teams for improving the cost, schedule, and performance
of defense acquisition programs. We have reported that meaningful and
lasting reform will not be achieved until the right incentives are
established and accountability is bolstered at all levels of the
acquisition process. The need for incentives emerged as a significant
issue in our recent discussions with acquisition experts examining
potential changes to the acquisition processes enumerated in last
year's defense authorization act. The discussions revealed that those
changes may not achieve the desired improvement in acquisition outcomes
unless they are accompanied by changes in the overall acquisition
environment and culture, and the incentives they provide for success.
concluding observations on what remains to be done
A broad consensus exists that weapon system problems are serious
and that their resolution is overdue. With the Federal budget under
increasing strain from the Nation's economic crisis, the time for
change is now. DOD is off to a good start with the recent revisions to
its acquisition policy, which, if implemented properly, should provide
a foundation for establishing sound, knowledge-based business cases
before launching into development and for maintaining discipline after
initiation. The new policy will not work effectively, however, without
changes to the overall acquisition environment. Resisting the urge to
achieve the revolutionary but unachievable capability, allowing
technologies to mature in the science and technology base before
bringing them onto programs, ensuring that requirements are well-
defined and doable, and instituting shorter development cycles would
all make it easier to estimate costs accurately, and then predict
funding needs and allocate resources effectively. But these measures
will succeed only if the department uses an incremental approach.
Constraining development cycle times to 5 or 6 years will force more
manageable commitments, make costs and schedules more predictable, and
facilitate the delivery of capabilities in a timely manner.
Acquisition problems are also likely to continue until DOD's
approach to managing its weapon system portfolio: (1) prioritizes needs
with available resources, thus eliminating unhealthy competition for
funding and the incentives for making programs look affordable when
they are not; (2) facilitates better decisions about which programs to
pursue and which not to pursue given existing and expected funding; and
(3) balances the near-term needs of the joint warfighter with the long-
term need to modernize the force. Achieving this affordable portfolio
will require strong leadership and accountability. Establishing a
single point of accountability could help the department align
competing needs with available resources.
The department has tough decisions to make about its weapon systems
and portfolio, and stakeholders, including military Services, industry,
and Congress, have to play a constructive role in the process toward
change. Reform will not be achieved until DOD changes its acquisition
environment and the incentives that drive the behavior of its
decisionmakers, the military Services, program managers, and the
defense industry.
Mr. Chairman, this concludes my prepared statement. I would be
happy to answer any questions you may have at this time.
contacts and acknowledgements
For further information about this statement, please contact
Michael J. Sullivan (202) 512-4841 or sullivanm@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this statement. Individuals who made key
contributions to this statement include John Oppenheim, Charlie
Shivers, Dayna Foster, Matt Lea, Susan Neill, Ron Schwenn, and Bruce
Thomas.
Chairman Levin. Thank you so much, Mr. Sullivan.
Dr. Gansler?
STATEMENT OF HON. JACQUES S. GANSLER, CHAIRMAN, DEFENSE SCIENCE
BOARD TASK FORCE ON INDUSTRIAL STRUCTURE FOR TRANSFORMATION
Dr. Gansler. Mr. Chairman, members of the committee, thank
you very much for this honor of appearing before you at what I
think is a critical period and on such an important topic.
I don't have to tell this committee of the incredible
national security challenges that the United States is facing
in the 21st century, brought on by the rather dramatic world
changes that I believe require a new, holistic view of
security--DOD, Department of State, Department of Homeland
Security, Director of National Intelligence, and so forth--and
utilizing both hard and soft power, and addressing a very broad
spectrum of the security missions with great unpredictability
and covering the full spectrum, from terrorism all the way
through nuclear deterrence.
I would also emphasize that we need to take full advantage
of globalization of the technology of industry, not restricting
or gaming the benefits from globalization through restrictive
legislation.
In recognizing the long-term national security implications
of the global financial crisis, the need for energy security,
worldwide pandemics, the impact of climate change, growing
anti-globalization backlash, and the challenging U.S.
demographics. We have to do all of this, as Senator McCain
emphasized, in a likely fiscally-constrained budget
environment.
Now, to address these challenges, I believe four highly
interrelated acquisition issues must be addressed, and they
have to be addressed by both DOD and Congress. First, what
goods and services to buy; that's the requirements process.
Second, how to buy them; that's the acquisition reform. Third,
who does the acquiring; we have major issues in the acquisition
workforce. Fourth, from whom it is acquired; namely, the
industrial base.
Now, I wish I could tell you that there is some silver
bullet to address all of these needed changes, but it truly
requires a very broad set of initiatives in each of the four
areas if the Nation is to achieve the required 21st century
national security posture.
In my prepared testimony, which, Mr. Chairman, I would
appreciate being put into the record----
Chairman Levin. It will be.
Dr. Gansler.----I listed the required actions in each of
these four areas, and I'd be pleased to discuss any of these
with you at any time. However, for now let me summarize.
I believe this is a very critical period, perhaps somewhat
similar to the period following the launch of Sputnik or the
fall of the Berlin Wall. Today, the security world is changing
dramatically, especially since September 11--geopolitically,
technologically, threats, missions, warfighting, commercially,
et cetera--and this holistic perspective that I mentioned is
required. Moreover, a decade of solid budget growth, which I
believe will almost certainly change, has deferred the
difficult choices--for example, between more 20th century
equipment versus 21st century equipment--and the controlling
acquisition policies, practices, laws, and so forth, as well as
the Services' budgets and requirements priorities have not been
transformed sufficiently to match the needs of this new world.
In fact, there's still an emphasis on resetting versus
modernization.
Now, leadership is required to achieve the needed changes.
You look at the literature on culture change, which I think
this clearly is, two things are required to successfully bring
about the needed changes. First is the recognition of the need,
a crisis. In this case, I believe it is a combination of the
economic--the budget, if you will, crisis--and the changing
security needs, along with the shortage of the senior
acquisition experienced personnel to address these needs.
Second, leadership, with a vision, a strategy, and an action
plan. I honestly believe that President Obama, Congress, and
Secretary Gates support the needed changes; however, it's
pretty clear that the changes can be expected to be severely
resisted. Significant change always is.
I would start, as my highest priority, with the important
role of the service chiefs and secretaries in recognizing and
promoting senior acquisition personnel, military and civilian.
Over the last decade, the DOD acquisition workforce has been
greatly undervalued. DOD leadership now must demonstrate their
personal recognition of the critical nature of senior
experienced acquisition personnel and of the smart acquisition
practices that they would bring to America's military posture
in the 21st century.
As my second priority, I would emphasize the importance of
weapons costs as a true military requirement, to achieve
adequate numbers of weapons in a resource-constrained
environment. This will require enhanced systems engineering,
including cost-performance tradeoffs, throughout both the
government and industry, and incentives to industry for
achieving lower cost.
By the way, this has been done before; for example, with
the Joint Direct Attack Munition missile, where the Air Force
Chief of Staff said it should hit the target and cost under
$40,000 each. It now sells for under $20,000 and precisely hits
the targets.
Finally, as my third priority, I would emphasize the value
of rapid acquisition, from both its military and its economic
benefits, which will require the full use of spiral
development, with each block based on proven, tested technology
and continuous user and logistician feedback for the subsequent
block improvements, and with the option of continuous effective
competition, at the prime or at the sublevel. If they're not
continuously achieving improved performance at lower costs,
then they should be competed.
Achieving these required changes will take political
courage and sustained, strong leadership by both the executive
and legislative branches, working together. I hope, and firmly
believe, that it can be achieved. The American public, and
particularly our fighting men and women, deserve it, and the
Nation's future security depends upon it.
Thank you.
[The prepared statement of Dr. Gansler follows:]
Prepared Statement by Hon. Jacques S. Gansler \1\
---------------------------------------------------------------------------
\1\ Dr. Gansler is Professor and Roger C. Lipitz Chair, as well as
Director of the Center for Public Policy and Private Enterprise at the
School of Public Policy, University of Maryland. He served as Under
Secretary of Defense (Acquisition, Technology, and Logistics) from
1997-2001.
---------------------------------------------------------------------------
Mr. Chairman, members of the committee, thank you for the honor of
appearing before you at this critical period, and on such an important
topic.
I need not tell you that the U.S., in the 21st century, faces
incredible national security challenges--brought on by dramatic world
changes that require:
A new, Holistic View of Security (e.g., DOD, State,
DHS, DNI, etc.)--utilizing both ``hard'' and ``soft'' power
Addressing a Broad Spectrum of Security Missions--with
great unpredictability (from Terrorism to Nuclear Deterrence)
Taking full advantage of Globalization (of Technology,
Industry, etc.)
Recognizing the long-term national security
implications of:
The global financial crisis
The need for energy security
Worldwide pandemics
The impact of climate change
The growing anti-globalization backlash
The challenging U.S. demographics
To do all of this in a likely fiscally-constrained
budget environment
To address these challenges, four, highly-interrelated acquisition
issues must be addressed (by the DOD and Congress):
What goods and services to buy (the ``requirements''
process)
How to buy them (``acquisition reform'')
Who does the acquiring (the acquisition workforce)
From whom is it acquired (the industrial base)
I wish I could tell you that there was a ``silver bullet'' to
address the needed changes; but this truly requires a broad set of
initiatives in each of the four areas--if the Nation is to achieve the
required 21st century national security posture.
This need, for the four sets of broad changes, was emphasized in a
recent Defense Science Board report; where they found:
``DOD policies, processes, and management of the
Defense Acquisition Enterprise (broadly defined) impede the
transition to an effective, agile, and affordable overall,
joint military force for the 21st century.''
``U.S. Government policies, practices, and processes
do not facilitate the development, deployment, and support of
the innovative, affordable, and rapidly acquired weapons,
systems, and services needed for the 21st century forces.''
``The absence of many of the needed skills, (e.g.,
experienced program management, systems engineering, biotech,
advanced IT) in DOD's acquisition workforce, (particularly at
the senior military and civilian levels), combined with the
coming retirement and prior, large acquisition workforce
reductions, significantly impedes the development, production,
support, and oversight of the military capabilities needed for
the 21st century.''
``Government acquisition policies and industry trends
(e.g., further horizontal and vertical consolidations) will not
produce the required competitive, responsive, efficient, and
innovative National Security Industrial Base.''
So let me (very briefly) summarize the changes required in each of
the four, critical (and interrelated) areas: [in priority order within
each category]
What is acquired:
To meet the wide range of challenges, within a resource-constrained
environment, the Nation must focus on:
1. Lower cost systems and services
2. Optimized, net-centric systems-of-systems (vs. individual
``platforms'')
3. A ``reserve'' of resources to rapidly respond to combat
commanders' urgent needs
4. More ``balanced'' allocation of resources (to address
``irregular'' operations): C\3\ISR, unmanned systems, Special
Forces, ``Land Warriors,'' cyberdefense, etc; [and these
resources must be moved from the Supplementals into the base
budget]
5. Interoperability of ``joint'' systems; and coalition
systems
6. Planning, equipping, and exercising ``as we'll fight'':
with allies, multi-agencies, and ``contractors on the
battlefield''
How goods and services are acquired:
To achieve higher performance at lower costs and faster:
1. Require ``cost'' as a design/military ``requirement''
(because cost, in a resource-constrained environment, is
numbers)
2. Provide viable, continuous ``competition options'' (as the
incentive for higher performance at lower costs) e.g.
competitive prototypes, competitive split-buys, etc.
3. Fully utilize ``spiral development,'' with demonstrated
technologies (because it is lower cost, lower risk, faster to
field; maintains the option of competition; avoids
obsolescence; can respond rapidly to combat needs)
4. Make maximum use of commercial products and services (at
all levels--utilizing Other Transactions Authority; especially
at lower tiers)
5. Institutionalize a ``Rapid acquisition,'' parallel process
(to respond to COCOM urgent needs)
6. Create incentives for contractors to achieve desired
results (in cost, schedule, and performance)
7. Implement modern, integrated, enterprise-wide IT systems
(logistics, business, personnel, etc.)--including linking
Government and Industry
8. Address Conflict of Interest concerns (from
LSIMake/BuySETA); but don't reduce the value
of relevant experience
Who does the acquiring:
A flexible, responsive, efficient, and effective acquisition
program (for sophisticated, hi-tech goods and services) requires
``smart buyers.''
This depends on both quantity and quality of senior and experienced
military and civilian personnel (especially for expeditionary
operations). In the last decade-plus, this ``requirement'' has not been
met! In fact, the acquisition workforce declined on seniority and
quantity even as procurement appropriations increased.
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Therefore, one of the Nation's highest priorities (not just in the
DOD) must be to address the acquisition workforce.
The DOD, especially, has an acquisition workforce
problem:
Greatly reduced senior officers and SESs
In 1990 the Army had five general
officers with contract background; in 2007 they
had zero.
In 1995, the Air Force had 40 General
Officers in Acquisition, today 24; and in 1995,
87 SESs and today 49
The Defense Contract Management Agency
(25,000 people in 1990 down to 10,000 today;
and 4 General Officers to 0)
These reductions (due to the under valuing of the importance of the
acquisition workforce) introduce ``opportunities'' for ``waste, fraud
and abuse'' (e.g., 90 fraud cases under review from war zone; examples
of poor acquisition process results, such as the Air Force Tanker, the
Presidential Helicopter, etc.). These Government acquisition workforce
issues must be addressed. I believe that President Obama, Congress, and
Secretary Gates all agree on this (but it will take the priority
attention of the Service Chiefs and Secretaries to make it a priority).
From whom goods and services are acquired:
To quote, again, from the recent Defense Science Board study (on
the desired 21st century defense industry):
``The last two decades have seen a consolidation of the Defense
Industry around 20th Century Needs--The next step is DOD
leadership in transforming to a 21st Century National Security
Industrial Structure.''
The ``vision'' for this 21st Century National Security Industrial
Base (which appropriate government actions, i.e. acquisitions,
policies, practices, and laws, must incentivize and facilitate) are:
1. Efficient, responsive, technologically advanced, highly-
competitive (at all levels, including public and private
sectors)
2. Globalized (utilizing ``best in class'')--requires
significant changes to U.S. export controls (i.e., changes to
ITAR, EAR, etc.)
3. Healthy (profitable); and investing in IR&D and capital
equipment (rules should separate IR&D and B&P)
4. Includes commercial firms and equipment, and maximizes
dual-use facilities and workforce (barriers must be removed)
5. ``Independent'' systems-of-systems architecture and
systems engineering firms (to support the Government--as the
integrator)
6. Merger and Acquisition policy guidelines to be based on
this vision
7. Strong Government-Industry Communications encouraged
8. All non-inherently-governmental work to be done
competitively (public vs. private, for current government work)
9. Structural changes to eliminate appearance, or reality, of
conflict of interest (regarding ``vertical integration'')--but
great care to assure relevant--experienced firms and people
involved
In summary, I believe this is a critical period, perhaps similar to
the period following the launch of Sputnik or the fall of the Berlin
Wall. Today the security world is changing dramatically--especially
since September 11, 2001 (geopolitically, technologically, threats,
missions, warfighting, commercially, etc.)--and a holistic perspective
is required (including State, DHS, and DNI, as well as coalition
operations). Moreover, a decade of solid budget growth--which will
almost certainly change--has deferred difficult choices (between more
20th century equipment vs. 21st century equipment). The controlling
acquisition policies, practices, laws, etc. and the Services' budgets
and ``requirements'' priorities have not been transformed sufficiently
to match the needs of this new world (in fact, there is still an
emphasis on ``resetting'' vs. ``modernization'').
Leadership is required to achieve the needed changes! All of the
literature on ``cultural change'' (which this clearly must be) state
that two things are required to successfully bring about the needed
changes:
Recognition of the need (a ``crisis'')
In this case, I believe it is the combination of economic/budget
``crisis,'' the changing security needs, and the shortage of the senior
acquisition-experienced personnel to address the needs; and
Leadership--with a ``vision,'' a ``strategy,'' and an
``action plan.'' I believe that President Obama, Congress, and
Secretary Gates support the needed changes. However, the
changes can be expected to be severely resisted--significant
change always is!
I would start with the important role of the Service Chiefs and
Secretaries in recognizing, and promoting senior acquisition personnel
(military and civilian) in order to demonstrate their personal
recognition of the critical nature of smart acquisition practices to
American's military posture in the 21st century. As my second priority,
I would emphasize the importance of weapons costs as a military
requirement (to achieve adequate members of weapons in a resource-
constrained environment)--which will require enhanced systems
engineering (throughout both government and industry) and incentives to
industry for achieving lower cost systems. Finally, as my third
priority, I would emphasize the value of ``rapid acquisition'', for
both its military and economic benefits--which will require the full
use of ``spiral development'' (with each ``block'' based on proven/
tested technology, and continuous user and logistician feedback, for
subsequent ``block'' improvement--and with the option of effective
competition (at the prime and/or sub-level, if they are not
continuously achieving improved performances at lower and lower costs).
Achieving these required changes will take political courage and
sustained, strong leadership--by both the executive and legislative
branches (working together). I hope, and firmly believe, it can be
achieved. The American public, and particularly, our fighting men and
women, deserve it--and the Nation's future security depends upon it.
Thank you.
Chairman Levin. Dr. Gansler, thank you so much.
Dr. Kaminski?
STATEMENT OF HON. PAUL G. KAMINSKI, CHAIR, COMMITTEE ON PRE-
MILESTONE A SYSTEMS ENGINEERING, AIR FORCE STUDIES BOARD,
NATIONAL RESEARCH COUNCIL
Dr. Kaminski. Mr. Chairman, Senator McCain, and members of
the committee, first of all, I want to thank you for your
leadership on these critical acquisition issues and for the
invitation to testify.
Since you've asked me to testify, first, in my role as
Chairman of the National Research Council's Study on Pre-
Milestone A Systems Engineering, with your permission I would
ask that my statement, which includes a full summary, be put in
the record, and then I will proceed to provide a short verbal
summary of the summary.
Chairman Levin. All the statements will be made part of the
record. Thank you.
Dr. Kaminski. Thank you, sir.
Recent years have seen a serious erosion in our ability to
field new weapon systems quickly in response to changing
threats, as well as a large increase in the cost of these
weapon systems. Our programs today for developing weapon
systems take two to three times longer than they did 30 years
ago. I note that time is money in this process, and time also
leaves room for disruptions, uncertainty, and changes in
commercial technology. In a 15- or 20-year period, we're seeing
commercial technology turnover three, four, or five times. When
a weapon system takes 15 to 20 years to develop, the technology
that you start with isn't going to be supported when it's
fielded. So, we have to vigorously attack this time issue.
Our committee also noted the importance of systems
engineering in reducing this acquisition time, when combined
with development planning. We further underscored the
importance of an early systems engineering effort, in that, the
decisions made prior to and the key Milestone A and B decisions
impact somewhere between 75 percent and 85 percent of the total
life-cycle cost. So, the time to address those issues is up
front, before those decisions are made.
Our committee also noted that many of the conclusions that
we reached have been reached in several previous studies. So,
the issue isn't disagreement on what the recommendations are,
the issue is implementing those recommendations. So, once
again, we thank you for your leadership in creating a forum for
that kind of implementation.
Let me address now the issues that you asked me to address.
First of all, just one overall comment on systems
engineering. I agree with Secretary Gates, who, when asked
about acquisition, said, ``There is no one silver bullet that
is going to correct all the problems.'' But, I do believe that
good systems engineering, coupled with effective development
planning early on, are two of the most important contributors
to successful acquisition.
Our report provided some formal definitions of ``systems
engineering,'' but they tend to be arcane, so I thought I might
start with a couple of examples. I'll briefly describe some
examples of good systems engineering in the work we've done,
and also where have we seen poor systems engineering.
One of the really good examples is the Apollo program. That
program, from a dead start, put man on the moon in about 8
years. When that program was started, we didn't have mature
technology. What we did was good upfront systems engineering
and development planning, so we could proceed in a sequential
way, step by step, with each new step building on the previous
step. In building hardware, we were also building the
experience of our acquisition workforce and our industry, so we
could, step by step, increase our capabilities, eventually
going to the moon.
Another really good example is the Air Force
Intercontinental Ballistic Missile (ICBM) programs that were
done in the 1970s and 1980s. What we saw there is that we would
never start a full-scale development contract for a new ICBM
until we had done the upfront systems engineering and
development planning. The development planning produced an
inertial guidance system for the ICBM, as well as critical
propulsion components, and a reentry vehicle. That not only
reduced the risk of the hardware development and integration in
the future, it also gave domain experience to our key people in
government and industry, so that when we threw the switch and
started full-scale development, we could typically expect a
first flight in 3 to 4 years, as a result of that experience
base. That's what we need to restore.
You asked: ``What were systemic contributors to acquisition
problems?'' I listed five.
The first of these is the lack of this early and continuing
systems engineering, coupled to a development planning program
early on, right upfront.
The second key impediment is the lack of alignment of
responsibility, authority, and accountability of the program
manager. A program manager needs to be able to exercise his or
her judgment. Much of the program manager's authority has been
taken away by one-size-fits-all approaches to acquisition and
by the oversight process, which has some onerous elements that
are nonvalue-added.
A third major impediment is the lack of stability in
program funding. Many contribute to that.
A fourth is the lack of early attention to test and
evaluation, with insufficient planning and investment in the
tools, such as modeling and simulation, test equipment,
facilities, and personnel, to provide us with the timely and
meaningful results needed by program management and for
continuous systems engineering to refine our performance
objectives and development plans.
Finally, the root fundamental issue here is this excessive
time to acquire that I had spoken about. Time is money. As this
time increases from a few years in the past to 15 years today,
it undermines our entire process, causing the key participants
to lose what I call the ``recipe'' for how we move forward and
also to lose a sense of accountability. When we see new
capabilities that are developed and fielded in 5 years, the
engineers, the managers, the testers, the cost analysts are all
able to benefit and apply their experience from previous
programs, and they can also be held accountable, since they can
be in place managing the programs deliverables during one
assignment. That all changes when we move to 15-year
acquisitions and we have five rollovers of management,
engineers, and cost analysts, and five rollovers of the
technology in the process.
So, attacking acquisition time is fundamental. I would say,
a testament to our failure today is the fact that we have to
discard our current acquisition approaches to deal with our
urgent needs and field systems, such as the Mine Resistant
Ambush Protected vehicle and counter-improvised explosive
devices (IEDs) by forming and using rapid-reaction
organizations, because our existing ones don't work. They can't
respond to the cycle time that we need.
So, what do we do about this? Again, I've listed in my
statement five steps.
The first is to ensure that we not only restore, but
enhance, this early and continuing systems engineering work,
coupled with development planning. This means restoring funding
upfront in the programs, and using independent estimates to
ensure we have enough funding upfront. It also means attracting
best and brightest to the critical systems engineering work,
and providing a path to career advancement, career tracking,
and leadership for the key people that we need to rebuild in
cadre.
Second issue is the alignment of the responsibility,
authority, and accountability of the program manager. I've
listed several steps in my statement about what's needed to be
done to do that.
Third issue is improving funding stability. We pay a great
deal for the instability we cause by making funding adjustments
to program. My experience shows that every time we make a cut
in a program, for financial or other reasons unrelated to
performance, we end up eventually putting in three times what
we cut to restore the program later and get it back to a base.
Fourth item is giving early and serious attention to the
test and evaluation issues that I noted earlier, so they can be
part of a rapid process. When we wait for test and evaluation
results because we haven't done a good job planning and
preparing, what we have is hundreds of people sitting on their
hands, waiting for results, and we're paying all those people
while they wait for results.
Finally, the last item is fundamentally attacking this
problem of long development times by the combination of the
previous four items.
I believe action on these five issues will have a
significant and demonstrable impact on our serious acquisition
problems. I believe that we need to move now with the same
urgency and priority that we expect in combat operations to
permit the timely and effective development and fielding of new
capabilities and services with what I expect will be more
limited future defense dollars.
Thank you, Mr. Chairman.
[The prepared statement of Dr. Kaminski follows:]
Prepared Statement by Dr. Paul G. Kaminski
Chairman Levin, and Ranking Member McCain:
Thank you for your leadership on the Department of Defense (DOD)
acquisition, and for the invitation to testify on these important
acquisition issues. Since you have asked me to testify in my role as
Chair, Committee on Pre-Milestone A Systems Engineering, Air Force
Studies Board, National Research Council, I will begin by providing a
summary of our report, which was approved by the Governing Board of the
National Research Council and published in 2008. The report is
available to the public at http://www.nap.edu. After the report
summary, I will provide my personal views on systems engineering and
respond to the key issues you asked that I address.
SUMMARY
Recent years have seen serious erosion in the ability of U.S.
forces to field new weapons systems quickly in response to changing
threats, as well as a large increase in the cost of these weapons
systems. Today the military's programs for developing weapons systems
take two to three times longer to move from program initiation to
system deployment than they did 30 years ago. This slowdown has
occurred during a period in which threats have been changing more
rapidly than ever and when technology advances and accumulated
experience should have been accelerating rather than slowing the
development process.
Many causes for this trend have been suggested, including the
increased complexity of the tasks and the systems involved from both
technological and human/organizational perspectives; funding
instability; loss of ``mission urgency'' after the end of the Cold War;
bureaucracy, which increases cost and schedule but not value; and the
need to satisfy the demands of an increasingly diverse user community.
The difficulty of focusing on a specific, homogeneous, post-Cold War
threat made problems even worse. Yet although the suggested causal
factors have merit, a common view is that better systems engineering
(SE) and development planning could help shorten the time required for
development, making it more like what it was 30 years ago.
Simply stated, SE is the translation of a user's needs into a
definition of a system and its architecture through an iterative
process that results in an effective system design. SE applies over the
entire program life cycle, from concept development to final disposal.
The Committee on Pre-Milestone A Systems Engineering was tasked by
the U.S. Air Force to examine the role that SE can play during the
defense acquisition life cycle in addressing the root causes of program
failure, especially during the pre-Milestone A and early phases of a
program. Currently, few formal SE processes are applied to Air Force
development programs before the Milestone A review.\1\
---------------------------------------------------------------------------
\1\ This is a result of the elimination in the 1990s of the
development planning function that had existed in the Air Force Systems
Command.
---------------------------------------------------------------------------
The committee devoted considerable time and space in its report to
trying to define a minimum set of systems engineering processes. The
most important of these processes are summarized in the checklist in
Box S-1 below. A few of the things that need to be taken care of before
Milestone A and just after it are the following: the consideration of
alternative concepts (solutions) up front; the setting of clear,
comprehensive key performance parameters (KPPs) and system
requirements; and early attention to interfaces and interface
complexity, to the concept of operations, and to the system
verification approach. It is these early-stage processes that are
covered in this report. The importance of stable requirements and
funding between Milestone B and the achievement of initial operational
capability (IOC) is stressed, as are processes including good
configuration management and change control. The committee further
stresses in the report what it regards as six of the most important
process areas in its discussion of six ``seeds of failure''.
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Figure S-1 DOD life-cycle acquisition process. Points A, B, and C
at the top of the figure represent Milestones A, B, and C. LCC, life-
cycle cost. Source: Richard Andrews, 2003, An Overview of Acquisition
Logistics. Fort Belvoir, VA; Defense Acquisition University. Available
at http://www.afcea.org/events/pastevents/documents/
Track4Session4AMCEmphasisonCustomerFocusedITInitiatives.ppt#364,12,Slide
12. Last accessed on November 20, 2007.
SYSTEMS ENGINEERING AND THE DOD ACQUISITION LIFE CYCLE
The use of formal systems engineering practices throughout the life
cycle of an acquisition program is critical to fielding the required
system on time and within budget. Across the top of Figure S-1 are the
points at which important management decisions are made: Milestones A,
B, and C. Concept development and refinement occur before Milestone A,
and further technology development, to reduce system design and
development (SDD) risk, occurs before Milestone B. Only after Milestone
B does a program become an enterprise with dedicated funding.
Importantly, Figure S-1 shows that about three-quarters of total system
life-cycle costs are influenced by decisions made before the end of the
concept refinement phase at Milestone A, while about three-quarters of
life-cycle funds are not actually spent until after Milestone C. This
means that although high-quality SE is necessary during the entire
acquisition cycle, the application of SE to decisions made in the pre-
Milestone A period is critical to avoiding (or at least minimizing)
cost and schedule overruns later in a program. Much of the value of
early, high-quality SE will be manifested as success in fulfilling
Milestone B requirements.
MAIN FINDINGS AND RECOMMENDATIONS
The committee's main findings and recommendations are given below.
Finding
Attention to a few critical systems engineering processes and
functions particularly during preparation for Milestones A and B is
essential to ensuring that Air Force acquisition programs deliver
products on time and on budget.
Today's weapons systems provide unprecedented capabilities but also
involve complex interfaces with external command, control, and
communications systems and rely on a greater volume of software than
ever before. Early decisions on the weapons system requirements and
capabilities have a disproportionately large impact on program cost and
schedule. The committee also recognizes that a lack of flexibility (a
result of overly rigid processes or a lack of trust among program
participants or stakeholders) can limit the ability of a program
manager to change early decisions that warrant changing.
The committee found many gaps and inconsistencies in the way that
the Air Force manages pre-Milestone A activities. The committee heard
from presenters of some cases for which required documents were
completed pro forma and filed away, never to be seen again, or for
which required steps were skipped completely. The current practice of
initiating programs at Milestone B denies the acquisition review
authority the earlier opportunity (at Milestone A) to make judgments
about the maturity of the technologies on which the program is based
and to decide whether technologies need to be further developed prior
to making a Milestone B commitment to system development and
demonstration.
Recommendation
The Air Force leadership should require that Milestones A and B be
treated as critical milestones in every acquisition program and that a
checklist such as the ``Pre-Milestone A/B Checklist'' suggested by the
committee (see Box S-1 in this Summary) be used to judge successful
completion.
A rigorous, standard checklist of systems engineering issues should
be addressed by each program through both the pre-Milestone A and pre-
Milestone B phases. The committee's recommended 20-item checklist is
shown in Box S-1.
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
While the committee considers that each item on the checklist is
important, it calls attention to several items that warrant further
discussion. Item 2 recognizes that the world changes too fast to be
friendly to long development cycles. The committee believes that the
Air Force should strive to structure major development programs so that
initial deployment is achieved within, say, 3 to 7 years. Thirty years
ago, this was a typical accomplishment--for example, nearly 40 years
ago, the Apollo program put the first man on the Moon in fewer than 8
years.
The development time issue is addressable by applying systems
engineering to Items 3, 4, and 13 through 15 before Milestones A and B.
The definition of clear KPPs by Milestone A and clear requirements by
Milestone B that can remain stable through IOC can be essential to an
efficient development phase. It is also important that critical
technologies be sufficiently mature prior to starting SDD. The
committee observed that although today's systems are not necessarily
more complex internally than those of 30 years ago, their ``external
complexity'' often is greater, because today's systems are more likely
to try to meet many diverse and sometimes contradictory requirements
from multiple users. This kind of complexity can often lead to
requirements being changed between Milestone B and IOC, and it can lead
to relying on immature technology.
Item 19 of the checklist stresses the importance of placing
experienced, domain-knowledgeable managers in key program positions.
The committee has observed that many of the truly extraordinary
development programs of the past, such as Apollo, the Manhattan
Project, the early imaging satellite programs, the U-2, the fleet
ballistic missile system, and nuclear submarines, were managed by
relatively small (and often immature) agencies with few established
processes and controls. In that environment, dedicated managers driven
by urgent missions accomplished feats that often seem incredible today.
The committee believes that the accumulation of processes and
controls over the years--well meant, of course--has stifled domain-
based judgment that is necessary for timely success. Formal SE
processes should be tailored to the application. But they cannot
replace domain expertise. In connection with item 19, the committee
recommends that the Air Force place great emphasis on putting seasoned,
domain-knowledgeable personnel in key positions--particularly the
program manager, the chief system engineer, and the person in charge of
``requirements''--and then empower them to tailor standardized
processes and procedures as they feel is necessary.
One key pre-Milestone A task is the analysis of alternatives (AoA),
which entails evaluating alternative concepts and comparing them in
terms of capabilities, costs, risks, and so on. Checklist items 1
through 4, 12, and 13 should be completed before the AoA, while items 5
through 11 and 14 through 20 may be addressed after the AoA.
Finding
The creation of a robust systems engineering process is critically
dependent on having experienced systems engineers with adequate
knowledge of the domain relevant to a contemplated program.
While the systems engineering process has broad use, effective
application depends on having domain experts who are aware of what has
gone wrong (and right) in the past, recognize the potential to repeat
the successes under new circumstances, and avoid repeating the errors.
Ideally, a person or persons with domain knowledge would have had
experience working on exactly the same problem, or at least a problem
related to the one at hand. If that is not so (and it might not be if
the problem has never been addressed before, as was the case for Apollo
and nuclear submarines), the term could be taken to refer to academic
training in the relevant field of engineering or science. It could also
refer to the practice of critical thinking and problem solving that
comes with learning to be a systems engineer and then building on that
foundation to gain the experiential knowledge and understanding of
engineering in the context of an entire system. Systems engineering is
enabled by tools that have been developed to assist in the management
of systems engineering (not to be confused with the practice of systems
engineering).
Both industry and Air Force presenters told the committee that
there are not enough domain-knowledgeable and experienced systems
engineers to support all of the programs that need them.
Recommendation
The Air Force should assess its needs for officers and civilians in
the systems engineering field and evaluate whether either its internal
training programs, which include assignments on Air Force programs that
provide mentoring by experienced people and hands-on experience in the
application of systems engineering principles, or external
organizations are able to produce the required quality and quantity of
systems engineers and systems engineering skills. Based on this
assessment, the Air Force first should determine how and where students
should be trained, in what numbers, and at what cost, and then
implement a program that meets its needs.
The Air Force needs to attract, develop, reward, and retain systems
engineers across the full spectrum of relevant domains, engage them in
the early (pre-Milestone A) phase of new programs (or modification
programs), and sustain their participation throughout the life of the
programs. One important step in this process would be to create an Air
Force occupational code for systems engineering so that engineers'
experience and education can be tracked and managed more effectively.
The Air Force should support an internal systems engineering career
track that rewards the mentoring of junior systems engineering
personnel, provides engineers with broad systems engineering
experience, provides appropriate financial compensation to senior
systems engineers, and enables an engineering career path into program
management and operations.
Finding
The Government, federally Funded Research and Development Centers
(FFRDCs), and industry all have important roles to play throughout the
acquisition life cycle of modern weapons systems.
Since the need for a new or upgraded weapons system is most often
first recognized by the military user, it is appropriate for the
military to codify its requirements and, with support from FFRDC and
independent systems engineering and technical assistance contractors,
to explore materiel and nonmateriel solutions (such as doctrinal,
organizational, or procedural changes) as well as to assess the
potential for new technology to provide enhanced capabilities. While it
is appropriate and usually desirable to engage development contractors
in the pre-Milestone B process using competitive study contracts, the
source selection for system development and demonstration should not be
made until after the work associated with Milestones A and B is
complete.
Recommendation
Decisions made prior to Milestone A should be supported by a
rigorous systems analysis and systems engineering process involving
teams of users, acquirers, and industry representatives.
Working together, government and industry can develop and explore
solutions using systems engineering methodology to arrive at an optimal
systems solution.
Finding
The Air Force used to have a development planning organization that
applied pre-Milestone A systems engineering processes to a number of
successful programs, but that organization was allowed to lapse.
The role of the Air Force development planning organization, which
was within the Air Force Systems Command, was to provide standard
evaluation tools and perform pre-Milestone A systems engineering
functions across acquisition programs. The early 1990s saw an erosion
of this front-end planning organization along with its funding as the
Air Force Systems Command (now the Air Force Materiel Command) began to
play a decreasing role in program execution. In the opinion of several
speakers who met with the committee, one main reason for the erosion of
funding was a lack of congressional support for the planning function.
Recommendation
A development planning function should be established in the
military departments to coordinate the concept development and
refinement phase of all acquisition programs to ensure that the
capabilities required by the country as a whole are considered and that
unifying strategies such as network-centric operations and
interoperability are addressed.
The Air Force and the other military services should establish a
development planning organization like that which existed in the early
1990s.
The roles and functions of the various organizations involved in
acquiring major weapons systems need to be clearly defined. The
responsibility for executing systems engineering and program management
in the pre-Milestone A and B phases should be vested in the military
departments that do the actual development planning functions. This
should not be the responsibility of the Office of the Secretary of
Defense (OSD) or of the Joint Staff. Instead, those offices need to
enable the creation and functioning of military department development
planning organizations with policy measures and, where appropriate,
resources. The Joint Staff, under the auspices of the Joint
Requirements Oversight Council, may help to define the requirements for
major programs in the course of the development planning process, but
it should not run the process itself.
The existence of ``joint'' programs or a program such as Missile
Defense, which has several related systems being developed by different
military services, requires clear guidance from both OSD and the Joint
Staff about who is in charge. These programs need to be harmonized and
integrated by the responsible integrating agency. However, development
planning activities should still take place in the military departments
where the expertise resides. Consequently, the development planning
should be managed by that agency.
While this committee cannot predict how Congress will view the
revival of a good planning process to support pre-Milestone A program
efforts, it is still important for the Air Force and DOD to make the
case for the critical importance of this process before Congress and
others. A development planning process is important not to start new
programs, but rather to ensure that any new program (or a new start of
any kind) is initiated with the foundation needed for success. Funding
for this planning function needs to be determined by the military
services, including both the acquisition communities and those (the
warfighters) who generate the operational requirements.
CONCLUDING THOUGHTS
Many of the conclusions reached and recommendations made by the
committee are similar to those of previous reviews. Most of the past
recommendations were never implemented, so one of this committee's most
critical thoughts relates to the importance of implementation. A
sampling of key findings and recommendations from previous studies
follows:
Government Accountability Office (GAO) 2,3
---------------------------------------------------------------------------
\2\ GAO, 2003, Defense Acquisitions: Improvements Needed in Space
Systems Acquisition Management Policy, September. Available at http://
www.gao.gov/new.items/d031073.pdf. Last accessed April 2, 2007.
\3\ GAO, 2005, Space Acquisitions: Stronger Development Practices
and Investment Planning Needed to Address Continuing Problems, July.
Available at http://www.gao.gov/new.items/d05891t.pdf. Last accessed
April 2, 2007.
Separate technology development from systems
acquisition. Commit to a program only if the technology
is sufficiently mature. Set the minimum Technology
Readiness Level (TRL).
Stabilize the requirements early.
Employ systems engineering techniques before
committing to product development.
Employ evolutionary approaches that pursue
incremental increases in capability.
Address shortfalls in science, engineering,
and program management staff.
National Defense Industrial Association (NDIA) \4\
---------------------------------------------------------------------------
\4\ NDIA Systems Engineering Division, 2003, Task Report: Top Five
Systems Engineering Issues in Defense Industry, January, Arlington, VA:
NDIA.
Increase SE awareness and recognize SE
authority in the program formulation and decision
process.
Incentivize career SE positions within the
government.
Defense Science Board (DSB) \5\
---------------------------------------------------------------------------
\5\ DSB/Air Force Scientific Advisory Board (AFSAB) Joint Task
Force, 2003, Acquisition of National Security Space Programs, May,
Washington, DC.
---------------------------------------------------------------------------
Overhaul the requirements process.
Stabilize acquisition tours.
Establish a robust SE capability.
Defense Acquisition Performance Assessment (DAPA) \6\
---------------------------------------------------------------------------
\6\ Ronald Kadish, Gerald Abbott, Frank Cappuccio, Richard Hawley,
Paul Kern, and Donald Kozlowski. 2006. Defense Acquisition Performance
Assessment. Available at http://www.acq.osd.mil/dapaproject/documents/
DAPA-Report-web/DAPA-Report-web-feb21.pdf. Last accessed on April 2,
2007.
Strategic technology exploitation is a key
U.S. advantage. Opportunities need to be identified
early.
The U.S. economic and security environments
have changed--for example, there are fewer prime
contractors, smaller production runs, reduced plant
capacity, fewer programs, and unpredictable threats.
The acquisition system must deal with
instability of external funding.
The DOD management model is based on a lack of
trust. Quantity of review has replaced quality. There
is no clear line of responsibility, authority, or
accountability.
Oversight is preferred to accountability.
Oversight is complex, not process- or program-
focused (as it should be).
The complexity of the acquisition process
increases costs and draws out the schedule.
Incremental improvement applied solely to the
``little a'' acquisition process \7\ requires all
processes to be stable--but they are not.
---------------------------------------------------------------------------
\7\ The Acquisition--``Big A''--system is often believed to be a
simple construct that efficiently integrates three independent
processes: requirements, budgeting, and acquisition. ``Little a'' on
the other hand, refers to the acquisition process that focuses on ``how
to buy'' in an effort to balance cost, schedule, and performance; it
does not include requirements and budgeting.
The committee notes that successful implementation of these
recommendations requires the ``zipper concept''--making connections at
all levels, from the senior leadership of the Air Force and DOD down to
the working levels within key program management offices and
supervisory staffs.
committee on pre-milestone a systems engineering: a retrospective
review and benefits for future air force systems acquisition
Paul Kaminski, Chair, Technovation, Inc.
Lester Lyles, Vice Chair, U.S. Air Force, (Ret.)
Dev Banerjee, The Boeing Company
Thomas Blakely, Lockheed Martin Aeronautics Company
Natalie Crawford, RAND Corporation
Stephen Cross, Georgia Tech Research Institute
Gilbert Decker, Independent Consultant
Llewelln Dougherty, Raytheon
John Farr, Stevens Institute of Technology
James Frey, Frey Associates
Robert Fuhrman, Lockheed Martin (Ret.)
David Gorney, The Aerospace Corporation
John Griffin, Griffin Consulting
Wesley Harris, Massachusetts Institute of Technology
Ronald Kadish, Booz Allen Hamilton
Robert Latiff, Science Applications International Corporation
Alden Munson, Independent Consultant
Mark Wilson, Mark Wilson Consulting
Staff
Michael A. Clarke, Board Director
James C. Garcia, Study Director
Greg Eyring, Senior Program Officer
Carter W. Ford, Associate Program Officer
William E. Campbell, Senior Program Associate
Lanita Jones, Program Associate
Having summarized the findings and recommendations of the Committee
on Pre-Milestone A Systems Engineering, let me now add my personal
views on systems engineering, and the two additional questions that you
asked me to address: 1) the systemic issues that have contributed to
cost, schedule, and performance problems in the acquisition of major
weapon systems; and 2) the steps that Congress and the Department need
to take to improve performance of the Department's acquisition
programs.
SYSTEMS ENGINEERING
I agree with Secretary Gates who said that there is no one silver
bullet that will correct all of the DOD acquisition problems. But I
believe that good systems engineering coupled with effective
development planning are the two most important contributors to
successful acquisition. Our report provided formal definitions of
systems engineering and development planning that are somewhat arcane.
So rather than provide further definition, I find it easier to
illustrate by choosing examples of good and bad systems engineering and
development planning. Examples of good work include the Apollo Program
and the U.S. Air Force intercontinental ballistic missile (ICBM)
programs (e.g., Minuteman, MX) in the 1970s-1980s. Apollo succeeded in
putting men on the moon in about 8 years. At the start of the program,
almost all of the key technologies were immature. But good systems
engineering and development planning were applied to develop a
systematic approach, reducing risk by taking a series of limited steps,
and applying the learning and domain experience gained from each step
to the subsequent step. The U.S. Air Force ICBM programs used a similar
approach, beginning with conceptual studies and technology development,
and holding initiation of full scale development (FSD) contracts until
key guidance system, re-entry system and propulsion technologies had
been demonstrated. As a result, the time from initiation of FSD until
first flight was typically 3-4 years. An example of poor systems
engineering is the SBIRS program, in which a lack of domain experience
and analysis led to a failure to anticipate the possibility of severe
radio frequency interference between two key payloads--discovering this
problem years after program initiation. Inadequate systems engineering
and development are therefore the first of five items listed below as
systemic contributors to acquisition problems.
SYSTEMIC CONTRIBUTORS TO ACQUISITION PROBLEMS
1. The lack of early and continuing systems engineering and
the absence of a closely-coupled development planning program
are a fundamental contributor, as identified in our report. The
root causes include: (a) lack of sufficient personnel (in both
government and industry) with adequate education, training and
domain experience (this includes personnel in requirements
development as well as in acquisition); and (b) lack of
sufficient front end investment necessary to understand the key
tradeoffs in cost/schedule/performance, and to identify and
address the key risks in a systematic manner.
2. Lack of alignment of responsibility, authority, and
accountability of the program manager. In many cases the
program manager's authority is diffused by many levels of
oversight in both the Department and in Congress, and the
financial and performance constraints imposed do not allow
sufficient freedom of action to apply informed judgment in a
timely manner. Flexibility is further limited by application of
a ``one-size-fits-all'' approach imposed by the DOD 5000
system, and the oversight practiced by the DOD and Congress. A
program manager needs the freedom to tailor the acquisition
approach to the problem, to ensure that the program response
time will fall within the response time of the threat, and to
apply a variety of tools and techniques (such as the use of
prototypes, competitive prototypes, modeling and simulation,
critical subsystem and component demostration). For this to
work, we need program managers with the education, training,
and domain experience needed to enable timely responses and
excellent judgment relevant to the domain.
3. Lack of stability in program funding.
4. Lack of early attention to test and evaluation, with
insufficient planning and investment in the tools (e.g.,
modeling and simulation, test equipment, facilities, and
personnel) to provide the timely and meaningful results needed
by program management and systems engineering to continually
refine performance objectives and development plans.
5. Excessive (and growing) time from program initiation to
fielding. As this time increases from a few years to 15 years
or more, it undermines the entire acquisition process by
causing key participants to ``lose the recipe'', and lose a
sense of accountability as well as a sense of being able to
make a difference. When new capabilities are developed and
fielded in 5 years, engineers, managers, testers, cost
analysts, etc. are able to benefit from and apply the
experience gained from a previous program or program phase.
They can also see the results of their decisions and be held
accountable. We can also meaningfully employ past performance
of the contractor as a factor in the award of future programs--
an important factor in incentivizing contractor performance.
This all changes dramatically when the time extends to 15
years, and we have five roll-overs of management, engineers,
cost analysts, and commercial technology during this time
period. This long and growing time period is a result of the
inflexibility inherent in our entire system of requirements
development, budgeting and acquisition, and it creates a
vicious cycle in which it further exacerbates the contributors
above, and they in turn further increase the time and cost
growth. We see the result when we must discard our current
acquisition system in order to deal with urgent needs and field
systems such as MRAP and jammers to counter IED's by forming
and using rapid reaction organizations. This cycle must be
broken by attacking the root causes.
steps that congress and the department need to take
1. The first step is to insure that we not only restore, but
enhance early and continuing systems engineering coupled with
effective development planning. This will require commitment of
more significant investment dollars earlier in our acquisition
programs, and a commitment to build a cadre of systems
engineers and development planners with the education, training
and domain experience needed to be effective. Attracting ``best
and brightest'' to this work--and keeping them--will require a
personnel system that will identify and track these important
human resources and establish a career path to allow those who
are successful to advance to senior program management and
leadership positions. Their domain experience will be enhanced
by managing the building of critical subsystems during the
development planning program, reducing risk and building skills
and experience at the same time. Congress and the Department
can assist by providing incentives for attracting and keeping
key personnel (not only financial incentives, but educational,
training, recognition, and most important--the opportunity to
take on challenging developments and see that they can make a
difference). We will need metrics to assess how well we are
doing in building and applying this cadre, and we must
recognize that this will not be accomplished in 4, or even 8
years. But we must begin in earnest and begin now. Finally, we
need a means to insure that we have adequate funding upfront
for new programs; one approach would require a report at
program initiation from an independent cost analyst working
with system engineers and development planners who have
developed their skills on previous programs.
2. Alignment of the responsibility, authority, and
accountability of the program manager requires that a degree of
trust be established between the program manager and those
responsible for our oversight mechanisms. We must be prepared
to delegate authority to the program manager, and provide him
or her with some flexibility to manage--to adjust levels and
allocation of funding, to adjust the allocation of performance
parameters, to adjust schedule, and to tailor the acquisition
approach to be responsive to the need. Clearly, there must be
bounds established beyond which the program manager must seek
approval from oversight authorities. But I believe these bounds
are too narrow and inflexible today. One size does not fit all
programs. Congress and the Department should be willing to
consider and tailor many of the restrictions which
unnecessarily limit and delay program managers today. I have
seen many of our successful classified special programs benefit
from greater management flexibility than that afforded to their
in conventional program counterparts. The good managers of
these special programs have used that flexibility to the
benefit of the program and the Department by operating with
transparency and maintaining trust. I realize that it seems
counterintuitive to recommend greater flexibility and trust in
an environment rife with acquisition problems, but I believe we
need to break the current cycle. One way to begin is with a
limited number of pilot programs, with first priority to those
programs addressing urgent needs, and assignment of our most
experienced program managers to meet those urgent needs. Since
these programs will be moving with dispatch, they offer the
best opportunity to produce early indications of whether this
is a sound approach which should be extended to other programs.
3. Improving funding stability will require that the
Department and Congress be willing to give up some of their
flexibility in making annual (or more frequent) adjustments in
funding. Doing so will require tradeoffs of the costs and
benefits, and I believe it is time to make explicit
consideration of these tradeoffs. I have seen the projected
benefits of stable funding by looking at theoretical Monte-
Carlo simulations (which show efficiency improvements of
perhaps 8-10 percent as a result of holding a small capital
Reserve of less than 10 percent). We can also see the benefits
of multiyear procurements saving similar or greater amounts. I
have also seen many examples in which funding cuts of x dollars
today result in later additions of 3x dollars to catch up.
4. Giving early and serious attention to test and evaluation
will require strengthening our test and evaluation
organizations and personnel. Test and evaluation is often an
afterthought, and contracts are often written without any
mention of how we will test the product. We spend large amounts
of money when a large development team waits for test results.
The alternative is to spend less money and time by considering
testing and investment in test resources as part of our systems
engineering and development planning efforts. The actions
recommended in paragraph 1 above are the same actions required
to address these critical test and evaluation needs.
5. Reducing the time from program initiation to fielding will
require the combination of all actions suggested above. Further
benefits will be derived by placing more emphasis on time-
certain acquisition. This will be helped by better development
planning and alignment of incentives. With good development
planning, we can assign managers to develop prototypes,
critical systems or components needed to better understand
cost/performance trades and reduce risk. It is reasonable to
expect that many of these developments can be completed in 2-4
years, so one manager will be in place from start to delivery.
This will help align authority and accountability in both
government and industry. As these critical subsystems are
delivered and tested, the risk reduction and domain experience
gained in both government and industry will allow us to reduce
the time required to develop, integrate and test the full
system. We can also apply meaningful incentive programs to link
profits to demonstrated performance, and use that performance
as a factor in making future competitive awards. We can rely on
the experience gained during development planning to apply
informed judgment to adjust requirements to improve value,
reduce time, and better estimate and manage costs. The
Department and Congress can assist by placing more emphasis on
time-certain acquisition, with the opportunity for milestone
reviews at the completion of major development planning
activities.
I believe action on these five issues will have a significant and
demonstrable impact on our serious acquisition problems. We need to
move now with the same urgency and priority that we expect in combat
operations to permit the timely and effective development and fielding
of new capabilities and services with what I expect will be more
limited future defense dollars.
Chairman Levin. Thank you so much, Dr. Kaminski.
Mr. Adolph?
STATEMENT OF CHARLES E. (PETE) ADOLPH, CHAIRMAN, DEFENSE
SCIENCE BOARD TASK FORCE ON DEVELOPMENTAL TEST AND EVALUATION
Mr. Adolph. Chairman Levin, Senator McCain, members of the
committee, I'd like to thank you for inviting me today.
I chaired a recent Defense Science Board (DSB) study of
developmental test and evaluation, and during my opening
remarks, I'll summarize the key points from the study. I ask
that my written testimony, which addresses the major findings
and recommendations in more detail, be put into the record.
Chairman Levin. It will be.
Mr. Adolph. The task force was originally convened in 2007
to investigate the causal factors for the high percentage of
programs completing initial operational test and evaluation in
recent years, which have been evaluated as not operationally
effective and/or suitable.
The task force was asked to assess roles and
responsibilities for test and evaluation oversight in the
Office of the Secretary of Defense (OSD). We were also tasked
to recommend changes to facilitate the discovery of suitability
problems earlier, and thus, improve the likelihood of
operational suitability during initial operational test and
evaluation.
Very early in the study, it became obvious that the high
suitability failure rates were the result of systemic changes
that had been made to the acquisition process, and that changes
in test and evaluation alone could not remedy poor program
formulation and execution.
A number of major changes in the last 15 years have had a
significant impact on the acquisition process. First,
congressional direction from 1996 through 1999 reduced the
acquisition workforce, which, of course, includes developmental
test and evaluation. In many instances, services acquisition
organizations went well beyond the mandated cuts, some making
up to 60 percent reductions in organizations providing
acquisition support.
Concurrent with acquisition reform, the general practice of
reliability growth during development was deemphasized and, in
most cases, eliminated. This departure from the widely
recognized best practice may not have been a direct result of
acquisition reform, but may instead be related to the loss of
key personnel and experience, as well as shortsighted attempts
to save acquisition funds at the expense of increased
sustainment and life-cycle costs. Numerous studies have
conclusively demonstrated that investing in system reliability
during development will yield a substantial reduction in
support costs.
Our study reached the conclusion that the single most
important step necessary to correct high suitability failure
rates is to ensure that programs are formulated to execute a
viable systems engineering strategy, including a robust
reliability, availability, and maintainability program, as an
integral part of design and development.
Moving on to government test organizations, in the last 15
years, with some exceptions, there's been a significant
decrease in government involvement in test planning, conduct,
and execution. One of our task force members observed that, in
many instances, the government has gone from oversight to
insight to out-of-sight. Our task force recommends that
government test organizations reconstitute and retain a cadre
of experienced test and evaluation personnel to perform the
test oversight function.
Regarding OSD roles and responsibilities for test
oversight, the study team found that the developmental test
office, which had existed for decades, was disestablished in
the late 1990s. Currently there is no OSD organization with
comprehensive developmental test oversight, responsibility,
authority, or staff. We recommend that the office be
reestablished as a direct report to the Deputy Under Secretary
for Acquisition and Technology, as outlined in the proposed
legislation.
I'd like to make a few additional observations about the
systemic issues that have contributed to the current problems.
First, during a time of increased programmatic and
technical complexity, there has been a loss of a large number
of the most experienced management and technical personnel
without an adequate replacement pipeline. Solutions to
acquisition problems must begin with reconstituting a trained
and experienced government acquisition workforce, which
includes program managers, subject-matter experts, as well as
systems engineers, contracts personnel, testers, and
evaluators.
Second, more attention must be paid to technology
readiness, to include prototyping and testing crucial
technologies.
Finally, I believe that the major recommendations in the
recent study chaired by Dr. Kaminski on pre-Milestone A systems
engineer would, if implemented and combined with a revitalized
acquisition workforce, go a long way towards correcting many of
the current acquisition problems.
Thank you.
[The prepared statement of Mr. Adolph follows:]
Prepared Statement by Charles ``Pete'' Adolph
Mr. Chairman, members of the committee, I am Pete Adolph, the
chairman of a recent Defense Science Board (DSB) Task Force study of
Developmental Test and Evaluation (DT&E). I am pleased to present a
summary of the study results. The findings and recommendations I will
discuss represent a consensus of the Task Force members and do not
reflect an official position of the Department of Defense (DOD).
A DSB Task Force on DT&E was convened in the summer of 2007 to
investigate the causal factors for the high percentage of programs
entering Initial Operational Test and Evaluation (IOT&E) in recent
years which have been evaluated as both not operationally effective and
not operationally suitable. The following are the specific issues which
the Task Force was asked to assess:
Office of the Secretary of Defense (OSD) organization,
roles, and responsibilities for Test and Evaluation (T&E)
oversight. Compare organization, roles, and responsibilities in
both DT&E and OT&E. Recommend changes that may contribute to
improved DT&E oversight, and facilitate integrated T&E.
Changes required to establish statutory authority for
OSD DT&E oversight. Title 10, U.S.C. has an OT&E focus, and
does not address OSD authority in oversight of DT&E. Recommend
changes to title 10 or other U.S. statutes that may improve OSD
authority in DT&E oversight.
Many IOT&E failures have been due to lack of
operational suitability. Specific problems have been in the
materiel readiness sustainment areas of reliability,
maintainability, and availability. Recommend improvements in
DT&E process to discover suitability problems earlier, and thus
improve likelihood of operational suitability in IOT&E.
PROBLEM DEFINITION
In recent years, there has been a dramatic increase in the number
of systems not meeting suitability requirements during IOT&E.
Reliability, Availability and Maintainability (RAM) deficiencies
comprise the primary shortfall areas. DOD IOT&E results from 2001 to
2006 are summarized in Figures 1 through 3. These charts graphically
depict the high suitability failure rates during IOT&E resulting from
RAM deficiencies.
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Early in the DSB study, it became obvious that the high suitability
failure rates were the result of systemic changes that had been made to
the acquisition process; and that changes in DT&E could not remedy poor
program formulation and execution. Accordingly, the Task Force study
was expanded to address the broader programmatic issues, as well as the
issues previously identified.
A number of major changes in the last 15 years have had a
significant impact on the acquisition process. First, congressional
direction in National Defense Authorization Acts for Fiscal Year 1996,
1997, 1998, and 1999 reduced the acquisition workforce (which includes
DT&E). Several changes resulted from the implementation of Acquisition
Reform in the late 1990s. The use of commercial specifications and
standards was encouraged, unless there was justification for the use of
military specifications. Industry was encouraged to use commercial
practices. Numerous military specifications and standards were
eliminated in some Service acquisition organizations. The requirement
for a reliability growth program during development was also
deemphasized, and in most cases, eliminated. At the same time, systems
became more complex, and systems-of-systems integration became more
common. Finally, there was a loss of a large number of the most
experienced management and technical personnel in government and
industry without an adequate replacement pipeline because of the
personnel cuts. The loss of personnel was compounded in many cases by
the lack of up-to-date standards and handbooks, which had been allowed
to atrophy, or in some cases, eliminated. It should be noted that
Acquisition Reform included numerous beneficial initiatives. There have
been many programs involving application of poor judgment in the last
15 years that can be attributed to acquisition/test workforce
inexperience and funding reductions. It is probable that these problems
would have occurred independently of most Acquisition Reform
initiatives.
All Service acquisition and test organizations experienced
significant personnel cuts, the magnitude varying from organization to
organization. Over time, in-house DOD offices of subject matter experts
(who specialized in multiple areas, such as promoting the use of proven
reliability development methods) were drastically reduced, and in some
cases, disestablished. A summary of reductions in developmental test
personnel follows. The Army essentially eliminated their military
Developmental Testing (DT) component and declared the conduct of DT by
the government to be discretionary in each program. The Navy reduced
their DT workforce by 10 percent but no shift of ``hands-on''
government DT to industry DT occurred. The trend within the Air Force
gave DT conduct and control to the contractor. Air Force test personnel
have been reduced by approximately 15 percent and engineering personnel
supporting program offices have been reduced by as much as 60 percent
in some organizations. The reduction of Acquisition Program Office and
Test personnel in the Services occurred during a time when programs
have become increasingly complex (e.g., significant increases in
software lines of code, off-board sensor data integration, and systems
of systems testing).
PRINCIPAL FINDINGS AND RECOMMENDATIONS
Reliability, Availability, and Maintainability
As a result of industry recommendations in the early 1970s, the
Services began a concerted effort to implement reliability growth
testing as an integral part of the development process. This
implementation consisted of a reliability growth process wherein a
system is continually tested from the beginning of development,
reliability problems are uncovered, and corrective actions are taken as
soon as possible. The Services captured this practice in their
reliability regulations, and DOD issued a new military standard on
reliability, which included reliability growth and development testing
as a best practice task. The goal of this process from 1980 until the
mid-1990s was to achieve good reliability by focusing on reliability
fundamentals during design and manufacturing rather than merely setting
numerical requirements and testing for compliance towards the end of
development.
The general practice of reliability growth was discontinued in the
mid- to late 1990s. This discontinuance may not be a direct result of
Acquisition Reform, but may be related instead to the loss of key
personnel and experience, as well as short-sighted attempts to save
acquisition funds at the expense of increased life-cycle costs. With
the current DOD policy, most development contracts do not include a
robust reliability growth program. The lack of failure prevention
during design, and the resulting low initial Mean Time Between Failure
and low growth potential are the most significant reasons that systems
are failing to meet their operational reliability requirements.
According to Army studies, almost 90 percent of the sustainment
costs are directly correlated with the reliability of the system. Given
the amount of resources consumed during sustainment, investments in
reliability enhancements can provide a very large return on that
investment. A case study conducted by the Logistics Management
Institute, provided data that indicated an investment in total program
reliability would yield a substantial reduction in support costs.
Findings
Acquisition personnel reductions combined with
acquisition system changes in the last 15 years had a
detrimental impact on RAM practices
With some exceptions, the practice of
reliability growth methodologies was discontinued
during System Design and Development
Relevant military specifications, standards,
and other guidance were not used
Suitability criteria, including RAM, were de-
emphasized
Improved RAM decreases life-cycle costs and reduces
demand on the logistics system
The Deficiency Report can be a valuable tool for early
identification of RAM-related suitability problems, when used
in conjunction with an adequately resourced deficiency
correction system
Recommendations
The single most important step necessary to correct high
suitability failure rates is to ensure programs are formulated to
execute a viable systems engineering strategy from the beginning,
including a robust RAM program, as an integral part of design and
development. No amount of testing will compensate for deficiencies in
RAM program formulation. To this end, the following RAM-related actions
are required as a minimum:
Identify and define RAM requirements during the Joint
Capabilities Integration Development System, and incorporate
them in the Request for Proposal (RFP) as a mandatory
contractual requirement
During source selection, evaluate the bidders'
approaches to satisfying RAM requirements
Ensure flow-down of RAM requirements to subcontractors
Require development of leading indicators to ensure
RAM requirements are met
Make RAM, to include a robust reliability growth
program, a mandatory contractual requirement and document
progress as part of every major program review
Ensure that a credible reliability assessment is
conducted during the various stages of the technical review
process and that reliability criteria are achievable in an
operational environment
Strengthen program manager accountability for RAM-
related achievements
Develop a military standard for RAM development and
testing that can be readily referenced in future DOD contracts
Ensure a adequate cadre of experienced RAM personnel
are part of the Service acquisition and engineering office
staffs
Roles and Responsibilities of Government Test and Evaluation
Organizations
The role of the government in the DT process has evolved over the
past 50 years. With some exceptions, there has been a significant
decrease in government involvement in test planning, conduct and
execution, in the last 15 years.
The traditional role of the government during the DT planning phase
included the identification of the test resource requirements and
government test facilities, the development of the test strategy and
detailed T&E plans, as well as the actual conduct of T&E. When a
program moved from the planning phase to the test execution phase, the
government traditionally participated in test conduct and analysis;
performing an evaluation of the test results for the program office.
With some exceptions, this is no longer the case. Until recently, it
was recognized that there should be some level of government
involvement and oversight even when the contractor has the primary
responsibility regarding planning and execution of the DT program.
In addition to the reduction in the number of government
acquisition and test personnel, the experience level of both government
and industry personnel has steadily diminished in recent years. A
significant percentage of the workforce became eligible to retire since
2000, and due to prior downsizing, there has not been a steady pipeline
of younger technical personnel to replace them.
Findings
The changes in the last 15 years, when aggregated, have had a
significant negative impact on DOD's ability to successfully execute
increasingly complex acquisition programs. Major contributors include
massive workforce reductions in acquisition and test personnel, a lack
of up-to-date process guidance in some acquisition organizations,
acquisition process changes, as well as the high retirement rate of the
most experienced technical and managerial personnel in government and
industry without an adequate replacement pipeline.
Major personnel reductions have strained the pool of
experienced government test personnel
A significant amount of DT is currently performed
without a needed degree of government involvement or oversight
and in some cases, with limited government access to contractor
data
Recommendations
As a minimum, government test organizations should
develop and retain a cadre of experienced T&E personnel to
perform the following functions:
Participate in the translation of operational
requirements into contract specifications, and in the
source selection process, including RFP preparation
Participate in DT&E planning including Test
and Evaluation Master Plan (TEMP) preparation and
approval
Participate in technical review processes
Participate in test conduct, data analysis,
and evaluation and reporting; with emphasis on analysis
and reporting
Utilize red teams, where appropriate, to compensate
for shortages in skilled, experienced T&E domain and process
experts
Develop programs to attract and retain government
personnel in T&E career fields so that the government can
properly perform its role as a contract administrator and as a
``smart buyer''
Integrated Test and Evaluation
Integrated testing is not a new concept within the DOD, but its
importance in recent years has been highlighted, due in part to the
growth of asymmetric threats and the adoption of net-centric warfare. A
December 2007 OSD Test and Evaluation Policy Revisions memorandum
reinforces the need for integrated testing. Implementation of
integrated test concepts has been allowed to evolve on an ad-hoc basis.
The time has come to pursue more consistency in integrated test
planning and execution.
Collaboration between developmental and operational testers to
build a robust integrated test program will increase the amount of
operationally relevant data that can be used by both communities. DT
and Operational Test (OT) planning is separate and this inhibits
efforts by the Services to streamline test schedules, thereby
increasing the acquisition timeline and program test costs.
Additionally, there is a widely held assumption by many in the OT
community that only data from independent OT is acceptable for
operational evaluation purposes. While not all information from DT may
be useable by the Operational Test Agency to support IOT&E, a
significant amount of developmental test data can be used to partially
satisfy OT requirements. More importantly, an operational perspective
earlier in the developmental process has often proven to be a catalyst
to early identification and correction of problems.
DOD policy should mandate integrated test planning and execution on
all programs to the extent possible. To accomplish this, programs must
establish a team made up of all relevant organizations (including
contractors, developmental and OT&E communities) to create and manage
the approach to incorporate integrated testing into the T&E Strategy
and the TEMP.
Findings
Service acquisition programs are incorporating
integrated testing to a limited degree through varying
approaches
Additional emphasis on integrated testing will result
in greater T&E process efficiency and program cost reductions
Recommendations
Implement OSD and Service policy mandating integrated
DT&E/OT&E planning and execution throughout the program
Require sharing and access to all appropriate
system-level and selected component-level test and
model data by government DT and OT organizations, as
well as the prime contractor, where appropriate
Integrate test events, where practical, to
satisfy OT and DT requirements
Operational Test Readiness Review
Each Service has an Operational Test Readiness Review (OTRR)
process. Although it varies from Service to Service, the process
generally results in in-depth reviews of readiness to undergo an IOT&E
event.
Findings
A DOD Instruction requires that ``the Service
Acquisition Executive shall evaluate and determine materiel
system readiness for IOT&E''
Decision authority is frequently delegated to
the appropriate Program Executive Officer
Materiel developer is also required to furnish
DT&E report to the Under Secretary of Defense for
Acquisition, Technology and Logistics (USD[AT&L]) and
Director, OT&E
Shortcomings in system performance, suitability, and
RAM are usually identified during the OTRR
In most cases, the operational test readiness
certifying authority is well aware of the risk of not meeting
OT criteria when major shortcomings exist
Because of funding constraints, the low priority given
to sustainment, as well as the urgency in recent years to get
new capabilities to the Warfighter, major suitability
shortcomings have rarely delayed the commencement of dedicated
IOT&E
Recommendations
Conduct periodic operational assessments to evaluate
progress and the potential for achieving pre-determined
entrance criteria for operational test events
Conduct an independent Assessment of Operational Test
Readiness prior to the OTRR
Include a detailed RAM template in preparation for the
OTRR
Require the Command Acquisition Executive to submit a
report to OSD that provides the rationale for the readiness
decision
OSD Test and Evaluation Organization
The Task Force was asked to assess OSD roles and responsibilities
for T&E oversight. T&E has been a visible part of OSD since the early
1970s, reporting to the Research and Engineering command section when
it was in charge of acquisition oversight and subsequently to the Under
Secretary of Defense for Acquisition (now AT&L). The early T&E office
was responsible for all T&E, ranges, resources oversight, and policy.
In 1983, Congress established an independent Director, OT&E
organization, reporting directly to the Secretary of Defense (SECDEF),
responsible for OT&E policy, budget review, and assessments of
operational effectiveness and suitability. The Live Fire Test (LFT)
oversight function was created and added to the DT&E office
responsibilities in the mid-1980s. Later, the LFT oversight function
was moved to the DOT&E organization.
In 1999, the DT&E organization was disestablished. Many functions
were moved to DOT&E, including test ranges and resources, and joint T&E
oversight. Some of the remaining T&E personnel billets were eliminated
to comply with a congressionally mandated (AT&L) acquisition staff
reduction. The residual DT&E policy and oversight functions were
separated and moved lower in the AT&L organization.
A 2000 DSB Task Force Study on Test and Evaluation Capabilities
recommended that DOD create a T&E resource enterprise within the office
of the DOT&E to provide more centralized management of T&E facilities.
This recommendation ultimately led to removing the test ranges and
resources oversight from DOT&E, abandoning the notion of centralized
management, and the establishment of the Test Resource Management
Center (TRMC) in AT&L (as directed by the National Defense
Authorization Act for Fiscal Year 2003).
Findings
Current policy as of December 2007 mandates that developmental and
operational test activities be integrated and seamless throughout the
system life cycle. There must be enough experts in OSD with the ability
to understand and articulate lessons learned in early testing and the
ability to execute the new T&E policy. That policy is to ``take into
account all available and relevant data and information from
contractors and government sources'' in order to ``maximize the
efficiency of the T&E process and effectively integrate developmental
and operational T&E.''
Currently there is not an OSD organization with
comprehensive DT oversight responsibility, authority or staff
to coordinate with the operational test office
The historic DT organization has been broken
up and residual DT functions were moved lower in
organization in 1999, and lower yet in 2002
Programmatic DT oversight is limited by staff
size and often performed by generalists vice T&E
experts
Recruitment of senior field test personnel is
hampered by DT's organizational status
Existing residual organizations are fragmented
and lack clout to provide DT guidance
System performance information and DT lessons
learned across DOD has been lost
DT is not viewed as a key element in AT&L
system acquisition oversight
Documentation of DT results by OSD is minimal
Access to models, data, and analysis results is
restricted by current practice in acquisition contracting, and
the lack of expertise in the DT organization
TRMC has minimal input to program-specific questions
or interaction with oversight organizations on specific
programs
Organizational separation is an impediment
Recommendations
Implementation of integrated and seamless DT and OT
will require, at a minimum, greater coordination and
cooperation between all testing organizations
Consolidate DT-related functions in AT&L to help
reestablish a focused, integrated, and robust organization
Program oversight and policy, and Foreign
Comparative Test (FCT)
Have Director, DT&E directly report to Deputy
Under Secretary of Defense, Acquisition and Technology
(DUSD[A&T])
Restore TEMP approval authority to Director,
DT&E
Integrate TRMC activities early into DT program
planning
Make TRMC responsible for reviewing the
resources portion of the TEMP
If such an organization is established and proves
itself effective, consider as part of a future consolidation
moving LFT back to its original DT location (this would require
congressional action and DOT&E concurrence)
Most of the organizational changes recommended above are within the
purview of AT&L. The LFT change requires the concurrence of DOT&E and a
legislative change to Title 10 because of the change in reporting
official. All the other recommendations can be implemented within
current DOD authority.
Chairman Levin. Thank you so much.
I think we'll try a 7-minute round. I'm not sure there will
be time for a second round; I think we have a vote scheduled
around 11:30 a.m., if I'm not mistaken.
Let me start with this question. Mr. Sullivan, you
commented on the reform bill, the Weapon Systems Acquisition
Reform Act of 2009, which I've introduced with Senator McCain.
Each of you, in a way, has commented on it. Let me start with
you, Dr. Gansler. Would you comment on the bill, any parts you
like, any parts you don't like, any additions you might be able
to recommend at this time?
Dr. Gansler. Overall, Mr. Chairman, I thought it was in the
right direction and important. I think that we have to
recognize that simply writing a memo, passing a law, doesn't
change the system. Each of the areas that you highlighted, I
think are in the right direction. What we have to be careful of
is not going too far in one direction. For example, in the
conflict of interest, clearly we have to avoid conflict of
interest, but we don't have to go so far that we have only
people who have no experience in those positions. That's the
danger of going too far, in terms of the legislation of it.
In each of the areas, I think that there's some clarity
that could be added, but, in general, I think you've gone in
the right direction. For example, you emphasize systems
engineering. I think you need to define that as including costs
so that there's no ambiguity as to whether that's a pure
engineering problem or a cost issue, from a design perspective.
I think the importance of test and evaluation that you
highlight is clearly something that's very important. It could
be emphasized without really changing the title of the office.
But, nonetheless, when I was Under Secretary, I felt it was a
very critical piece. But, it's important, when you talk about
test and evaluation, that it be viewed as a part of a
development process. We learn from that testing. We do it
early, as Mr. Adolph indicated, but we also don't view it as a
pass-fail final exam because we're doing spiral development,
and we're continuing to learn from the testing as it goes
along, and some people have tended to think of the test process
as a final exam.
In the area of independent cost analysis, it is absolutely
essential. On the other hand, we have the Cost Analysis
Improvement Group (CAIG) office, which I used all the time, and
I felt it was critically important. The problem is, people
don't want to use their numbers. If they say it's realistically
going to cost more, and government and industry want to put in
the low bid in order to get a program into the overall budget,
that's a management question; it's not a matter of what
organization you set up. But, it's a very important function,
as you highlighted.
In terms of getting the combatant commanders involved in
the requirements process, that was exactly what we intended
with the Packard Commission and Goldwater-Nichols intended with
the establishment of the Vice Chairman of the Joint Chiefs.
That was the purpose of that statement. Somehow that hasn't
gotten the intended strength, it has been much more the
suppliers than the users, if you will, the warfighters. I think
it is important to get the combatant commanders much more into
the loop on the requirements process, and I think you've gotten
that properly emphasized.
I think, in the acquisition area I would emphasize that
projected unit costs are a military requirement because that's
one of the things we've lost in some of the programs. Global
Hawk started out that way, got off track. You mentioned the
JSF. That started off as a unit cost being one of the
requirements for globalization of that program. It got out off
track because it lost sight of that unit cost as being one of
its principal considerations in design.
As I mentioned, in terms of competition, I think it's
really important that we view competition as an option
throughout the programs, at the prime and at the lower tiers,
but not as a law. You don't compete for its own sake; you
compete when the current contractors, prime or sub, aren't
getting better performance at lower cost. That's their
incentive for doing it. If you tell them we're going to compete
it anyhow, they have no incentive. If you tell them that if
they can get higher performance and lower cost, then, in fact,
they will try to achieve that so that they don't have to
compete in the next round, and it's doing exactly what the
government wants them to do.
In the same way, if they achieve the objectives, we ought
to use, in effect, the same way that the commercial world does:
through price elasticity. If you get a lower cost, we'll buy
more of them. We don't take the money away and put it in the
general treasury, so there will be a need to create incentives
for industry, as well as government, for doing a better job of
achieving higher performance at lower cost, continuously.
In terms of conflict of interest, I think we need to focus
on some structural ways to address conflict of interest. We do
this, in terms of foreign ownership, through limited liability
corporations, in effect, the special boards are set up. Maybe
there's some ways we can do that in order to address conflict
of interest without the sort of blanket requirement that
someone in an engineering job shouldn't know anything about
that job. That's wrong.
Those are the suggestions that I would have. But, overall,
I think you are definitely going in the right direction with
the bill.
Chairman Levin. Thank you. We would welcome any specific
language changes you would recommend.
Now, Mr. Sullivan, you already commented. Do you have any
additional comments?
Mr. Sullivan. Yes, just briefly. I would say that, given
the package that we've looked at, we support everything in it.
We thought it was very well-targeted to the key problems on
acquisition programs, and that the idea to give more authority
to the combatant commanders, in terms of getting urgent needs
met, was a good provision.
I think the most important thing on an acquisition program
is, at the outset--we've all discussed it here--to have more
knowledge about the requirements that you're going to build to,
before you begin, than they have now.
So, the two provisions that we think are most important--
first is the cost-estimating provision. We wrote that we don't
see any reason why the CAIG actually couldn't fulfill that
position. The important thing there is to probably provide the
CAIG with more resources so they can do their jobs on a more
regular basis and rather than just periodically, at every
milestone. The other critical thing there is that if the
director was moved up, out of the bureaucracy a little bit
more, the estimates that the CAIG make might be looked at as
less personality-driven, if you will, depending upon who's in
office.
As Under Secretary of Defense, Dr. Gansler used the CAIG,
others may not use it as much. If they're reporting to a higher
level, and they owe Congress a report, I think that'll really
improve their authority and their visibility and their
independence. But, the key is always going to go back to the
requirement-setting process, and jelling that with cost
estimates. If you begin with not enough information about what
it is you're going to, what you want to build, you are not
going to get a good cost estimate. So, it really has to be
based on knowledge.
So, the cost-estimating provision, I think, is very good.
The systems-engineering provisions that you have in there go a
long way to providing the knowledge that the cost estimators
would need up front.
Chairman Levin. Thank you.
Dr. Kaminski, do you have any comment?
Dr. Kaminski. Yes, sir, I do. I made some specific comments
in my statement, but let me make a couple of big-picture
comments, if I may.
If there's a direction in which I would try to move
language in the proposed legislation, it would be to focus, not
only on process, on oversight mechanisms, but a focus on
people; people that make this system work. I don't care how
good a process you put in place, if you don't have people who
are experienced and know what they're doing, I think you're
going to end up with problems.
When I say ``people,'' I am referring very broadly to our
acquisition workforce. Think about this for a minute. These
people and requirements development skills are also key for
those who are doing development of requirements. They have to
participate in this tradeoff process to consider the cost of
what they're asking for and how those tradeoff with the
performance capabilities that are desired.
Testing is a critical piece of this process. If you think
about it--I know my experience is, there isn't any program I've
ever worked on in which I didn't know a heck of a lot more
about the program 6 months or a year into it than I did when I
started. So this is a continuing learning process as we find
out things from testing--what's hard to do, what's easier to
do. We need to have a continuing dialogue in this requirements
tradeoff loop.
So, requirements developers and acquisition personnel have
to have training and experience in these systems engineering
tools and techniques. We would never let a fighter pilot get
into an aircraft without a very extensive training program to
prepare him or her for that operation. It's one of the reasons
we do so well with our forces. When I compare our requirements
for operational requirements with the training and education
requirements of people going into acquisition requirements
generation, they pale in comparison. We have to be able to
develop the training, education, and the domain experience that
go with this to make it work.
I also believe it is worth it for us to look very hard at:
How can we systematically reduce this long development time?
That's just killing us. There are programs in which we can do
time-certain acquisition. Time is money, here. By doing good
development planning to be able to reduce the risk, following
the example of Apollo, following the example of our ICBM
programs, I believe we can compress that time. But, we need
some targets, and some incentives to do that.
We also need to make better and more extensive use of
prototypes, in a sensible way. Competitive prototypes for some
programs, perhaps single-thread programs for others. We also
need to realize that one size does not fit all, here. For
example, in dealing with IEDs, we find that we need to have an
acquisition system whose cycle time is measured in weeks.
That's a different acquisition system than you need for a
strategic bomber. The acquisition system has to fit the cycle
of the threat that we're dealing with, so we have to tailor it
in that way.
The last comment I would make is that, with respect to
implementation, what better place could we find to start to
implement some of these processes, procedures, and people
development than in these urgent programs that are doing rapid
acquisition. Why start there? One, it's urgent. Two, we'll be
able to see the impact of changes that we make more quickly in
programs that are operating in cycle times of weeks or months,
and see what's benefiting us and what is not. I'd like to see
that commitment to implementation because it'll happen much
more quickly than if we simply write new processes or new 5000
series in the DOD.
Chairman Levin. Thank you.
My time's way up, but, Mr. Adolph, do you have a brief
comment on the bill? Any changes?
Mr. Adolph. Very brief comment. In general, I believe that
the proposed legislation will go a long way toward alleviating
the major acquisition problems which have occurred in recent
years.
I also have a quick comment regarding the legislation
pertaining to the director of the developmental test function.
I would add one responsibility, that the director participate
in the acquisition program reviews conducted by the Under
Secretary for Acquisition and submit a status of the
developmental testing for the programs under review. That was
the norm in the past, for many years, and I think that's
important.
Chairman Levin. Thank you so much.
Senator McCain.
Senator McCain. Thank you, Mr. Chairman. Again, thanks, to
the witnesses.
I think there's general agreement on the part of the
witnesses that there's been a dramatic erosion in acquisition
workforce and test and evaluation personnel. Maybe I could ask
you, Dr. Kaminski, what happened?
Dr. Kaminski. I think a number of things happened. Just as
there's no one silver bullet, there's no single entity or
action to blame. But, let me list some of the things that
happened. I think, as we looked at major programs of
acquisition reform, we tried to do more with less. We also had
some pretty strong direction from some portions of Congress on
this. I can recall the chairman of a key committee who publicly
made statements that said we had too many ``shoppers'' in the
Department, and we really needed to reduce this acquisition
workforce. So, over a period of 3 or 4 years, nearly 50 percent
of the acquisition workforce was taken out. Now, the Department
was a partner in that, agreeing to those reductions.
Senator McCain. We removed incentives for people to remain
in the acquisition workforce, in the form of lack of promotion
or career enhancement?
Dr. Kaminski. Exactly. If we are able to attract and train
the right system engineers--if that system engineer doesn't see
some path for advancement in the DOD, he or she is going to go
find a place where they can make a difference and have an
opportunity for advancement.
Senator McCain. Which means a revolving door evolves?
Dr. Kaminski. Yes, sir. But, that revolving door actually
may be the commercial industry. Again, I return to the comment
I was making about the importance of cycle time in the
acquisition system. If you have someone who's really worth
their salt and able to make contributions, and they get into a
DOD acquisition system which is going to produce something in
15 or 20 years, it won't take them long to realize that their
knowledge base is going to erode so that they will no longer be
valuable to commercial industry, which is producing things in
2- or 3-year timeframes.
So, to be able to have some revolving door from commercial
industry back to DOD will benefit from getting these cycle
times down.
Senator McCain. Dr. Gansler?
Dr. Gansler. Yes, Senator McCain. In my prepared testimony,
I actually showed a graph that came out of the commission that
I ran. I was shocked at how much the acquisition workforce has
been allowed to deteriorate. In fact, as we came out of the
Cold War, the procurement budgets dropped, the number of
acquisition people came out with it. Then, as Dr. Kaminski
said, there was a legislative mandate in 1996 to take another
25 percent out. So we've ended up, now, dramatically, where we
had about 500,000 people in 1990, we now have about 200,000
people. But, the acquisition dollars have gone up dramatically,
so you have this huge gap between the dollars and the people.
But, much more important is the point that you just made
about the officers and the senior people. In 1990, the Army had
five general officers; in 2007, when we did the study, there
were zero general officers with a contracting background. In
the contract management organization, the Defense Contract
Management Agency (DCMA), they had 25,000 people in 1990, they
have 10,000 today. There's basically an undervaluing. They used
to have four general officers; they have zero. So, as you
suggest, if you're a young major, you're not getting into that
career field, and, as a result, it's been just totally
undervalued. Without those experienced senior people, both
civilian and military, they don't know what questions to ask,
and they are not going to be able to make the right judgments.
Senator McCain. Obviously, we need to have some personnel
policy changes, as well. Mr. Sullivan, the issue of Nunn-
McCurdy--when it was first passed, we thought it was really
important and effective. For a while, it was. I think a breach
of Nunn-McCurdy was a big deal. Now it seems to be a routine
kind of event that the notification comes over, we see it, and
ho-hum. Are we in danger of experiencing the same thing with
this measure?
Mr. Sullivan. We have noticed that the Nunn-McCurdy
breaches we see take a lot longer to resolve and come out with
a new program, and the funding is continuing on that program as
they do that.
As I read the proposal that the committee has now, I think
it's really good to have a termination criteria like that. I
think GAO thinks that's a good thing. In other words, a program
termination would not just have someone do a review, but also
probably look at what happens, what the triggers are for that.
I think, right now, it states that the program cannot change
the scope of work, cannot start new contracts, and there might
be one other thing that the program can do. I don't know if
legislation can basically say that that program can no longer
obligate money, but that would, to me, be a much more direct
way to get the point across. If a program is automatically
terminated when it passes a threshold, and cannot obligate
money, that might get people's attention.
Senator McCain. I can imagine the blowback when some vital
program is shut down because of our failure to act, but I think
we're in agreement that there has to be a more robust oversight
and ability to exercise that oversight as we see these costs
spiral completely out of control.
Mr. Sullivan. Yes, sir.
Senator McCain. Mr. Adolph, on the issue of Nunn-McCurdy,
do you think it's sufficient measure to impose more discipline
on the cost overruns?
Mr. Adolph. I believe it's sufficient. But, the issue that
concerns me, and the remedy has already been discussed, is to
get the technology readiness right, at the outset; and second,
to get a realistic cost estimate. There's too much concurrency,
there are actions that are taken in the program to kick the can
down the road on Nunn-McCurdy, which, in the long run, in my
opinion, adversely impact the program. The example from the F-
35 is getting rid of two of the test articles.
Again, I believe Nunn-McCurdy is fundamentally sound. The
problems in recent years are a combination of issues which I
believe the proposed legislation, if it's really implemented in
the Services--and, again, back to Dr. Kaminski and Dr.
Gansler's point--in order to implement it, we have to
reconstitute the acquisition workforce. That's the first step.
Senator McCain. Also I think it would be important for us
to have the combatant commanders more involved in the
requirements process, as well. I think sometimes we have
neglected that aspect of the equation.
I thank you, Mr. Chairman. I thank the witnesses.
Chairman Levin. Thank you, Senator McCain.
Senator Akaka.
Senator Akaka. Thank you very much, Mr. Chairman.
I want to add my welcome to the witnesses before us today,
and to point out that I'm glad that, Mr. Chairman, you are
holding this hearing on acquisition. For me over the years, I'm
beginning to feel that we need to change, or try to change, the
heart and soul of DOD, and, really, the culture of DOD, and to
get to acquisition.
Gentlemen, as you well know, the reform of DOD acquisition
process is an extraordinarily complex undertaking involving
many, many moving parts. However, I believe the first step to
tackling any problem is to prioritize. So, let me ask this
question to each of you. In order to most effectively reform
the DOD acquisition process, what do we need to focus on first?
First, Dr. Gansler.
Dr. Gansler. I believe it's the people. I think if we've
undervalued the importance of this area, in terms of promotion,
in terms of experience, in terms of numbers, all across the
board, both civilians and military, that we're not going to get
there, even if we pass all the laws in the world. We need the
people who are going to be driving this process. That is my
number-one priority. We have neglected it and, of course, in
the last 8 years we've been living in a rich man's world, so
money doesn't matter, and if people overrun or they don't
perform, ``Let's spend more money.'' Now, that's not going to
be the case, and we need people who are smart, experienced, and
competent to run their programs--with flexibility, though. They
have to make tradeoffs of cost and performance, systems
engineering kinds of work, test and evaluation, so forth. That
requires management judgment, and you can't just legislate
that, and therefore, you need people with experience to be able
to make those management judgments. That's my number one
priority.
Senator Akaka. Dr. Gansler, would you also say that another
part of that would be inadequate staffing?
Dr. Gansler. Yes.
Senator Akaka. From what I gathered, there were positions
that were not filled.
Dr. Gansler. Absolutely. I was shocked to find that, in
Iraq and Afghanistan, only 35 percent of the people that were
in their jobs were qualified for those jobs, even with the
minimal qualifications that Dr. Kaminski mentioned. Besides
that, most of the positions weren't even filled, and they were
almost all volunteer civilians in the warzone. We need to be
able to get some senior military there, as well. So, there's a
great lack of people, numbers, but you don't want just numbers,
you also want qualified people. Numbers won't do it. It has to
be qualified, experienced people. Some of those by the way, can
come from industry. You can rotate people from industry,
without conflict of interest, very easily. That's what we've
had with many of the people with past experience. We do that in
DARPA, we bring them in and out, and we've done it in other
parts of the government. I think that people are out there with
experience; we just have to make sure that we make it
attractive to get them in these government acquisition
positions.
Senator Akaka. Mr. Sullivan.
Mr. Sullivan. I think, at a strategic level, one of the
biggest problems the DOD faces is that it is unable to
prioritize what weapon system capabilities should be put into
programs. They have 95 weapon system acquisition programs,
major acquisitions, that are underway right now. That's up from
about 75 or so in 2000.
There is a tendency to have too many programs vying for the
acquisition dollars that are available to DOD on an annual
basis. When that happens, you get a very unhealthy competition,
where many programs and few dollars drive the requirements-
setting process, which is stovepiped in many ways by the
Services.
So, the Services are all vying for a solution, they want
requirements that are very, very tough to make, so that their
weapon system can do the most. They, as a result of that, tend
to put in optimistic cost estimates. The funders, of course,
are looking at those very optimistic cost estimates that are
very heavy on assumption and very light on facts, data, and
actual costs. The acquisition process begins with the lack of
the systems engineering that we've talked about here today.
So, you have too many programs chasing too few dollars,
with business cases that are unexecutable. The whole system is
segregated in such a way with process owners and stakeholders
that, in a way, it works for everyone. That's the culture. It's
a performance-driven culture, and we all understand that and
accept that, but there's also a lot of players involved in the
culture that create this kind of unhealthy competition at the
outset. I think that's the culture change that has to take
place in this DOD, and that is almost intractable, when you
think about it: the difficulty of changing that. I think
legislation can go a long way to requiring people to do certain
things, but, in the end, I agree with Dr. Gansler, it falls on
the people to have the right principles, if you will, to change
what this system is really supposed to create.
Senator Akaka. Dr. Kaminski, with your research and study
background, do you care to comment?
Dr. Kaminski. Yes. Let me not repeat because I agree with
everything that's been said. Let me put another dimension on
the people. I, too, would answer: most important is people.
But, what you have to do is recognize the dynamics that involve
people. If we're going to attract our very best and brightest
people to an activity, I've found that the principal incentive
usually isn't money. With the salaries we pay military officers
and civilians in DOD, we are still able to attract very capable
people to key jobs.
What attracts them? What usually attracts them is the
ability to make a difference, to see that they can have a
personal impact on a major program, on the security of this
Nation.
So, I want to come back again to my issue about time. When
programs are taking 15 or 20 years, many of those best and
brightest people say, ``What's the difference? If I'm not going
to see something happen for 15 or 20 years, why don't I go
someplace where I can make something happen sooner?'' If they
don't have any freedom to make decisions and influence things
because of excessive oversight processes and complexity of the
process, they'll go find another place to work.
I want to share with you a perspective that I got from a
different position. Rather than a perspective of a previous
Under Secretary of Defense, I want to share the perspective
that I gained when I served on Active Duty in the Air Force.
I'd say I spent two-thirds of my career there working on
special access programs, part of that career, in the early days
of the National Reconnaissance Office, where I was a program
manager for one of our National Reconnaissance spacecraft
that's up and flying today, and for several years in the
stealth program. Let me just pick one example there,
the F-117 program. I was heavily involved in that program.
When we initiated that program, our plan was, from the
beginning of full-scale development to field the aircraft in 3
years. We missed that, sir. We missed it by a year. We fielded
the aircraft in 4 years. Everybody who worked on that program
could see that they were making a difference. They could see
that capability coming along to be fielded. There was excellent
interation and tradeoffs in that process, between the testers
and the program managers and the users who were going to use
that aircraft. We made continual adjustments. So, that
motivation of people and reduced time go together. To attract
the right kind of people, we have to work on this time issue.
Senator Akaka. Mr. Adolph?
Mr. Adolph. As I mentioned in my earlier remarks, people
are certainly the highest priority, and that's been reiterated
here by the previous comments.
Another area for improvement is certainly overly ambitious
requirements. We need to continue to push technology, without a
doubt. But, at the same time, we need to ensure that the
technology is sufficiently matured to incorporate in a weapon
system. That means prototyping and testing, and testing the
prototype item in an environment in which it's going to be
placed in a combat environment. So, again, first, people.
Second, getting the requirements right and making sure that
they're not overly ambitious. Again, Dr. Kaminski's study
really addresses the latter issue quite well.
Senator Akaka. Thank you very much, Mr. Chairman.
Chairman Levin. Thank you, Senator Akaka.
Senator Collins.
Senator Collins. Thank you. Thank you, Mr. Chairman.
Mr. Chairman, I first want to salute you and Senator McCain
for your initiative. I think you've brought forth a bill that
will really make a difference. I also want to point out that
I'm very pleased to hear the witnesses today all stress the
importance of the acquisition workforce. This is an area that
Senator Lieberman, Senator Akaka, and many of us, have said is
the number-one problem, over and over again. When I brought
this up at the White House Conference last week, however, some
of my colleagues felt that it was a lesser problem. So, I was
very pleased to see the panel of experts before us list this as
perhaps the greatest problem that we're facing.
Dr. Gansler, you and I also worked together, many, many
years ago, on how to increase competition in Federal
contracting, and it's very good to see you here again, as well.
I want to ask the panel about some of the specific
provisions in the Levin-McCain bill. In particular, this bill
would require that costs be considered right up front, when the
requirements are set. That is a pretty dramatic change from how
military requirements are set now, when they're done in an
environment that does not consider costs, but, rather, an
idealized world, where costs would not be a factor.
Dr. Gansler, you endorsed including cost as a design
military requirement right up front, so I'm going to skip over
you for this question and go to the rest of the panel and ask
all of you: Should costs be considered up front when military
requirements are first established?
Mr. Sullivan?
Mr. Sullivan. I think that is a good idea. Dr. Gansler said
that should be one of the key performance parameters on any
major weapon systems, in their business case, and I think we
would agree with that.
When you're trying to set requirements, and you're doing
the requirements analysis that's needed, I think you begin, at
first, in an unconstrained manner, and try to get from the user
what the user would like, in an unconstrained environment. But
then it's critical, at some point, to start bonding that with
the realities of the time it's going to take to get that to the
user, the amount of money, and the technologies you have
available to do it. So, I don't think you could do that without
precluding the ability to think, unconstrained. But the exit
criteria would be something that is constrained, at least in a
cost range. Then once it exits the requirements process, there
is a stage before it would become a set business case and begin
as an acquisition program, where that cost range could be
further reduced to more of a point estimate by continuing to
make trades.
Senator Collins. Dr. Kaminski?
Dr. Kaminski. Senator Collins, I also agree that cost
should be an upfront factor. I'd add another factor to go with
it, though.
Senator Collins. Yes?
Dr. Kaminski. Back to my comment about time. Time and money
go together. When we're doing requirements tradeoffs, if we're
going to have an acquisition cycle time that operates within
the threat cycle time, those developing requirements have to
look at the time they want something fielded, as well. So, that
needs to be an important consideration in this process, and we
need to manage the time.
The one other comment I wanted to make with respect to cost
estimates: we talked earlier about breaches in Nunn-McCurdy.
The surveillance system that finds the breaches, I think, is
fine. But, one of the things we want to do is look at root
causes. What's causing us to get into Nunn-McCurdy? One of the
constructive uses for the independent cost estimate that was in
the bill might be to add another consideration. We've talked
about the importance of the upfront work in systems engineering
and development planning. One of the things that would be
useful for us all to ensure is that there is adequate funding
at the beginning of a program, between Milestone A and B, for
us to apply our systems engineering and development planning
capabilities to get a good handle on what those cost estimates
are, and do a thorough job involving stakeholders, the
requirements part of the equation, and the program manager in
that process, along with the CAIG.
I, like Dr. Gansler, used the CAIG very heavily. In fact, I
advocated that we fund programs to the CAIG estimate, but we
hold the program manager and the contractor to the estimates
they developed, so we had some finance reserve between the two
estimates.
Senator Collins. Thank you.
Mr. Adolph?
Mr. Adolph. Senator Collins, certainly we need to consider
the cost issue upfront. Another very important driver early on
is technology readiness. In program after program after
program, we're into full-scale development and discover that
some of the critical technologies simply aren't mature enough.
So the program is delayed, and that drives the schedule, and
these slippages occur.
A key to getting the costs right is to ensure that the
technology is really sufficiently mature.
Senator Collins. Thank you.
Dr. Gansler, an issue that hasn't been discussed today is
the impact of a lack of stability and predictability on a
program's cost growth. That obviously can drive up the unit
cost. When the military starts out with one plan for acquiring
a weapon system, and then switches direction or reduces the
number of units, doesn't that also drive up the cost?
The reason I bring this up is, we talk a lot about the
errors made by contractors, we talk about weapon systems that
get gold-plated because additional requirements are added, but
there's also an important issue, as far as the stability of
funding and the lack of predictability driving up the unit
cost. Could you address that issue?
Dr. Gansler. Yes. But, let me just briefly comment on your
first question, though, because frequently the military doesn't
think that cost is a military requirement. What they neglect is
the fact that numbers are a military requirement. If you're
resource-constrained, the total dollars that you have divided
by the unit cost gives you the numbers. Numbers really matter
in military operations, whether it's by Lancaster's law of N-
squared or by numbers. Either way, numbers really matter, and
therefore, cost really matters. That's why it's so important to
have the unit cost as part of the requirements.
Now you get to your changes, and unless we estimate the
cost of those changes and their impact on the ultimate cost of
the equipment, we let things get out of hand. So, if cost is a
requirement, then every time a change comes in, and as Senator
McCain said earlier, it was 75 a week on the LCS, you have to
price those each out to make sure it is not having a big cost
impact on the program.
Then, when programs, in general, become relatively stable,
you don't get this ripple effect through the budget, which is
the point that you're really making, Senator. If I want to pay
for program A, I take it out of program B, not recognizing that
program B now is in really bad shape because they don't have
the stability of the funding. That stability of funding is a
critical issue.
Senator Collins. Thank you.
Chairman Levin. Thank you, Senator Collins.
Senator Burris.
Senator Burris. Thank you, Mr. Chairman.
This is an interesting experience for me, to hear the
distinguished testimony, Mr. Chairman. I would rather listen
than to talk. I yield my time.
Chairman Levin. Thank you.
Senator Burr is next on this side; he's not there.
Senator Chambliss.
Senator Chambliss. Thank you, Mr. Chairman.
Gentlemen, it's good to see all of you, and thanks for your
service over the years, and thanks for your being here on what
I think is, if not the most important issue at the Pentagon
today, certainly it ranks up pretty high. With these budget
times that we're in, trying to figure out a way to buy the
weapon systems that we need within the timeframe we need is
virtually impossible. That's why your testimony is so critical.
I want to go back to what Senator Collins was talking about
there, and what all of you have alluded to in some part, and
that's this issue of instability, whether it's requirements,
whether it's personnel, whether it's funding, or whether, as
you say, Mr. Adolph, it's the technology aspect of it, where
we, too many times, tend to come up with a great idea, and by
the time we get into the production phase, we've wasted, not
only time, we've wasted money, but oftentimes, there's a new
idea that has been developed in the meantime.
I'm amazed, Dr. Kaminski, that you say, on the F-117, that
we were in production in 4 years. Gosh, if we had done that
with the F-22, we wouldn't be having the arguments we're having
today, and we'd have a great airplane, and we'd be worrying
about other issues.
But, it doesn't make any difference whether you're talking
about an aircraft carrier, whether you're talking about FCS or
a tactical fighter, we come up with this idea, and we get into
the R&D phase, and there is, all of a sudden, a great idea, but
instability in all four of those areas runs that cost up
tremendously. Then, you throw in what Senator Collins alluded
to, about the number of these units that we're going to buy,
and all of a sudden it explodes again and it becomes such a
negative at the Pentagon, rather than the positive that it
started out to be.
My question for a comment from each of you is: How do we
get back to this? How do we get back to the point to where we
come up with this idea? If it's a tactical fighter, it's
supposed to be air-to-air, or supposed to be air-to-ground,
whatever it may be, how do we develop that and get it into
production right away, without technology intervening and all
of a sudden having to add this and add that? How do we get our
arms around that issue?
Jack, let's start with you.
Dr. Gansler. I think one of the main opportunities we have
is to accept the concept of spiral development, that for the
block 1 system, we have a fixed set of requirements, we have a
fixed price that we're trying to get a fixed schedule, as Dr.
Kaminski said, and we go ahead with block 1 under the
assumption that if we can then demonstrate new technology, if
we find that the user needs something different, if we find
that even the logistician has a problem with maintenance of
that equipment or the reliability as we deploy it, that becomes
block 2, block 3, block 4. But, block 1 has to use proven
technology and get out there quickly, all with a set of
constraints. So, it's a stable program, as you point out.
The most successful acquisition that, in fact, Congressman
Aspin used to always highlight, was the Navy's Polaris,
Poseidon, Trident. When I was in industry, I always knew how
much money I was going to get next year for that program. I
could hire, I could plan my workforce, and so forth. That
stability is very important for efficiency. I think, if we go
to a spiral development model, whereby block 1 is stable, and
block 2 is being developed while block 1 is being deployed, you
have the concept of stability built in, and evolutionary
systems are still stable. That's the way the real world, the
commercial world, works. You constantly are upgrading the
software, the hardware of computers, but you're constantly
getting higher performance at lower cost. That has to the be
objective of each of the blocks as we're going along in spiral
development.
Senator Chambliss. Mike?
Mr. Sullivan. Goldwater-Nichols legislation, from years
ago, tried to bring jointness into the DOD. I think it
succeeded on the operations side. We now have the combatant
command's matrix, the military forces that fight wars jointly
very well. The same thing did not occur on the acquisition
side. I think if you look at what's going on now, there's a
kind of a stovepipe system for how you get programs started,
and that creates this kind of competition for big requirements
and cost estimates that are heavy on assumption.
I would agree with everything that Dr. Gansler just said.
If you can work on that and get a more joint requirement
setting and funding system, and try to get the proper balance
of weapon systems started, try to get rid of the stovepipes,
you'd have an environment that could do what Dr. Gansler, I
think, is describing, a little more easily.
Another program I would throw out is the F-16 program. Back
in the 1970s, it was a block program. It was the capabilities
that the Air Force wanted for the F-16. They knew that they
needed an aircraft faster than they could develop the
technologies to get those capabilities, so they had blocks. If
you look at the F-16s, and, for that matter, the F-15s,
performance over the last 30 years, it's pretty impressive.
They basically upgraded those aircraft pretty efficiently as
they went because they started without that big revolutionary
leap, that one-step, big-bang kind of a thing.
So, we've done this before, and I think it's possible to
get back to it, but this is where the culture comes in. I think
there's some culture change that needs to take place.
Dr. Gansler has other things that I read in his report that
would help this significantly. Open systems, for example, on
these weapon systems, when, if you can make interfaces on the
weapon systems uniform, you can keep proprietary data that
subcontractors have that supply subsystems to them, and all
they have to do is have the proper interface. Then you can open
up competition.
One more point I would make is the difference between
technology development and product development probably needs
to be better understood. Technology is the kind of thing you
should think of when you think of scientists and lab coats and
trial and error, and it's done in a smaller-dollar environment,
where you can test, use trial and error, and make mistakes. You
have to keep that off of these acquisition programs. I think
someone up here, I don't know if it was Dr. Kaminski or Mr.
Adolph, said that when you have a technology that's not mature,
and it's on an acquisition program that's driving towards
production, you have an entire workforce--an entire supply
chain, for that matter--that's waiting for that technology to
mature. The burn rate is pretty big on that workforce that you
have.
Dr. Kaminski. I'd emphasize that point. While everybody's
waiting, we're paying, and so what you want to do is decouple
those two.
In terms of approaches to do this, I agree completely with
the spiral development approach. One problem I see is in the
application of spiral. In the few spirals I've seen that we
actually implemented, we had everything but the kitchen sink in
the first spiral, rather than stretching this out over a period
of time, like we did in Apollo. So, we need development
planning. You have to have a plan with stability to do this.
The program manager has to have the discipline and the
experience to reject things that aren't in the plan or that
aren't mature enough to be harvested.
I have found that one of the characteristics of a good
program manager is a big lower right-hand drawer in their desk,
and what went into that drawer were all the ideas for
improvement. The drawer was kept closed until such time as we
fielded the first system; that is the time open the drawer and
look to see what development plans we need to deal with
shortfalls or upgrades for that system.
This time-certain development is important because, as time
goes by, the technology gets old, and new ideas are introduced
that end up being disruptive to the process. So, time is a key
factor here.
This stability issue is really key. If I look back through
my whole career, there is only one program I ever worked on
where we actually produced the system at the rate we planned.
That was the F-117. We built one a month. Every other program I
can think of, by the time we were done with development, we
couldn't afford the build rate that we planned.
Senator Chambliss. Anything to add, Mr. Adolph?
Mr. Adolph. Just one thing, since I certainly agree with
the other panel members, but with my background in tests, I'll
add that issue. In the case of the F-15 and F-16, which were
mentioned earlier, I worked on both programs, and I was out in
the field, test business, working for the Air Force.
Chairman Levin. Please talk a little bit louder, if you
would.
Mr. Adolph. Yes. In the case of the F-15 and F-16, I was
working for the Air Force in the field at the time, and the F-
15 test program was structured well; we had 19 test articles
and sufficient articles in the case of the F-16 as well. In the
latter F-16 case, the propulsion system had been matured. In
order to move a program along, you need an adequate number of
test articles, and you need to be able to do as much testing as
you can. Avionics is a good example, where considerable testing
can be accomplished on a test bed platform, rather than the
developmental platform.
So, that's, I think, one key component of keeping a program
moving. When a program stagnates, when you only have one or two
test articles, and you have a `standing army' of test personnel
waiting, and the fixed cost of those people is almost as great
as the incremental cost of doing additional testing,
particularly when you recover the article. That's not the case
in a missile program, where the test article is destroyed on
each test.
Chairman Levin. Thank you, Senator Chambliss.
Senator McCaskill.
Senator McCaskill. Thank you, Mr. Chairman.
Let me start with Mr. Sullivan, and there have been several
references that sometimes we have excessive oversight with no
value added. That's a frustrating comment to me, in that I have
watched the Defense Contracting Audit Agency basically get
taken to task by their peers for failure to even follow basic
auditing standards. I've read, I can't tell how many of your
reports that have, in fact, identified weapons acquisition as
high risk, since 1990. It is not as though the oversight's not
occurring, it's just that it's being ignored. It's not adding
value because nobody's paying any attention to it.
Let me ask you, in that regard, about JCIDS, Planning,
Programming, Budgeting, and Execution (PPBE), and Future Years
Defense Program (FYDP). Now, first of all, I think we ought to
pass a law that they quit talking in initials because you all
know what those things are I just said, but I guarantee you,
nobody that I work for in Missouri has any idea what JCIDS is,
PPBE is, or FYDP. What they are is: one, is all the Services
getting together and basically giving each other what they
want; two, is a 2-year calendar-driven process, where they're
supposed to be figuring how they're going to spend the money;
and three, it is the Secretary of Defense trying to low-ball
what it's going to cost, long-term, in order to make sure that
the other two go along with it. Is that an accurate summary of
what those three are?
Mr. Sullivan. The--that---- [Laughter.]
I would say, number one, that the JCIDS is a requirement-
setting process; it's where all of the Services tend to get
together and figure out what it is that they require.
Senator McCaskill. But, in your testimony, Mr. Sullivan,
you pointed out, they never say no.
Mr. Sullivan. That's true.
Senator McCaskill. Have they ever said no that you're aware
of since you've been looking at this?
Mr. Sullivan. We issued a report for this committee, I
think, about a year ago, where we looked at that, and it was--I
think the JCIDS process then, if I'm not mistaken--this may not
be exact, but I can get it for you--about 90 percent of the
proposals that went in were granted.
[Excerpt of GAO-08-1060 follows:]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Senator McCaskill. Explain to me, so I understand, what is
the makeup of the three groups of people. These three are
really the stovepipes that cause a lot of the problem because
you have, ``Here's what we want, here's how we pay for it, and
here's how we figure out how much it's going to cost in the
long run.''
Mr. Sullivan. Yes, ma'am.
Senator McCaskill. What I don't understand is: what is
keeping them from making that one thing, so they all have to do
all of that at once?
Mr. Sullivan. The requirement-setting process, the JCIDS,
is run by the JROC, and the JROC is made up of the military
service chiefs, more or less. It's run by the Vice Chief of
Staff. So, it, in a sense, is a matrix organization, but it
receives most of the proposals for needs to be validated as
weapon systems from the three Services, as stovepipes.
Now, the JCIDS was established to have something called
``functional capability boards,'' which were supposed to be a
matrixed organization based on looking at things like
battlespace awareness or force protection or force projection,
looking at it functionally instead of across the Services. What
we found when we did that study was that the DOD has not
staffed those functional capability boards properly, it hasn't
resourced them properly, so they don't really do a lot of joint
decisionmaking to send proposals forward to the JROC. So,
mostly what they are receiving is proposals for capabilities
that are coming from the Air Force, the Navy, and the Army, and
they compete with each other.
Senator McCaskill. The sense is, at JCIDS, that if you say
no to what they want, then they're going to say no to what you
want. Isn't that part of the problem, in terms of the way this
is actually supposed to be oversight with no value added.
Mr. Sullivan. I would say that the oversight is far from
perfect. There's not a lot of value to it.
Senator McCaskill. The idea of the Joint Staff is that
they're supposed to be picking winners and losers. What is your
sense of how effective the organization of the Joint Chiefs
office has been, in terms of weapon acquisition and picking
winners and losers?
Mr. Sullivan. In terms of that organization that does that
picking of the winners and losers, the JROC, there are a lot of
redundancies right now in the weapon system portfolio because
they can't make proper decisions, it seems to us.
I'll give you an example. Right now, there are some
unmanned aerial systems that are in development that we
believe, and I think the Department actually believes, should
have been joint programs, but the Services had unique-enough
requirements and missions to be separate. Right now, there is
the Predator, which has been very valuable in Iraq and
Afghanistan and is an Air Force program. Air Force is currently
making a bigger, more powerful Predator, called the Reaper.
Another one of the Services, I believe it's the Army, has
started what they call the Sky Warrior. Both drones are done by
the same contractor and both with very similar requirements,
but the Services have determined that they're different enough
that they each have to have their own acquisition program.
That's the sort of thing that the JROC is contending with; it's
a very parochial kind of an attitude.
Senator McCaskill. So, is this not fixable?
Mr. Sullivan. This is the cultural aspects of this, I
think, that we've all been addressing, where you can write
legislation, you can have policies.
Senator McCaskill. It doesn't do any good.
Mr. Sullivan. But, unless you change the culture, and I
guess that is the number one question: How do you change that
culture, that has been in existence for so long, to try to turn
it a little bit, to do things a little more efficiently? It's
very much a culture issue.
Senator McCaskill. I also think, Mr. Chairman, one of the
things we have to do, in some instances, is take out a mirror
because I think there are times that, when the military has
tried, either by the way they've done the budget or by actually
being so bold as to say we need to wind down a program, that
Congress decides, because of our parochial interests, that it's
important that we go to bat to augment the budget to take care
of the weapon system that we think is important in our part of
the world. So we contribute to this problem, and I think we
shouldn't complete this hearing without at least acknowledging
that sometimes Congress has their hand in this stew.
Mr. Sullivan. Yes, ma'am. I would say, sometimes we should
think of this as a system that is in equilibrium because there
are very many stakeholders in this system that are getting
specific things, even the GAO. It's a pretty good employment
program for us. We report on cost schedule and performance
problems, and we have been doing it 30 years. So, culturally
speaking, if you examine it as a system that, maybe, is in
equilibrium, in a sense--it's not necessarily broken for the
people that are involved in it.
Senator McCaskill. Thank you.
Thank you, Mr. Chairman.
Chairman Levin. Thank you very much.
Just one quick comment on that. We do, in this bill,
attempt to make the JROC system cover some of the issues that
Senator McCaskill talked about by requiring it to make these
early tradeoffs, by looking at cost and at schedule, by the
way, as well as the requirements and the performance
requirements.
Mr. Sullivan. Yes, sir.
Chairman Levin. There is an effort in this bill, Senator
McCaskill. I just want to give you assurance that at least this
bill attempts to do what we can, legislatively, to put those
elements into the JROC process, which would, hopefully, cut
down the parochialism by forcing consideration of cost and
schedule, not just requirements. So, I'd just get that on the
record.
Thank you, Senator McCaskill.
Senator Martinez is next.
Senator Martinez. Mr. Chairman, thank you very much.
Dr. Gansler, I wonder if I could ask of you to define for
me a little more broadly what you mention as a ``holistic
approach'' to defense needs.
Dr. Gansler. Yes. In fact, when I even looked at the
situation in Iraq and Afghanistan, I was surprised to find that
the State Department, United Sates Agency for International
Development, and DOD were all there, but not integrated, in
terms of the buying, the contracting, and planning purposes.
But, much more at a higher level than that, it seems to me that
the world of the 21st century is going to require us to combine
hard and soft power in the kinds of operations involving
expeditionary operations or insurgent operations that we get
involved with around the world. So, we're going to need a much
closer tie between, in that case, State and Defense. But, I
would go further and say, Homeland Security and Defense. It
shocked me, in fact, that Saddam Hussein didn't try and pull
some terrorist actions at the same time as we attacked him. I
would expect that's going to happen in the future. So, it is an
integrated, holistic perspective that is the meaning of
``national security'' in the 21st century, that will involve
Homeland Security, that will involve State, will involve the
intelligence community, will involve the DOD.
I think that's basically what National Security Advisor
General Jim Jones is now trying to do, some of the
restructuring that's taking place at the national security
level within the President's office. I think this combination
of soft and hard power is going to be required, very clearly.
That's what I was thinking of, in terms of the ``holistic
perspective.'' But, also the types of threats. Think about it,
the energy case, the pathogen spreading worldwide, the economic
crisis that we're in, these are all national security issues
for the 21st century that we have to start to incorporate into
our thinking of national security. That's what I had in mind.
Senator Martinez. Thank you. I think it is an intriguing
future that we all are stepping into. I think, by the way,
Saddam would have if he could have.
But, the LCS program is one that I'm very much fond of, and
one that I think is essential to the national security
interests of our country. I wonder how you believe the Navy got
so far off track on that particular program? Was it too many
requirements being put on the platform by the Navy? Was it the
length it's taken to develop it? We now have two hulls being
developed. So, to the extent that any of you could speak to the
LCS program and what you see, going forward, it would be
helpful to me.
Dr. Gansler. I just actually published, I think last week,
a DSB report which looks specifically at the LCS and the
presidential helicopter, a couple of programs of that sort,
where the initial concept was: get something relatively fast,
take something ``off-the-shelf'' that could be used, and
addressing Dr. Kaminski's point about rapid acquisition--then
maybe block 2, 3, and 4 would add some of the additional
things.
What happened on the LCS: the first thing they said was,
``Has to go through Sea State 8.'' Well, that's like going
through a hurricane and it wasn't initially designed for that.
Then the next thing they said was, ``Well, it has to have a new
Navy sprinkler system.'' The sprinkler that was in the system
in those two ships that you talked about, it wasn't going to be
adequate, for some reason or other. So, each of these special
requirements ended up basically changing the original block 1
system and introduced the instability, cost growth, and
schedule impact that we've talked about in all these other
programs.
Yes, we badly needed the LCS, but is it going to have to be
a battleship? Does it have to do everything that a battleship
does? How it's going to be used by the Navy was resisted, in
terms of the nontraditional solutions. So, this is the culture
change that we've been talking about, as well. If you kept the
cost and the schedule, and got a block 1 system out there much
faster--I like the idea that we did it competitively. I think
that was a very important step. So, I would encourage that to
be done in these earlier demonstration systems.
Senator Martinez. Dr. Kaminski?
Dr. Kaminski. Yes, sir. I think the concept initially was a
good concept. What was missing was the upfront systems
engineering and development planning that I spoke about. I can
remember back to the time I was serving in the acquisition
executive's job in DOD; I was a big proponent of commercial
practice, and also in buying commercial systems where that made
sense. But, if somebody came to me and said we were going to
buy a commercial ship for this mission, my first question to
him was, ``What are we going to change in the mission to be
able to do it with a commercial ship?'' If the answer is
``nothing,'' then I have to ask a second question, ``Wait a
minute, this commercial ship doesn't have the kind of military
requirements you would have for fire-safe cables, for a fire
sprinkler system, or a whole variety of things. What are you
going to do about those?'' I don't think we started asking
those questions about the LCS program until we were well into
the program, so we missed this upfront set of tradeoffs. Those
are tradeoffs that you have to make. They can be made sensibly
if you approach them, understand them, look at the costs, the
performance, and the schedule to make those tradeoffs. I don't
believe we made those tradeoffs upfront. That, for me, does not
necessarily damn the LCS program. There may still be value
derived from looking at these tradeoffs and now making sensible
decisions to go forward. I agree with Dr. Gansler about the
advantage of having a competitive environment to be able to do
that.
Senator Martinez. Mr. Adolph?
Mr. Adolph. I'm not really familiar with the LCS. I've had
no involvement with the program.
Senator Martinez. Okay, and you, Mr. Sullivan? I don't know
whether you've had any experience.
Mr. Sullivan. I don't have any specific experience with
that, but we do have a team that looks at all of our Navy
ships, and I'm sure we've had a report on that recently, which
I could submit for the record.
Senator Martinez. That would be great. I'd appreciate that.
Mr. Sullivan. Okay.
[Excerpt of GAO-05-255 follows:]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Senator Martinez. I'm sure going to try to look up your
article, as well, Dr. Gansler.
You haven't written on DGD-1000 and the DGD-51 to date,
have you?
Dr. Gansler. Not lately.
Senator Martinez. Okay. Do any of you share an opinion on
the needs of the Navy as it relates to these two programs?
Mr. Sullivan. I would just say that we're going to come out
with our annual assessment of programs, and I think both the
DGD-1000 and LCS are going to be programs that are covered in
that, so you will get our take on them, probably in the next
couple of weeks. I'll follow up, as well.
Senator Martinez. Very good. Thank you.
[Excerpts of GAO-09-326SP follows:]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Senator Martinez. My time's up, thank you very much.
Chairman Levin. Thank you, Senator Martinez.
Senator Begich.
Senator Begich. Thank you. Thank you very much, Mr.
Chairman.
It has been very interesting, listening to all the
testimony. I will try to ask my questions very quickly, and I
may not ask every single one of you to answer because I want to
get some specifics here.
Before I talk about the personnel issue, I'm a former
mayor; I've just become a Senator. So, I believe I'm a mayor
who just happens to be a Senator, so I like some of the
comments you've made on personnel issues that I'll talk about
in a second.
I'm not familiar with the CAIG. Is that the CAIG? I looked
through the bill as quickly as I could, and I see an annual
report to Congress. Do you think it would be helpful for us to
require more? I believe it's a combination of oversight; it's
not just internal oversight, but this body needs to do more. I
think this bill attempts to do that. But, do you think there
should be a report--and maybe there is and I'm just not
familiar with it--that comes to this committee on a regular
basis, maybe quarterly or twice a year, that shows what the
CAIG said it would cost before a system is started?
Mr. Gansler, you made the comment that--I may get these
words wrong--but it's almost like they ignore it. It comes out,
it says, maybe, what it might really cost, but then they kind
of push it down. I'm familiar with this, as a mayor. We call
them HMS, Inc., studies, where the contractor wants to build
it, as well the person inside the system who wants to build it,
always seems to have a different price. When it's all done,
it's pretty close to the one that HMS, Inc., did. Is that
something we should have in this legislation, so we can see,
before these systems start kicking off, here is what the real,
or what another group said it could cost, so we at least have
some understanding? I don't know who can answer that.
Dr. Gansler. The idea of the independent cost analysis,
right upfront, when we're doing the early systems engineering,
is to be able to see the impact of the various requirements,
and to be able to trade requirements, schedule, and cost as
part of that early design requirements setting. Cost is
actually an engineering challenge, just as schedule and
technical performance are. So, if you, upfront, try to say,
``What's the cost impact of this?'' and the independent cost
analysis group will say, ``Historically here's what it's
been,'' and then people will come back and say, ``But, this
time it's going to be different.''
Senator Begich. Everything's different the next time.
Dr. Gansler. Right, and so it's important to keep that in
perspective, that the new technology comes along, and so forth.
But, it's absolutely essential to get that independent
estimate. Now, do you want to have Congress legislate what the
price should be? I don't think so.
Senator Begich. No, I'm not asking that. But here we are,
complaining about all these cost overruns. I agree with all
your comments. You can write all the legislation you want, but
you do not change the culture and remove people who are not
doing the job they should be doing, and putting people in there
who should be doing the job, you don't change anything, we'll
be back here in a couple of years. So, I'm not saying,
legislate the price, but we become more aware, so we are
putting the pressure where it should be. Let's be honest about
the pricing, so when we do the budgeting authorization and
appropriation, we don't go from 75 to 95, with two-thirds
probably in the planning and design stage, and we always get
the answer, ``We're this far, we have to do a little bit more,
a little bit more, a little bit more.'' So what I'm asking is,
should we have a more regular reporting period? Because once
you're a year into a project, even though some are longer,
they're already obligating more money. By the time this system
here moves, you could be 2 years into having any commentary on
it.
That's my question. Should it be more regular reporting?
Dr. Gansler. I think the thing you're missing is the fact
that, in the development of a program, the program changes
weekly. There's always technical issues that come up, there's
always schedule problems, there's personnel problems, and so
forth. You don't want Congress to be micromanaging the
programs. You do want to make sure that the process is a good
process. That's where I think the independent cost analysis is
a very important thing and that your emphasis on it is the
right thing to do. But, I don't think you want to get down to
the point where you have a weekly report from the DOD.
Senator Begich. I didn't ask that. I'm trying to get to the
point. Is annual enough?
Dr. Gansler. I think that--if you've convinced the DOD that
you care about this, that you are going to be monitoring it,
and that's what you're trying to do with the threshold numbers
and the controls on that--I think that you're giving the
message that the DOD needs to care about cost.
Senator Begich. Okay. Let me make one comment. I forget who
said it, about, ``stop the obligation of money at a certain
point.'' I know, as a mayor, that's what you do. You turn the
dollars off, and suddenly you get a response, and you get
people paying more attention. So, I might be a little different
than earlier comments that were made. I'm a little more direct
on that. But, the personnel issue, to me, this is the
challenge. If you don't change the culture, nothing changes.
We'll be back here, and the numbers will grow, as they have
over the years. It's not about adding more people, and I think
Congress made a huge error by reducing down the amount of
people. That was a huge mistake. We basically took the people
who manage our programs out of the equation.
So, besides putting more people into the system to make
sure we have more folks out there, do you think--and I'm not
sure I want to ask this question because I'm not sure you'll
want to answer it--but, do you think, within the system that
currently exists today, we have to change the deck? When I say
``change the deck,'' change personnel, people, and not just add
more people and move people around so we satisfy their issues.
This is a very hard, direct question. I've had to do this, as
mayor. You might have half a dozen, a dozen, or 100 people; it
doesn't really matter--they're in the wrong place. Anyone dare
to want to answer that question?
Mr. Sullivan. I would just say, just real quickly, that a
lot of this is organizational. I think the people that work in
the DOD now are great people, and really, really capable
people, and good public servants and everything else.
Senator Begich. Right.
Mr. Sullivan. But, when we talk about culture, I guess,
it's more the way things are organized. For example, I think,
in our written statement, we have that the Under Secretary of
Defense for Acquisition, Technology, and Logistics probably
should have more ability to make the final decisions on things,
acquisition, than he or she has now. One of the things that
probably gets in the way of that is the fact that that position
had an average turnover of 20 months since it started in 1986.
I think that's part of the problem; there's an accountability
issue, people change over too much, there's not a lot of direct
communication. The three processes, I think, that this
legislation does address, that you're trying to get the three
big processes that we've talked about to communicate with each
other more and to share in decisionmaking; right now, that's
not there. So, I don't know if the people are good people. The
structures and the way they're organized and the way they come
and go is the big issue.
Dr. Gansler. If I might comment that if we implemented what
Goldwater-Nichols says to do, relative to promotion potential
for the acquisition community, that you'd get a big step
forward there. Instead of putting someone into a four-star
position who has no acquisition background, but happens to be
called an acquisition job, that's where we lose out. Each of
the promotion-potential reviews and so forth need to really
show that we value the acquisition workforce, civilians and
military, and that's a critical point, I think, in order to
keep people coming in. Dr. Kaminski said they're not doing it
for the money, they're doing it so they can have an impact, and
they need to have promotion potential.
Senator Begich. I know my time's out, but you're about to
jump out of your seat, Mr. Adolph.
Chairman Levin. Can you do it real quickly?
Mr. Adolph. Very quickly. There are three issues: numbers,
training, and people. Particularly in the Air Force, they need
to plus up because they drew down their acquisition workforce
to a greater extent.
I think the training that Defense Acquisition University
provides is on the mark, for the most part. They were a part of
our study. They ground in what we found into their training
courses. I believe people need to be moved around a bit more,
particularly in the civilian workforce, so that they get a
variety of experiences rather than 1 or 2 years' experience 10
times.
Senator Begich. Thank you very much, Mr. Chairman.
Chairman Levin. Thank you, Senator Begich.
Senator Lieberman.
Senator Lieberman. Thanks, Mr. Chairman. Thanks to the
panel. It's probably not been an exciting hearing in content,
but it's actually very important, an educational hearing, and
there's a lot on the line in it, so I thank the four of you for
the accumulated experience and wisdom you've brought to the
table. I thank Senator Levin and Senator McCain for their
legislation, S. 454, which certainly would take us forward in
significant ways. I hope it passes.
Earlier, Senator Collins referred to the fiscal
responsibility summit that President Obama convened last week
at the White House, and then a group of us on this committee
happened to be in a breakout session on procurement reform. As
Senator Collins indicated, most, though not all, but most, of
us focused on the acquisition workforce, and the size of it. I
want to talk to you about that in the time I have, hopefully,
and at least one other subject we talked about.
Dr. Gansler, you had a chart here in your testimony which
has two lines; one is procurement by DOD, in dollars, from 1990
to 2006, about as close as we can get to today, and then the
other is the acquisition workforce. Obviously, the procurement
dollars go up dramatically as the acquisition workforce goes
down. But, beneath that, there are some numbers that are quite
stunning, I think, and I know when I was here, Mr. Sullivan
referred to them. The acquisition workforce, in 1990, was
actually 500,000 people, and today it's dropped, but it's still
200,000 people, which is an enormous number of people in
acquisition. I note in your testimony that you focus in on the
DCMA and say that it had 25,000 people in 1990, down to 10,000
today. Then the other four general officers, and down to zero
today. Give me some sense of the 200,000. Because my first
reaction to it is, ``Wow, 200,000 people, isn't it enough to
handle acquisitions by the DOD, even though acquisitions are so
large?''
Dr. Gansler. A large share of those are in the military
depots that you have insisted do 50 percent of the maintenance
work. A depot that has 20,000 people, that adds up pretty fast.
To get to 200,000, you only need 10 of those.
Senator Lieberman. So, acquisition wouldn't be what most of
those do?
Dr. Gansler. That's part of the acquisition workforce
because logistics is part of the acquisition.
Senator Lieberman. Got it. But in the conventional meaning,
I think that's important, so I wanted to bring out.
Dr. Gansler. Very few people are actually doing contract
work, program management work, or things like that. As Mr.
Adolph pointed out, the test and evaluation community is down
significantly, but they're part of that community. So it's the
total encompassing the research labs that the government runs
all the way through the maintenance and logistics support.
Senator Lieberman. Is there any way, now or maybe
afterward, to submit for the record, you could give us a sense
of what we would normally, in conversation, consider to be the
acquisition workforce--how many people in the DOD are actually
involved in acquisition, contracting, et cetera?
Dr. Gansler. It's a small percentage of the people that are
actually involved in that.
In other words, you have the comptroller people, you have
the personnel people, you have the policy people, and you have
the acquisition workforce.
Senator Lieberman. Right. I'm curious as to really what the
size of the real acquisition workforce is, leaving out the
depots and the rest.
Mr. Sullivan. I don't have that answer right now. I know
we're doing work on that.
Senator Lieberman. Good. Get it for the record, please.
Mr. Sullivan. Yes, sir.
[The information referred to follows:]
In our recent review of the Department of Defense's (DOD)
acquisition workforce management (GAO-09-342), we found that, according
to DOD data, the number of civilian and military personnel in the
Department's acquisition workforce totaled about 126,000 at the end of
fiscal year 2007 compared to about 129,000 personnel in 2001, a decline
of about 2.5 percent. During this same time period, the number of
contracting actions valued at over $100,000 increased by 62 percent and
dollars obligated on contracts increased by 116 percent, according to
DOD. Moreover, DOD has reported that the number of major defense
acquisition programs has increased from 70 to 95. To augment its
declining in-house acquisition workforce, DOD has relied more heavily
on contractor personnel. However, DOD does not collect or track
information on contractor personnel, despite their being a key segment
of the total acquisition workforce.
Senator Lieberman. There were two other things that there
seemed to be an interesting consensus on, and they're quite
different, about, you might say, principles for acquisition; a
little different than anything we've talked about today.
They're not unfamiliar. One was that our original position on
acquisition should be to favor fixed-price contracts over cost-
plus, and to favor competitive bidding, as opposed to
negotiated contracts. I want to ask, to the extent that we have
time--let's just focus on the fixed-price. Our sense, as we
discussed at our breakout session with probably about 25
people, was, generally speaking, private sector favors
competition. So, why are we favoring cost-plus? Does the
taxpayer really benefit from that?
Mr. Adolph, you always get asked last because we're going
left to right, so let's start from the right and ask you about
that.
Mr. Adolph. I think the other three panel members have more
expertise in this issue. But, when you're in the basic research
arena, it's very difficult to do on a fixed-price basis. It
should be accomplished using cost-plus contracts.
Once you get beyond research, then the next challenge in
the development is system integration. That's not an
insignificant task with the very complex systems we're
developing today. But, once you get beyond that point then I
think you reach a point where you can really consider going to
fixed-price for downstream procurement.
Senator Lieberman. Right. That's interesting.
Dr. Kaminski, I think, just to clarify--if I could really
make it too simple--what if we had a law that said, ``Defense
contracts ought to be done on a fixed-price basis unless the
Secretary certifies that there's a good reason not to''?
Dr. Kaminski. I think that would end up requiring a lot of
certification, Senator Lieberman, for the following reason.
Senator Lieberman. Right.
Dr. Kaminski. I believe fixed-price contracts are
completely appropriate when we know precisely what it is we're
going to buy. If there's uncertainty in what we're going to
buy, and we know we're going to change, and we don't know yet
quite how we're going to change, I think we end up on the wrong
end of the bargain negotiating a fixed-price contract and then
having to go back and renegotiate that effort for every change
that occurs, especially when the contractor has already priced
in some contingency in the fixed price.
So, there's a time, for example, in the program, where
perhaps we are working through this in development, and then we
settle in on what we want to buy, and we're ready to enter a
well-defined production program. That would be a fine time to
do a fixed-price contract.
So, I think you have to pass that criteria, knowing what it
is you want to buy, and have it be predictable.
Senator Lieberman. Okay. My time is running out, but, Mr.
Sullivan, Dr. Gansler, if you'd give me a quick answer to a big
question and follow up with writing.
Mr. Sullivan. For major developments, I think it would be
very difficult to go to a fixed-price contract for that because
of the unknowns that are involved. But, I would say that if you
work on requirements and try to do some of the things that
we've been talking about here today, in terms of staying in
what is doable, and having shorter cycle times to get these
things done, you could have cost-plus development contracts
that don't get so out of control. It really goes back to how
much knowledge you have when you set out.
Senator Lieberman. Okay.
Dr. Gansler?
Dr. Gansler. I think that it's a question of what's the
meaning of a ``fixed-price contract''? As Senator McCain said,
you get 75 changes every week. The contract continues to change
hourly, in effect. I think it's very clear, when you have a
stabilized and lower-risk program, that a fixed-price makes a
lot of sense, it does give an incentive for the contractors. On
the other hand, the cost-plus, I would say, we haven't been
using the incentives that are available with the cost-plus-type
contracts as well as we should, and I think, clearly, for
research-and-development-type activities, cost-plus is an
appropriate way to do it, but the ``plus'' part is an incentive
rather than a fixed fee, I think. I would use the incentive
more.
Senator Lieberman. Those are very helpful answers. You
encourage me to think that we ought to take a look, not at
fixed prices on across-the-board answer, but to apply it by
some selective means, and that, in doing so, we might benefit
the taxpayers.
Thanks very much.
Chairman Levin. Thank you, Senator Lieberman.
Senator Webb.
Senator Webb. Thank you, Mr. Chairman. I'd like to add my
appreciation for your taking on this issue. As has been so
clear today, we're burning up a lot of money, and we're not
getting a lot of product right now, particularly in the
shipbuilding programs and aircraft programs.
Dr. Gansler, when you're talking about people, which
everyone seems to agree is the major issue, I was thinking
about all the different years and different positions I've had
on different sides of the table, here, working on this issue.
It's so clear that what we need is disciplined management, not
only on the people side, but in the system itself. ``People''
include people in government, on this side of government, it
includes people in business. We have challenges because there
are not a lot of people in the military who use the business
concepts, quite frankly, and they're asked to manage these
programs. There are not a lot of businesspeople who are used to
how product comes through a governmental system.
I believe it was Mr. Sullivan who mentioned the creation of
the Under Secretary of Defense for Acquisition in 1986. I was
actually on Caspar Weinberger's staff when that position was
created. We had a very talented individual who came into the
position. He was bewildered with all the different steps that
were required to get a system through the process. It's
something you just don't see in the normal business world.
There are lots of checks and balances. Some of them are
appropriate, and some of them are less than appropriate.
Dr. Gansler, I found your analogy with the LCS brought back
a lot of different memories. They say that a camel is a horse
created by a committee. I can go all the way back to the M-16,
when they first developed the M-16, and they developed it
directly toward a jungle environment, and then the different
requirements were put on it to be able to be used in the desert
and other different places. They put a different round in it
that carboned up the chamber so people were dying in Vietnam
because the weapon system requirements had changed as it
evolved.
I was in DOD when we were trying to do the Bradley fighting
vehicle, and there were different requirements put on it here
in Congress, so that it was very similar to what you were
talking about with the LCS. They were saying it should perform
different functions from the original design, and then there
were all these press reports about the Bradley fighting vehicle
falling over when it was going through a water obstacle because
it got top heavy.
Or the FG-7s, the USS Stark-class ships, which were
designed to build to cost. So, we have fixed cost that we were
going to build a ship toward, and then you go inside one of
these FG-7s, you could plink the bulkheads on a FG-7, they were
so thin. So when an Exocet missile hit the USS Stark in the
Persian Gulf when I was Secretary of the Navy, it went all the
way through the ship because they had had to make adjustments
based on the cost rather than on some other areas.
So it's a very complicated question. I think the key, when
I look back, is if you can find the right leadership at the top
on a program, negotiate and agree on general requirements--
there are always going to be fixes--we'll get a program
through. Probably the best example of that is when they put Al
Gray, who later became Commandant of the Marine Corps, and he
had the Development Center, they put him on the Light Armored
Vehicle, and he got that program through in about a year. He
just pushed it through, made all the negotiations, was very
firm with people over here in Congress, as to what the
requirements were for the Marine Corps, and it was a very
successful program.
I have one question I would like to put in front of you
because I'm very concerned about it, and that is the state of
all these programs in the United States Navy right now. You
talk about the Polaris as having been probably the best
analytical prototype of how to build a weapon system. One of
the things about our submarine programs is that we built the
frame, and then we added the technology onto the frame, similar
to, say, the C-130, rather than continually building a new
frame with all the costs and the time that goes into that. I've
just been really struck over the last couple of years at how
difficult the Navy procurements appear to be, and I'm trying to
get my arms around why. I would be happy to hear from any of
you who would like to begin answering that question.
Dr. Kaminski. I'll make one comment. I think a key thing
that impacts the programs are stability. So, if you see a
program that we've had in production for some period of time,
they have very good learning curves on those ships when we have
a stable production program. Our issues sometimes are with
first-ship cost. But, if you look at the cost of subsequent
ships, what's happening there is very competitive with
commercial-kind of production experience. Where we're producing
ships regularly--one of the ship families in which we're doing
that is the DDG-51. We have two yards. We have some competitive
arrangements. Not quite head-to-head competitions, but there
are some incentives in those programs. But, it is a well-
planned, stable program that we've been producing. I think that
approach would benefit us. It's the areas where we've had
instability where we've more problems.
Dr. Gansler. Yes, I guess I would approach it by thinking
about, ``What is it the Navy really needs for the 21st
century?'' and what types of ships they're going to want. There
is a resistance to change. Take the Arsenal ship, for example,
which was primarily to support the Marines and the Army
onshore, it was resisted significantly in terms of it being a
low-cost ship for few people. The highest costs in the Navy are
for people and fuel if you look at the life-cycle costs of a
ship. So trying to drive down the number of people on the ship
and improve the fuel utilization are things we need to stress.
Those are not the traditional things that are emphasized in the
Navy construction of ships. I think it's a different look that
we need to think about. The LCS has the same thing. Is that
something the Navy really wanted or really resisted? So it's
more the institutional inertia that has to be changed, I think,
in terms of what the future Navy for the 21st century is going
to need.
Senator Webb. Do any of you see this as leadership failures
in the Navy? [No response.]
In terms of defining these objectives?
Dr. Gansler. There have been some real successes. The F-
18E/F on the Navy program was extremely well-managed, but that
was because they had some really top people doing it, they had
a clear objective, it was a incremental version of a prior
demonstrated program, and it was well done.
Another big success is the patrol frigate. It kept up
competition throughout its life, and it had the steepest
learning curves of any ship in the Navy. So, there are some
success stories, but I think lessons learned haven't been
widely applied.
Senator Webb. Thank you, Mr. Chairman. My time's up.
Chairman Levin. Thank you, Senator Webb.
Any other questions of this panel before we excuse them?
[No response.]
Thank you all for your time. Some of you came some distance
to get here, at some inconvenience. At least one of you had to
give up a family commitment, and we won't identify who that was
because the family is better off not knowing, maybe.
[Laughter.]
But we're very grateful to all of you for your testimony.
It's very, very helpful.
I will also submit for the record the text of the Weapon
Systems Acquisition Reform Act of 2009; a statement from Ken
Krieg, former Under Secretary of Defense for Acquisition,
Technology, and Logistics; a statement from Moshe Schwartz from
the Congressional Research Service; and the DOD Inspector
General Acquisition Workforce Count Report.
[Additional material included for the record.]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
[GRAPHIC(S)] [NOT AVAILABLE IN TIFF FORMAT]
Chairman Levin. We'll stand adjourned.
[Questions for the record with answers supplied follow:]
Questions Submitted by Senator Joseph I. Lieberman
STATUS OF CURRENT ACQUISITION SYSTEM
1. Senator Lieberman. Mr. Sullivan and Dr. Gansler, I have several
concerns with the acquisition system as it currently operates. Most
importantly, it seems that the requirements validation process under
the Joint Capabilities Integration and Development System (JCIDS) is
not synchronized with the Planning, Programming, Budgeting, and
Execution (PPBE) funding process. Can any effort at reform succeed
without addressing this fundamental flaw in acquisition management?
Mr. Sullivan. We do not believe that there can be success until
there is better synchronization between the Department of Defense's
(DOD) requirements, funding, and acquisition processes. The Government
Accountability Office (GAO) has previously reported that the lack of
integration between the JCIDS and PPBE processes is a key factor that
contributes to the Department's inability to achieve a balanced
portfolio of weapon system programs that meets the needs of the joint
warfighter and matches needs with available resources. The JCIDS
process has not been effective in prioritizing needs from a joint,
Department-wide perspective, and it largely approves capability needs
without accounting for the resources or technologies that will be
needed to acquire the desired capabilities. Resource allocation
decisions in DOD take place through the PPBE process, which is separate
from JCIDS. PPBE largely allocates resources on a Service-by-Service
basis and does not effectively link resources to capabilities. In
addition, the process allows too many programs to start development
with unreliable cost estimates and without a commitment to fully fund
them. Until DOD establishes a more integrated approach to weapon system
acquisition, it will continue to struggle to effectively prioritize
warfighting needs, make informed trade-offs, and achieve a balanced mix
of weapon systems that are affordable, feasible, and provide the best
value to the warfighter.
Dr. Gansler. I believe ``cost'' should be an essential part of the
``requirements process''--since, in a resource-constrained environment,
it directly determines ``numbers'' (of systems procured) and that
determines force effectiveness (even more than individual weapon's
effectiveness).
2. Senator Lieberman. Mr. Sullivan and Dr. Gansler, what are your
recommendations for effectively balancing the requirement validation
process with the calendar-driven funding process?
Mr. Sullivan. To improve DOD's ability to deliver weapon systems at
the right time and right cost, GAO has recommended that the Department
implement an enterprise-wide portfolio management approach to making
weapon system investments that integrates the determination of
warfighting needs with available resources and cuts across the Services
by functional or capability areas. Such an approach would focus on
assessing weapon system investments collectively from an enterprise
level, rather than as independent and unrelated initiatives. By
following a disciplined, integrated process--during which the relative
pros and cons of competing weapon system proposals are assessed based
on strategic objectives, warfighter needs, and available resources, and
where tough decisions about which investments to pursue and not to
pursue are made--DOD could minimize duplication between Service
components and more effectively support each new development program it
commits to. Furthermore, to ensure effective weapon investment
decisions are made, GAO has recommended that a single point of
accountability be established at the Department level with the
authority, responsibility, and tools to implement portfolio management.
Dr. Gansler. My answer to question #1 (above) will greatly aid in
this; but, in general, the ``requirements process'' must look well
beyond the near-term budget (funding) process. Nonetheless, the
frequent budget changes (particularly, by Congress) introduce great
instability and inefficiency into the implementation of the weapons
acquisition process.
JOINT REQUIREMENTS
3. Senator Lieberman. Mr. Sullivan and Dr. Kaminski, I am concerned
that DOD has historically done a poor job of procuring capabilities
that address joint requirements, and more specifically, that the
Services have little incentive to spend what they see as their limited
budgets on capabilities that will principally benefit sister Services.
One particular instance of this is the Multi-Platform Radar Technology
Insertion Program, and the Air Force's apparent reluctance to procure a
system that will principally serve the Army's battlefield management
requirements. How can the acquisition process better prioritize and
procure such crucial joint capabilities?
Mr. Sullivan. GAO has recommended that DOD should determine and
allocate appropriate resources for more effective joint capabilities
development planning. The functional capabilities boards, which were
established to manage the JCIDS process and facilitate the
prioritization of needs, have not been staffed or resourced to
effectively carry out these duties. Similarly, the combatant commands
(COCOMs) also lack the analytic capacity and resources to become more
fully engaged in JCIDS. GAO recently reviewed JCIDS documentation
related to new warfighting capabilities and found that most--almost 70
percent--were sponsored by the military Services with little
involvement from the joint community, including the COCOMs, which are
responsible for carrying out military missions. The Services drive the
determination of capability needs, in part because they retain most of
DOD's analytical capacity and resources for requirements development.
Within the Department's PPBE process, the individual military
Services are responsible for budgeting and allocating resources under
authority that is commonly understood to be based on title 10 of the
United States Code--to organize, train, and equip military forces. In
this structure, the budget is based more on individual, service-focused
needs than on joint warfighting needs. In the past, GAO has reported
that this situation has contributed to interoperability problems among
weapon systems and unnecessary duplication of capabilities in some
areas. The Office of the Secretary of Defense (OSD) reviews and makes
adjustments to the Services' budgets, but this takes place toward the
end of the PPBE cycle when it can be difficult and disruptive to make
changes, such as moving funds to higher-priority programs or to support
joint needs. DOD has recently taken steps to establish capability
portfolio management in selected areas in an attempt to overcome the
service-centric nature of the resource allocation process. However,
because the portfolios are largely advisory and lack authority and
control over the allocation of resources, there may be limits to
achieving joint capabilities.
Dr. Kaminski. The priority must be set in the budget process, with
key oversight from OSD and the Joint Chief of Staff. The procurement
can be improved by first developing (a several year process) then
assigning key program management personnel with the requisite training
and domain experience. Alignment of the responsibility, authority, and
accountability of the program manager will require that a degree of
trust be established between the program manager and those responsible
for our oversight mechanisms. We must be prepared to delegate authority
to the program manager, and provide him or her with some flexibility to
manage--to adjust levels and allocation of funding, to adjust the
allocation of performance parameters, to adjust schedule, and to tailor
the acquisition approach to be responsive to the need. Clearly, there
must be bounds established beyond which the program manager must seek
approval from oversight authorities. But I believe these bounds are too
narrow and inflexible today. We also need to provide sufficient upfront
funding, and maintain funding stability throughout program execution.
4. Senator Lieberman. Mr. Sullivan and Dr. Kaminski, are either the
JCIDS or PPBE as currently constructed able to overcome parochial
service interests in this regard?
Mr. Sullivan. No, I do not believe that the JCIDS and PPBE
processes are constructed effectively to address the service-centric
manner in which weapon systems are acquired in DOD. Past studies
chartered by DOD and other organizations have reported similar
conclusions and made a number of recommendations to improve the
acquisition of joint warfighting capabilities.
Dr. Kaminski. Both help, but I don't believe they are sufficient.
DEPARTMENT OF DEFENSE SCIENCE AND TECHNOLOGY PROGRAM
5. Senator Lieberman. Dr. Gansler and Dr. Kaminski, as we try to
reduce risk in acquisition programs, what role does (or should) DOD
science and technology (S&T) program play?
Dr. Gansler. S&T are critical to both ``staying ahead'' (which is
the basis of our national security strategy of ``technological
superiority'') as well as to risk reduction--since we need to prove out
the S&T first, before introducing it into weapons acquisition.
Dr. Kaminski. With good S&T programs supporting development
planning, we can assign managers to develop prototypes, critical
systems or components needed to better understand cost/performance
trades and reduce risk. With the appropriate technology base in place,
it is reasonable to expect that many of these developments can be
completed in 2-4 years, so one manager will be in place from start to
delivery of those critical subsystems during the development planning
phase. This should allow the full system development to proceed on a
shorter schedule as a result of the risk reduction.
6. Senator Lieberman. Dr. Gansler and Dr. Kaminski, do you think
that the Department's S&T programs, and in particular the DOD
laboratories, are sufficiently resourced and staffed to provide support
to the acquisition programs and address the problems you are discussing
today?
Dr. Gansler. The DOD's S&T program must be adequately funded to
``stay ahead''; and, traditionally, whenever there is a shortage of DOD
money it is one of the areas the Services tend to cut (since it is
long-term). Thus, in the likelihood of a coming budget crunch, it must
be protected--not just in the DOD labs, but particularly in the
universities and in industry.
Dr. Kaminski. No. We are short in both quantity and quality of
staffing, and are not using the latest tools to support early modeling,
simulation, and testing.
7. Senator Lieberman. Dr. Gansler and Dr. Kaminski, what
recommendations do you have for this committee to improve the quality
of the DOD S&T program and the quality of the DOD laboratories?
Dr. Gansler. Adequate funding, and allowance for foreign graduate
students, scholars, and researchers (per Presidential Decision
Directive-189) to work on DOD S&T, along with ``peer review'' of all
proposals, are critical to the United States staying ahead. The SBIR
and STTR programs are also a great benefit, and must be maintained.
Dr. Kaminski. The first step is to ensure that we not only restore,
but also enhance early and continuing systems engineering coupled with
the effective development planning needed to drive our S&T program.
This will require commitment of more significant investment dollars
earlier in our acquisition programs, and a commitment to build a cadre
of systems engineers and development planners with the education,
training, and domain experience needed to be effective. Attracting
``best and brightest'' to this work--and keeping them--will require a
personnel system that will identify and track these important human
resources and establish a career path to allow those who are successful
to advance to senior program management and leadership positions. Their
domain experience will be enhanced by managing the building of critical
technology demonstrators and subsystems during the development planning
program, reducing risk, and building skills and experience at the same
time.
______
Question Submitted by Senator Jack Reed
S. 454 LEGISLATION
8. Senator Reed. Dr. Gansler, in the proposed S. 454 legislation,
section 203 focuses on maximizing competition throughout the life cycle
of major acquisition programs. One proposed provision is that modular,
open architectures are used to enable competition for upgrades. Please
comment on how the implementation of this recommendation would
positively impact major weapons programs that have been sole sourced
for decades, and how the Services can rapidly implement these changes
to achieve maximum benefits. In addition, please comment on the
feasibility of applying this provision to current programs, not just
new programs.
Dr. Gansler. Let me begin by noting that the objective should not
be ``maximizing competition''; rather, it should be ``maximizing the
effectiveness of competition.'' Competition for its own sake, can be
detrimental. For example, there must always be the ``option'' of
competition, but if a firm is constantly improving performance and
lowering costs, they should be rewarded by not recompeting the program;
but there must always be the option of introducing competition if costs
are rising (so the ``threat'' of competition is what creates the
incentive for lowering costs; and the follow-on is the reward).
Open architectures are clearly one way to allow competition to be
introduced at the subsystem level (which often is 70-80 percent of the
total weapons cost); so it should be greatly encouraged--both on
upgrades of current systems and on new ones.
______
Questions Submitted by Senator Daniel K. Akaka
WEAPON SYSTEMS ACQUISITION REFORM ACT
9. Senator Akaka. Mr. Sullivan, section 105 of the Weapon Systems
Acquisition Reform Act recommends the Joint Requirement Oversight
Council (JROC) seeks input from the combatant commanders in identifying
joint military requirements. Currently, the combatant commanders
produce an annual Integrated Priority List (IPL) that attempts to
satisfy this same goal. In your prepared testimony, you stated some of
the combatant commanders do not feel their needs are sufficiently
addressed. What must be done within DOD to fully implement the intent
of this legislation to ensure combatant commander's influence
throughout the acquisition system to decrease the Service-centric
environment we've seen for years?
Mr. Sullivan. GAO and several other studies have recommended that
DOD increase joint analytic resources for a less stovepiped
understanding of warfighting needs. While each COCOM submits an IPL to
the Chairman of the Joint Chiefs of Staff each year, which defines the
COCOM's highest-priority needs, it does not appear that the COCOM needs
are well integrated with JCIDS. According to officials from several
COCOMs that we have met with, needs identified through IPLs are not
typically developed into JCIDS capability proposals. Some COCOM
officials pointed out to us that because of their limited resources,
they must pick and choose capability needs and persuade one or more of
the military Services to sponsor a proposal that addresses the COCOM's
need. The Department has been working to give more time and priority to
the commanders' IPLs, but more attention is needed.
10. Senator Akaka. Mr. Sullivan, in your prepared testimony, you
mentioned the establishment of the DOD capability portfolio management
process. DOD identified nine joint capability area portfolios to
mitigate the tendency to develop Service-centric, stovepiped solutions.
However, the capability portfolio managers lack the necessary authority
and control required to be successful. How can DOD reform their
capability portfolio management process to mitigate the tendency to
develop Service-centric, stovepiped solutions?
Mr. Sullivan. DOD's capability portfolios are intended to advise
the Deputy Secretary of Defense on how to optimize investments within
their respective capability areas. They have no independent
decisionmaking authority and must operate within the Department's
existing framework for acquiring weapon systems. That is, capability
needs are still determined through JCIDS and resources are allocated
through PPBE. Although it is too soon to assess the impact of
capability portfolio management, according to some DOD officials, the
portfolios provided key input and recommendations in the 2010 budget
process. However, without portfolios in which managers have authority
and control over resources, DOD is at risk of continuing to acquire
weapon systems in a stovepiped manner. Ultimately, DOD needs to develop
an integrated portfolio approach that brings the determination of
requirements together with the allocation of resources. To be
effective, such an approach must have committed leadership, empowered
portfolio managers, and accountability at all levels of the Department.
11. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph,
changes made to a system after acceptance of review cost many times
more than the same changes made during the requirements gathering
phase. How can we improve the requirements gathering phases of the
acquisition of major weapons systems to front-load these changes to
achieve significant cost savings?
Dr. Gansler. Please refer to my answer to question #1. If cost is a
requirement, then changes that come in must not only have showed how
they will improve performance but how they will live within the cost
requirement.
Clearly, the earlier this is done, the better; but technology
changes, threat changes, et cetera, do come along, and as long as they
are explicitly within the overall cost requirement they can be allowed
in.
Dr. Kaminski. Requirements adjustments during execution of
acquisition programs are not necessarily bad if they are conducted
within the framework of value propositions. Good program management
requires continuing adjustment of the requirements flow-down and
allocation to major subsystems based upon expected performance,
schedule, and cost. Even top level requirements require continuing
assessment and adjustment based upon similar value propositions. As we
discover more about costs and performance during execution, good
program managers should be interacting with requirements developers to
adjust as needed to provide the best value for the user. This requires
some flexibility in cost allocation, and it also places demands upon
requirements developers to become involved in tradeoff decisions. To be
effective, requirements developers as well as acquisition managers must
have experience with systems engineering tools and techniques, and both
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
Mr. Adolph. A disciplined analysis of alternatives process should
be employed to support the JCIDS and acquisition processes, from
capability needs identification to include system design and
development as well as life-cycle improvement. Emphasis must be placed
on improving the processes for relating mission effectiveness and cost
to system design, system performance, and suitability. Effective
feedback processes are of special importance for spiral developments to
identify enhancements which will improve performance and suitability.
Improving the quality and speed of this feedback is increasingly
important in responding to rapid changes in threat environments.
12. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph,
significant cost overruns are incurred from the instability of mission
and design requirements during testing and even, in some cases,
production. To what extent do you think it would be advantageous to
more rigorously enforce a freeze on fundamental requirements at the
system requirements review gate?
Dr. Gansler. The greatest causes of cost growth, after program
initiation, are budget changes (often caused by Congress) and
``requirements'' changes--both of which can be controlled by
discipline. (See question #11.)
Dr. Kaminski. It is always beneficial to have a firm understanding
of very top level requirements at the start of a program. But
adjustments of subordinate requirements during execution of acquisition
programs are not necessarily bad if they are conducted within the
framework of value propositions. Even top level requirements need
continuing assessment based upon evolving value propositions. As we
discover more about costs and performance during execution, good
program managers should be interacting with requirements developers to
adjust as needed to provide best value for the user. This requires some
flexibility in cost allocation, and it also places demands upon
requirements developers to become involved in tradeoff decisions. To be
effective, requirements developers as well as acquisition managers must
have experience with systems engineering tools and techniques, and both
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
Mr. Adolph. Rigorous enforcement of key requirements thresholds
should be the norm when entering system development and demonstration.
There needs to be more rigor and discipline in the entire requirements
definition and acquisition process. Issues that need to be addressed in
relation to requirements setting include technology readiness (see
response to question 13 below), the translation of requirements into
design criteria, with attention to testability at the subsystem and
system levels, as well as defining thresholds for key performance
parameters.
13. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, cost
overruns are also caused by the inherent unpredictability of the
development of new technologies in the course of fulfilling system
contracts. Would you recommend restricting the proposed designs for
weapons systems to proven technologies while adequately funding
contractual vehicles for pure research?
Dr. Gansler. I would restrict the first ``block'' of the system to
proven technology; then--using ``spiral development''--evolve the
system's future blocks as future technology is proven out; and as long
as the future technology either improves performance (at the same cost)
or lowers the cost. This future technology should be funded in parallel
with the system, until it is proven.
Dr. Kaminski. This will be helped by better development planning
and alignment of incentives. With good development planning, we can
assign managers to develop prototypes, critical systems or components
needed to better understand cost/performance trades and reduce risk. It
is reasonable to expect that many of these developments can be
completed in 2-4 years, so we do not need to restrict ourselves to
proven technology at the beginning of development planning. As these
critical subsystems are delivered and tested, the risk reduction and
domain experience gained in both government and industry will allow us
to prove the critical technologies involved and reduce the time
required to develop, integrate, and test the full system. We can rely
on the experience gained during development planning to apply informed
judgment to adjust requirements to improve value, reduce time, and
better estimate and manage costs.
Mr. Adolph. Weapon system design should be based on proven
technologies. History has shown repeatedly that doing otherwise
introduces unacceptable risk in to weapons system development which
necessarily translates into cost growth and schedule delays. Assessing
whether a technology is sufficiently mature is currently accomplished
through the Technology Readiness Levels (TRL) construct which should
employ government test and evaluation (T&E) as the key component in
moving a technology from TRL 4 to TRL 6. Recent consideration has been
given to strengthening early involvement of the test community by
reinvigorating the government developmental test process and by
involvement of OT&E in readiness assessments. These are sound
initiatives and should be focused on providing rigor to the TRL
assessments in terms of the test methodology as well as the test
environment. Implementation of this more disciplined approach will
require a reversal to the trend of cutting back on government test
personnel as well as strengthening their role in supporting the
acquisition process. In that regard, the Director of Developmental Test
and Evaluation should make an assessment of technological maturity and
integration risk, based on test results and report the findings to the
Undersecretary of Defense for Acquisition, Technology and Logistics.
There may be rare exceptions when a requirement is so pressing that
risks must be taken to design a system which incorporates an unproven
technology. In those instances, a high-priority, adequately funded, and
closely monitored effort must be undertaken to mature the technology.
This must include an early, rigorous assessment of the difficulties
associated with the development effort as well as testing to fully
assess the requisite maturity.
Additionally, the TRL process has been focused on the technology of
system components. Problems with manufacturing these components in
production quantities have also led to significant cost growth for
weapons systems. Previous legislation established the Manufacturing
Technology Program to identify and develop initiatives to improve
manufacturing quality, productivity and technology. Consideration
should be given to expanding the TRL process to evaluate the maturity
of the production methods by which parts are manufactured.
______
Questions Submitted by Senator Bill Nelson
MODELING AND SIMULATION
14. Senator Bill Nelson. Dr. Gansler, Dr. Kaminski, and Mr. Adolph,
computer modeling and simulation (M&S) tools can be very useful in
discovering problems in major systems before we spend a lot of money or
time on hardware development. DOD has some programs in M&S--which in my
opinion are underfunded and disjointed. What is your assessment of M&S
activities within DOD and their ability to address some of the
acquisition problems we are discussing here today?
Dr. Gansler. M&S can be a great aid in system design and
performance evaluation, and should be an important element in the
development of all systems.
Dr. Kaminski. M&S can have a very significant positive impact on
acquisition performance if there is sufficient emphasis on continuing
validation of M&S tools and integration with T&E. Giving early and
serious attention T&E will require strengthening our T&E organizations
and personnel. T&E is often an afterthought, and contracts are often
written without any mention of how we will test the product. We spend
large amounts of money when a large development team waits for test
results. The alternative is to spend less money and time by early
development of M&S tools, continuously validating the models with
testing at critical points, and considering the investment in M&S and
test resources as a critical part of our systems engineering and
development planning efforts.
Mr. Adolph. With some exceptions, the capabilities and limitations
of simulations, simulators and physics-based models of individual
subsystems are well understood. These tools are generally used where
appropriate. However the use of modeling to support requirements
definition, analysis of alternatives, and test design needs to be
pursued more vigorously. The expectations regarding the use of M&S to
support elements of the acquisition process, including system
integration, T&E, and manufacturing have increased along with advances
in computer technology. Distributed T&E (and training) using multiple
ranges, laboratories, and facilities have benefited from the use of
live, virtual, and constructive simulations. However, the availability
of adequate models and modeling tools involving human dynamics for use
in T&E, mission planning, systems engineering, and risk assessment has
not kept pace.
M&S to support acquisition/test programs has been the norm for
decades, and received additional impetus in the 1990s with advances in
information processing technology. In many cases, the incremental
improvements as related to acquisition were oversold in the late 1980s
and 1990s. Additionally, the requirements for verification, validation,
and accreditation have proven to be more costly and timeconsuming than
anticipated. Many of the M&S activities initiated in the last 15 years
were disjointed, with few successful attempts to coordinate
requirements, build and share common models, correlate performance, and
ensure that the models were adequate for the intended applications.
Numerous recent M&S developments have been terminated after spending
millions because of the failure to produce a useful product.
Building models to meet every need is not feasible. An overall M&S
vision is required to identify where efforts should be focused, and to
ensure that new efforts are within that vision. If high priority M&S
efforts within that vision could be identified, it would be possible to
establish requirements, understand what is currently available, and
then determine shortfalls in M&S capability. One approach is
exemplified by the Testing in a Joint Environment Roadmap, which was
recently developed by DOT&E and approved by the Deputy Secretary of
Defense. While focused on T&E usage, this capability would meet a
significant percentage of contractor development and testing M&S
requirements. The T&E roadmap identifies a requirement for a
distributed live, virtual, constructive T&E capability; largely
supported by existing models of friendly and threat systems. Much of
this capability could come from existing models of acquisition systems
developed by program contractors, in conjunction with threat models
available from the intelligence agencies. However, there has been
virtually no effort to identify requirements for high priority joint
missions, to determine what friendly and threat force representations
are required, assess the availability of existing models that meet
those requirements, and then use that information to define M&S
shortfalls. Additionally, there are insufficient mechanisms and
processes to feed back data from operational tests and field exercises
to further validate and refine models. Archiving mechanisms that can
store and locate data for future applications are needed.
There are also supporting M&S requirements (engineering and
physics-based models) which are required in the validation of higher-
level engagement models. Higher level models, such as envisioned in the
T&E roadmap, could be used to identify areas with the greatest risk and
uncertainty to better define areas where engineering and physics-based
models are required. The initial effort should be to review the past
studies on M&S in the DOD and determine those requirements that could
contribute the most to improving M&S capability. While most studies
have identical or similar recommendations, the majority of those
recommendations have not been implemented.
15. Senator Akaka. Dr. Gansler, Dr. Kaminski, and Mr. Adolph, how
would you recommend addressing shortfalls in DOD activities in M&S?
Dr. Gansler. To address shortfalls, I would ask an independent
group (such as the Defense Science Board) for an assessment and a set
of recommendations (including appropriate funding).
Dr. Kaminski. We need upfront investment in M&S tools to understand
the value propositions and the tradeoffs among cost, schedule, and
performance of the mission. We spend large amounts of money when a
large development team waits for test results. The alternative is to
spend less money and time by early development of M&S tools,
continuously validating the models with testing at critical points, and
considering the investment in M&S and test resources as a critical part
of our systems engineering and development planning efforts.
Mr. Adolph. Focused M&S business plans must be developed. Separate
plans are needed for requirements definition, acquisition/test,
training and force structure evaluations. As discussed above, the
latter plans should be used to identify critical joint missions and
provide a basis for prioritizing M&S requirements; to understand
current capabilities and identify major shortfalls.
An issue that needs to be addressed is the availability of existing
models to meet current requirements. Many models were developed by
system contractors to meet specific needs. In some cases they are
proprietary and there are rarely provisions for models to be maintained
current so they represent a system in the field with normal
improvements and modifications, or changes to the treats. Often
contractors are not funded to make their models generally available.
The government must address issues relating to long-term configuration
control, upgrades, ownership, and funding throughout the system
lifecycle, to include training. In addition, addressing the
requirements for a single joint mission doesn't capture requirements
across several potential joint missions and scenarios.
However, it would provide a starting point for future requirements
and identify the most sensitive issues in implementation of the overall
concept. Addressing one major joint mission scenario would identify a
large percentage of requirements for other joint missions.
Finally, an important consideration is the physical location of
major simulation laboratories and related facilities. There are often
valid reasons to locate a simulation capability at a contractor
development facility to support initial development; however it places
limitations on the future utility and accessibility of the simulation,
as well as the need to replicate parts of the simulation capability at
a government test facility or logistics center. Prior to making a
decision regarding the location of simulation facilities for each major
program, an assessment should be made as to the most cost-effective
locations(s), taking into account follow-on and future programs. This
decisionmaking process should include the appropriate Service MRTFB
representatives, as well as the TRMC. (See also my response to question
#17.)
TEST AND EVALUATION ENTERPRISE
16. Senator Akaka. Mr. Adolph, a major step in developing weapons
systems is ensuring that they are appropriately tested during their
development and before their operational use. I believe that DOD is
letting its testing enterprise decline precipitously in terms of
testing equipment, infrastructure, and workforce. What role do you feel
that the DOD testing enterprise plays in ensuring acquisition programs
deliver their systems on schedule and on budget?
Mr. Adolph. The role of the DOD testing enterprise is addressed in
detail in the recent Defense Science Board Task Force Report on
Developmental T&E. My written testimony covers the major findings and
recommendations from that study, and while it emphasizes the impacts of
reliability, availability, and maintainability on life-cycle costs, the
implementation of some acquisition reform initiatives diminished the
role of government T&E in the acquisition process.
A brief summary response follows. The traditional role of the
government during the test planning phase included the identification
of test resource requirements and test facilities, the development of a
test strategy and detailed T&E plans. When programs moved from the
planning phase to the execution phase, the government traditionally
performed operational assessments, participated in test conduct and
analysis, and performed an evaluation of the test results for the
government Program Office. With a few exceptions, this is no longer the
case. In many instances the government testing enterprise has gone from
vigorous test participation to oversight to out-of-sight.
As a minimum, government test organizations should reconstitute and
retain a cadre of experienced T&E personnel to perform the following
functions:
Participate in the translation of operational
requirements into contract specifications and key performance
parameters
Participate in RFP preparation and source selection,
including assurance that contractor developed models and
simulations are available to the government
Participate in technology maturity determination and
prototype evaluations
Perform operational assessments in conjunction with
the systems engineering process
Participate in developmental T&E planning and
technical reviews
Participate in test conduct, analysis, reporting
capabilities, and limitations, with emphasis on government
analysis and reporting
17. Senator Akaka. Mr. Adolph, what is your assessment of the state
of the DOD's testing facilities and their ability to perform their
designated missions?
Mr. Adolph. The focus of DOD's testing facilities is the Major
Range and Test Facility Base (MRTFB) which is comprised of those ranges
and facilities designated most critical to supporting the T&E needs of
DOD development and acquisition programs. Since the 1970s, DOD's intent
has been to ensure the capabilities and capacities of these MRTFB
facilities/ranges meet the T&E requirements of emerging technologies
and weapon system programs, taking into consideration the long-lead
times necessary to acquire these capabilities, which often dictates
that new test technologies be developed and matured. Likewise, a
skilled T&E workforce must be in place to address the challenges of
testing and evaluating new weapon systems with increasing complexities
of technologies, integration, and missions. Given that this is the
intention of the MRTFB, where do we stand today?
As I noted in my written statement, several changes resulted from
the implementation of acquisition reform initiatives in the late 1990s.
Among them was a de-emphasis on the use of government test facilities,
testers, and evaluators. When programs need to use government test
facilities, the first inclination has sometimes been to use non-MRTFB
capabilities to avoid reimbursement costs required by the MRTFB charge
policies, often using training ranges having little or no T&E
infrastructure. If the MRTFB facilities/ranges are used, programs
typically prefer to request only the raw data to perform their own
evaluations. Consequently, the Services have been reducing their
budgets and T&E workforce for sustaining the MRTFB. Not only has the
MRTFB experienced loss of a large number of the experienced subject
matter engineering experts, scientific, and mathematical personnel,
several MRTFB test facilities are being mothballed or closed. The
government skills to restore or replace them will be lost over time.
Investments in new test capabilities in the MRTFB for anticipated
weapon system technologies and test programs have been increasingly
awaiting the first program to need them for funding rather than using
T&E institutional funds. This practice is counter to the intent of
having an MRTFB; i.e., anticipating and having T&E capabilities in
place when new development or acquisition programs require them. Some
new T&E capabilities require long-lead times to plan, budget, and to
develop to be ready when needed by a program, thus can cause the test
program schedules to slip, inadequate capabilities to be accepted as
work-arounds, or not to be tested at all. Furthermore, these facilities
are normally specialized for the programs paying for them, and may have
little or no residual use for other programs, particularly when they
are co-located with the prime contractor's manufacturing facility.
While the Test Resources Management Center (TRMC) ``Strategic Plan for
DOD Test and Evaluation Resources'' serves as a good guide to the
actions required to sustain a capable MRTFB, the TRMC has virtually no
power to require the military departments to sustain adequate operating
or investment funding beyond the funds managed through the Central Test
and Evaluation Investment Program and T&E/S&T Program. As a result, my
assessment is that in some instances, the MRTFB no longer meet its
intended purpose of having adequate capabilities and capacity available
for future acquisition programs.
______
Questions Submitted by Senator E. Benjamin Nelson
FISCAL RESPONSIBILITY
18. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, cost overruns and fiscal irresponsibility are great
concerns for me. During his hearing in January, the Secretary of
Defense stated that many programs that cost more than anticipated are
built on an inadequate initial foundation, which in my opinion begins
with defining requirements. It has been reported that many of the
COCOMs do not believe their requirements are sufficiently represented
by the current process, which seems to be centered on each Service, not
the Armed Forces as a whole. This results in duplication of
capabilities, non-alignment of requirements to capability, and
interoperability problems, all of which lead to expensive requirements
creep. Should COCOMs play a lead role in defining needed capabilities?
Mr. Sullivan. Yes, we believe that the COCOMs should play a
stronger role in defining capability needs. Requirements determination
in DOD, particularly for major weapon system programs, continues to be
driven largely by the military Services. Our work has revealed that
although the COCOMs--which are responsible for planning and executing
military missions--are the principal joint warfighting customer in DOD,
they have played a limited role in determining requirements. In
analyzing requirements documentation related to new capability
proposals, we recently reported that most--almost 70 percent--were
sponsored by the military Services with little involvement from the
COCOMs. Other studies have also raised concerns that the Services and
the COCOMS do not routinely collaborate to identify possible joint
capability solutions.
The Defense Acquisition Performance Assessment Panel
concluded that JCIDS resulted in capabilities that did not meet
warfighter needs in a timely manner and recommended that JCIDS
be replaced with a COCOM-led requirements process in which the
Services and defense agencies compete to provide solutions.
The Defense Science Board similarly reported that
JCIDS has not provided for increased warfighter influence, but
instead actually suppresses joint needs in favor of military
Service interests, and recommended an increase in the formal
participation role of the COCOMs in the JCIDS process.
The Center for Strategic and International Studies has
also recommended that the Joint Forces Command take the lead in
conducting capabilities development planning for the COCOMs and
become a formal member of the JROC.
By continuing to rely on capability needs defined primarily by the
Services, DOD may be losing opportunities for improving joint
warfighting capabilities and reducing the duplication of capabilities.
Dr. Gansler. The Packard Commission and (subsequently) Goldwater-
Nichols stated that all requirements were ``joint'' (since that is the
way we fight). The JROC was set up to represent the COCOMs in the
``requirements process.'' Interoperability should always be a critical
design and test requirement. The COCOMs should be the source of
requirements for both ``urgent needs'' and for JCTDs.
Dr. Kaminski. I believe that the COCOMs should play a strong role.
But to do so, they need to develop the expertise needed to engage in
cost, schedule, and performance tradeoffs. Even top level requirements
need continuing assessment based upon evolving value propositions. As
we discover more about costs and performance during execution, good
program managers should be interacting with requirements developers to
adjust as need to provide best value for the user. This requires some
flexibility in cost allocation, and it also places demands upon
requirements developers to become involved in tradeoff decisions. To be
effective, requirements developers as well as acquisition managers must
have experience with systems engineering tools and techniques, and both
must have sufficient domain experience to engage productively in cost-
performance tradeoffs.
Mr. Adolph. I don't have the background or experience to respond to
this question.
19. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, how successful are DOD's processes for matching
warfighter needs with resources?
Mr. Sullivan. GAO has previously reported that DOD's inability to
achieve a balanced portfolio of weapon system programs that matches
joint warfighting needs with available resources is due in part to the
fragmented structure of the Department's requirements, funding, and
acquisition processes. The JCIDS process has not been effective in
prioritizing needs from a joint, Department-wide perspective, and it
largely approves capability needs without accounting for the resources
or technologies that will be needed to acquire the desired
capabilities. Resource allocation decisions in DOD take place through
the PPBE process, which is separate from JCIDS. PPBE largely allocates
resources on a Service-by-Service basis and does not effectively link
resources to capabilities. In addition, the process allows too many
programs to start development with unreliable cost estimates and
without a commitment to fully fund them. Until DOD establishes a more
integrated approach to weapon system acquisition, it will continue to
struggle to effectively prioritize warfighting needs, make informed
trade-offs, and achieve a balanced mix of weapon systems that are
affordable, feasible, and provide the best value to the warfighter.
While DOD recently reported that the JROC is doing more to seek out
and consider input from the COCOMs through regular trips and meetings
to discuss capability needs and resourcing issues, we found that many
of the COCOMs still do not believe their needs--reflected through the
IPL process--are sufficiently addressed through DOD's requirements
process. In order to grant the COCOMs more influence in identifying
requirements, DOD should consider providing the COCOMs with additional
resources to establish robust analytical capabilities for identifying
and assessing their capability needs. In addition, DOD and the Joint
Staff should ensure the JROC gives equal consideration to COCOM needs.
Dr. Gansler. As Secretary Gates has stressed, we need a better
``balance'' between likely warfighter needs and resource allocations.
Today, the budgets are unbalanced in the direction of potential, future
peer competitors.
Dr. Kaminski. The success is mixed. Warfighters are often very able
to address their immediate needs, but addressing future needs can be
challenging. Warfighters also need to be open to new conops--new
approaches to performing their mission that can be derived from new,
enabling technologies. Similarly, developers and cost estimators are
more able to address near-term estimates involving technologies that
are mature and familiar, and less able to do so with future
technologies that are less mature and less familiar. Matching needs and
resources must be a continuing, iterative process. This process must
have a solid foundation in systems engineering, and the systems
engineering must in turn have some foundation in M&S which is validated
by building and testing. Effective value proposition assessments will
occur only with skilled, trained, and experienced personnel from
requirements and acquisition communities working together within a
systems engineering framework to refine needs and solutions.
Mr. Adolph. I don't have the background or experience to respond to
this question.
TECHNOLOGICAL CHANGE AND WEAPONS SYSTEMS
20. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, one of the challenges we face with new technology and
the rapid advancement of technology is the desire to outfit our
military men and women with the latest and most advanced equipment.
While we all want to provide our military with the latest advances in
technology, I worry that as technology changes we are in an ever state
of trying to upgrade a system while it is in development. How can we
best handle constant technology changes for weapons systems that take
years to develop and deliver?
Mr. Sullivan. First, technology development must be placed on the
critical path of any new acquisition program. Technology development
requires trial and error, discovery, and invention, which should all be
done in a S&T environment before technologies are brought onto
programs. Second, GAO has found that successful commercial companies
use an evolutionary or incremental approach that reduces development
risks and enables quicker delivery of products to the customer. These
commercial companies have implemented an evolutionary approach by
establishing time-phased plans to develop a new product in increments
(5 years or less) based on technologies and resources achievable now
and later. They set requirements for products that require only
technologies that they know will work, and wait to put more risky
technologies on future increments once their risks have been
diminished. An open architecture approach to development is also
important so that product components, such as technologies, can be
upgraded or added in subsequent product development increments, without
having to do a major redesign of the product.
Historically, DOD's approach to developing new weapon systems often
attempts to satisfy the full capability in a single step, ``big-bang''
approach regardless of the design complexity or the maturity of the
technologies. Under this approach, the warfighter can wait over a
decade to receive any new capabilities and by the time the capabilities
are delivered, they may be out of date. Implementing knowledge-based
evolutionary acquisition practices--where individual program increments
are defined on the basis of mature technologies and a feasible design
that incorporates an open architecture approach--would allow DOD to
more effectively manage technology changes in acquisition programs.
Shorter, more manageable development cycles would also provide the
warfighter with useful technologies quicker and provide more confidence
that technologies can be developed within program cost estimates.
Dr. Gansler. See my answer to question #13.
Dr. Kaminski. I have always found that the mark of a good program
manager is a large lower right hand desk drawer. In this drawer are
placed all the ideas for incorporation of new technology advancements
until the first version of the system is fielded. At that time the
drawer is opened, and representatives from the requirements and
acquisition communities meet and apply systems engineering tools and
techniques to assess shortfalls in the fielded system, evolving needs,
and evolving technologies to examine value propositions and decide what
to incorporate in the next block of that fielded system.
Mr. Adolph. With platforms and major systems being procured less
frequently and taking many years in development, there is often a
tendency to over-reach since another opportunity to provide advanced
technology to the warfighter may not occur for many years. The first,
and most, beneficial action would be to work toward shortening the
acquisition cycle. For various reasons, the acquisition cycle has
become overly extended, due in part, as discussed in my response to
question #13, to initiating acquisition programs based on immature
technologies. This trend must be reversed. Recommendations have been
made repeatedly in numerous studies that would result in a shortened
cycle.
One approach to a shortened cycle for large programs would be to
incorporate new technologies in ``block upgrades.'' This approach has
been successful for many defense systems, sometimes as a result of
necessity and sometimes as preplanned improvements. The latter approach
has been implemented on many aircraft programs where planned multi-
staged improvement programs are executed at regular intervals.
Strategies such as spiral acquisition and pre-planned product
improvement have been pursued with various degrees of success. If too
many new technologies are incorporated into a system, the risk is
compounded by the complex task of integrating advanced technologies
with sometimes insufficient technical maturity. These risks may be
mitigated through disciplined processes to properly evaluate technology
maturity, larger investments in technical domain expertise as well as
in systems engineering. More emphasis should also be placed on analysis
of alternatives, to include an OSD-level review. Designs that support
``plug and play,'' e.g. standard interfaces are also a partial
solution. Producing the technology and integrating it into the
logistics system as well as training maintenance personnel, et cetera,
can still be challenges if the technology has never been deployed to
the warfighter. (See also my response to question #13.)
ACQUISITION PERSONNEL
21. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, a consistent theme throughout the hearing was the
insufficiency of the current acquisition workforce. What would your
recommendation be for increasing the number of Active Duty billets
specifically for acquisition?
Mr. Sullivan. Although we did not examine DOD's decisionmaking
process for using military versus civilian personnel, we recently
reported that DOD lacks critical Department-wide information to
determine the sufficiency of its acquisition workforce to meet its
mission needs. We recommended that DOD identify the number and skill
sets of its total acquisition workforce--including civilian, military,
and contractor personnel--and conduct analyses using this information
to inform acquisition workforce decisions regarding the appropriate mix
of civilian, military, and contractor personnel. We are encouraged by
the Department's recent announcement that it plans to significantly
increase the size of DOD's acquisition workforce by converting 11,000
contractors and hiring an additional 9,000 government acquisition
professionals by 2015--beginning with 4,100 in fiscal year 2010.
Dr. Gansler. In the Commission I chaired (the ``Commission on Army
Acquisition and Program Management in Expeditionary Operations'') we
stressed that it is not just numbers that matter, but rank as well as
experience. The senior military acquisition positions should be staffed
with General Officers that have extensive acquisition experience. As to
numbers, we should at least get back the 25 percent that Congress
mandated be cut in fiscal year 1996 (which, across the DOD acquisition
workforce, military and civilian, is approximately 50,000 additions--of
which perhaps 20 percent are military).
Dr. Kaminski. Numbers alone aren't the answer. We need an
appropriate mix of quality and quantity. We need personnel able to
provide early and continuing systems engineering and closely-coupled
development planning. We need: a) sufficient personnel (in both
government and industry) with adequate education, training and domain
experience (this includes personnel in requirements development as well
as in acquisition); and b) sufficient front end investment in the tools
necessary to understand the key tradeoffs in cost/schedule/performance,
and to identify and address the key risks in a systematic manner.
Mr. Adolph. As a minimum, the government acquisition workforce,
which includes military and civilian personnel, should be increased to
the levels that existed in the mid-1990s prior to the congressional and
Service reductions. The military/civilian acquisition position mix
should be left to the individual Services as each is procuring
different commodities, and has a different philosophy regarding a
career military acquisition workforce. As I noted in my opening
statement, during a time of increased programmatic and technical
complexity, there has been a loss of a large number of the most
experienced management and technical personnel, without an adequate
replacement pipeline. Solutions to the acquisition problems must begin
with reconstituting a trained and experienced government workforce,
which includes programs managers, domain subject matter experts as well
as systems engineers, contracts personnel, testers and evaluators.
22. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, to ensure we get the quality and expertise we require,
how do you recommend we recruit and retain this force?
Mr. Sullivan. DOD has just announced plans to substantially
increase its in-house acquisition workforce over the next few years.
Our recent report on DOD's acquisition workforce included practices of
leading companies that could provide insights for DOD as it moves
forward to hire and retain these employees. We found, for example, that
the companies employed a variety of recruitment and retention
initiatives. More importantly, some companies assessed their efforts at
filling workforce gaps by tracking data on specific recruiting and
retention metrics. One company assessed the quality of their hiring
sources by assessing the performance of new hires over their first 2
years.
Dr. Gansler. Promotion potential and acquisition-experienced
leaders are the keys to recruiting, developing, and retaining top
acquisition personnel.
Dr. Kaminski. You are quite right in noting that we must address
retention as well as recruitment to build the requisite work force. We
want people who want to make a difference. When we recruit these
people, we must expect that we won't keep them if they can't see that
they in fact are making a difference. Excessive (and growing) time from
program initiation to fielding is a big problem for young people who
want to see the impact of their work. As this time increases from a few
years to 15 years or more, it undermines the entire acquisition process
by causing key participants to ``lose the recipe'', and lose a sense of
accountability as well as a sense of being able to make a difference.
When new capabilities are developed and fielded in 5 years, engineers,
managers, testers, cost analysts, et cetera, are able to benefit from
and apply the experience gained from a previous program or program
phase. They can also see the results of their decisions and be held
accountable. We can also meaningfully employ past performance of the
contractor as a factor in the award of future programs--an important
factor in incentivizing contractor performance. This all changes
dramatically when the time extends to 15 years, and we have five roll-
overs of management, engineers, cost analysts, and commercial
technology during this time period. This long and growing time period
is a result of the inflexibility inherent in our entire system of
requirements development, budgeting, and acquisition, and it creates a
vicious cycle in which it further exacerbates the contributors above,
and they in turn further increase the time and cost growth. We see the
result when we must discard our current acquisition system in order to
deal with urgent needs and field systems such as MRAP and jammers to
counter IEDs by forming and using rapid reaction organizations. This
cycle must be broken by attacking the root causes.
Mr. Adolph. When the acquisition workforce reductions were made in
the late 1990s, they were implemented by encouraging early retirements
as well as hiring freezes, which often lasted for years. As a
consequence of the latter, the pipeline of new civilian employees was
shut off, and today there are large gaps in experience as well as
inadequate numbers of personnel. Some Service test organizations are
aggressively addressing these issues and have more insight into what
works in today's environment than I do. Based on my past experience,
initial hiring, particularly for positions at remote test locations, is
the biggest challenge. It requires aggressive recruiting at
universities, along with co-op programs. The challenge of the work
environment and the learning experiences, dealing with the latest
technologies, combined with well-funded full-time advanced degree
educational opportunities (with associated follow-on career
commitments), are powerful positive retention forces after people are
brought on board. On-the-job learning experiences have been diminished
in some test organizations, where government personnel have been
relegated to less challenging supporting functions.
23. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, what can we do in the meantime to handle the personnel
shortage since it will take many years to develop the level of
expertise required for proper acquisition reform?
Mr. Sullivan. As we recently reported in our review of DOD's
acquisition workforce management, DOD is currently implementing
initiatives aimed at improving the capacity of its acquisition
workforce using the newly established Acquisition Workforce Development
Fund. These efforts are focused in three key areas: (1) recruiting and
retention; (2) training and development; and (3) retention and
recognition. DOD has also recently announced plans for significantly
increasing its in-house acquisition workforce over the next few years.
While DOD integrates these additional employees into its acquisition
workforce, the Department can also leverage the experience and
knowledge of its current workforce. DOD can take steps to retain
experienced personnel to aide in the development of newer staff and to
ensure it maintains and transfers key institutional knowledge.
Dr. Gansler. To fill the gap, while developing a new group, I would
bring into the government some middle and senior people from industry;
and I would use contractors (with experience, but without conflicts) to
fill the rest of the slots.
Dr. Kaminski. We can begin to hire people with education in
engineering and systems engineering. We can send our current personnel
back to universities to enhance their education. We can send our
current personnel to industry to gain domain experience. We can bring
back retirees on a part time basis to provide mentoring to our new
hires and help them gain domain experience. We need to do all of these.
Mr. Adolph. During the period when the normal pipeline is being
replenished, Centers of Excellence must be created in selected test and
acquisition engineering support organizations, with a focus on
technical domain subject matter expertise. These cadres of expertise
will provide the basic oversight functions such as those I outlined in
my answer to question #16. The first step is to provide oversight in
key technical disciplines at key programmatic times, e.g., RFP
preparation, source selection, TEMP preparation, test reporting to
support program reviews; transitioning to more active, continuous
involvement as the workforce increases. The Services may be reluctant
to take this approach. These Service cadres can also be augmented by
subject matter experts from FFRDCs. The use of red teaming by outside
experts (e.g., cross-Service, FFRDCs, and SETAs) should also be
employed to provide the requisite expertise, as well as a measure of
independence which is too often lacking in government program office-
centric reviews.
MANDATORY PROCEDURES FOR MAJOR DEFENSE ACQUISITION PROGRAMS
24. Senator Ben Nelson. Mr. Sullivan, Dr. Gansler, Dr. Kaminski,
and Mr. Adolph, what oversight and reporting mechanisms do you
recommend we establish to ensure the Mandatory Procedures for Major
Defense Acquisition Programs (MDAPs) meet warfighter requirements while
eliminating redundancy and wasteful spending?
Mr. Sullivan. DOD has recently revised its acquisition policy in
ways intended to provide key Department leaders with the knowledge
needed to make informed decisions before a program starts and to
maintain disciplined development once it begins. The revised policy
recommends the completion of key systems engineering activities before
the start of development, requires early prototyping, establishes
review boards to identify and mitigate technical risks, and implements
early milestone reviews. If implemented, these policy changes could
help programs replace risk with knowledge, thereby increasing the
chances of developing weapon systems within cost and schedule targets
while meeting user needs. To achieve improved outcomes in acquisition
programs, DOD must ensure its policy changes are consistently
implemented and reflected in decisions on individual programs. However,
DOD could do more by requiring new programs to have manageable
development cycles and establish knowledge-based cost and schedule
estimates. Limiting the length of development cycles would make it
easier to more accurately estimate costs, predict future funding needs,
effectively allocate resources, and hold decision makers accountable.
The acquisition reform legislation recently proposed by the Senate
Armed Services Committee should also help achieve further improvements.
Provisions increasing the emphasis on systems engineering, requiring
early preliminary design reviews, and strengthening independent cost
estimates and technology readiness assessments should make the critical
front end of the acquisition process more disciplined. Establishing a
termination criterion for critical cost breaches could help prevent the
acceptance of unrealistic cost estimates at program initiation. In
addition, having greater COCOM involvement in determining requirements
and greater consultation between the requirements, budget, and
acquisition processes could help improve DOD's efforts to balance its
portfolio of weapon system programs.
While policy revisions and legislation may lead to improvements,
they will not be effective without changes to DOD's overall acquisition
environment and the incentives that drive the behavior of its
decisionmakers, the military Services, program managers, and the
defense industry. Changing the environment will require strong
leadership and accountability within the Department.
Dr. Gansler. I believe we currently have enough oversight and
reporting mechanisms for MDAPs (and more would only slow the system
down even more). The critical need is to use these mechanisms more
effectively--with experienced personnel (who know what to look for, and
what questions to ask).
Dr. Kaminski. We first need to realize that ``one size does not fit
all.'' For example, programs dealing with countering IEDs need cycle
times measured in weeks, while a next generation bomber program must
plan for fielding and support cycles measured in years.
In most cases today the program manager's authority is diffused by
many levels of oversight in both the Department and in Congress, and
the financial and performance constraints imposed do not allow
sufficient freedom of action to apply informed judgment in a timely
manner. Flexibility is further limited by application of a ``one-size-
fits-all'' approach imposed by the DOD 5000 system, and the oversight
practiced by the DOD and Congress. A program manager needs the freedom
to tailor the acquisition approach to the problem, to ensure that the
program response time will fall within the response time of the threat,
and to apply a variety of tools and techniques (such as the use of
prototypes, competitive prototypes, M&S, critical subsystem and
component demonstration). For this to work, we need program managers
with the education, training, and domain experience needed to enable
timely responses and excellent judgment relevant to the domain. They
need the flexibility to apply their expertise.
Mr. Adolph. I believe the current oversight and reporting
mechanisms, when combined with the changes in the proposed legislation,
will provide adequate direction and guidance for the acquisition
process. The key is putting more discipline in every step of the
process by the Services and OSD. This was achieved in numerous past
successful programs in the 1960s, 1970s, and 1980s without being overly
bureaucratic. One impediment is the recent trend towards contractual
vehicles which relieve development contractors of reporting
requirements, including regular status reports on costs and technical
performance. The government must have continued access to cost and
performance data to provide effective oversight.
Most of the basic policies and directives for the acquisition
process represent best practices based on decades of experience. The
fundamental problem is a lack of adequate discipline throughout the
process. This begins with proper definition of operational requirements
at the outset of a program and stating those requirements in terms that
are reasonable and quantifiable for design and test purposes. Another
major issue is the lack of adequate technology maturity (See my
response to question #13.)
There have been some problems with military specifications and
standards in the past, in that they were not properly tailored to the
system to be procured. However, these standards have evolved and were
updated over decades and were, for the most part, excellent compendiums
of best practices and lessons learned. If properly used, they provide
guidance for system development to ensure a systematic and disciplined
approach. Proper application of these guidance documents also ensures
that problems, which are a normal part of any complex high technology
development, are identified early. Many of these documents have been
allowed to atrophy over the last several years or have been eliminated.
Previously eliminated specifications and standards should be
selectively updated and reinstated; while retaining the option to use
commercial specifications and standards when available, provided the
latter adequately address the requisite military performance and
suitability requirements in the intended operating environment.
Finally, the most important ingredient remains a capable and
experienced government acquisition workforce with equally capable
leaders, who remain on the job long enough to realize the consequences
of their programmatic decisions.
______
Questions Submitted by Senator David Vitter
PERSONNEL IN ACQUISITION REFORM
25. Senator Vitter. Dr. Gansler, a recurring message from the panel
is the importance of improving the quality and experience of personnel
in acquisition reform, and I'm glad that Chairman Levin and Ranking
Member McCain included section 206 in their acquisition reform bill to
reward excellence. This section requires the establishment of a program
recognizing excellent performance and the award of cash bonuses.
However, it was stated that cash isn't necessarily the greatest
incentive for encouraging excellence, and making a difference is the
best incentive. In your opinion, what can DOD implement in an
excellence program to demonstrate improvements so that personnel can
track and actually see their work reflected in tangible results to
provide and maintain motivation?
Dr. Gansler. Combining appropriate metrics with rewards for success
(including recognition, and even token financial rewards) are
desirable--but the managers must be allowed to have sufficient
flexibility to exercise their judgment, if they are to succeed.
26. Senator Vitter. Dr. Gansler, how much of the problem in
attracting the right quantity of qualified acquisition personnel is a
product of the overall Federal employee hiring rules established by
Office of Personnel Management (OPM) or the specific rules established
by DOD?
Dr. Gansler. The DOD must be able to compete with industry for
``the best and brightest.'' The process must be flexible and fast--yet
fair. Salary will not be the driver (government jobs offer the rewards)
but the process can be neither bureaucratic nor political.
27. Senator Vitter. Dr. Gansler, last year, Congress authorized
$300 million to rebuild the acquisition workforce. Do you believe this
is sufficient?
Dr. Gansler. It is an important start, and will likely need to be
continued for a few years.
28. Senator Vitter. Dr. Gansler, cost overruns were endemic in the
1980s and 1990s when the acquisition workforce was robust. Is it likely
then that adding more people will solve the problem of cost overruns?
Dr. Gansler. As I noted in question #21, numbers alone are not the
answer, senior and experienced people with an improved acquisition
process (as covered in my answers above--especially numbers 1, 8, and
13).
29. Senator Vitter. Dr. Gansler, it seems that there is a problem
with keeping quality program managers in their roles for an extended
period of time. How can we ensure continuity at these positions?
Dr. Gansler. Goldwater-Nichols intended the program managers to
remain at least through the next milestone. In the Packard Commission
we emphasized that they be allowed to be promoted in place (so that the
extended period is a reward, not a career-limiting assignment).
FUNDING FOR DEVELOPMENTAL TESTING
30. Senator Vitter. Mr. Adolph, to what extent does a lack of
funding for developmental testing (DT) early in a program's schedule
result in cost increases and delays later in its schedule?
Mr. Adolph. The lack of adequate funding for DT has a major adverse
impact on developmental cost and schedule. There are numerous issues
relating to DT funding. The weapons timelines for RDT&E continue to
increase, driven by inadequate resourcing of the entire developmental
process as well as the inclusion of immature technologies in systems.
Inadequate funding of DT, which includes the number of test assets,
results in delayed identification and correction of problems, many of
which are a normal part of a high-technology developmental program.
Late identification results in more difficulty and expense involved in
fixes. Most recent programs have had an inadequate number of assets to
execute a robust DT program in a timely manner. When there are an
inadequate number of test assets, the DT program is drastically
impacted, often for months, when a test vehicle must be laid up for a
retrofit. By the time a program reaches the full-scale platform test
phase, there is a huge cadre of test and test support personnel who
cannot be efficiently used whenever a test program comes to a halt. The
fixed cost of maintaining this cadre often exceeds the incremental
variable costs of conducting test missions. Another factor in efficient
testing is the availability of adequate test personnel and test
facilities. The latter includes physical test support facilities and
assets, instrumentation, and data processing (see also my response to
question #17). Numerous developmental programs have been delayed
because of the lack of adequate facilities, capacity, and people to
accomplish the basic data processing tasks quickly, as well as
inadequate number of domain subject matter experts to analyze test
results. This is exacerbated when earlier involvement by DT&E and OT&E
personnel is limited by a lack of sufficient experienced personnel.
Commercial programs routinely resource facilities and personnel to
conduct testing on a multiple shift basis during critical phases of the
developmental process.
COMPETITIVE PROTOTYPING
31. Senator Vitter. Dr. Kaminski, how much can competitive
prototyping help in identifying the best product for warfighters?
Dr. Kaminski. Competitive prototyping is one arrow in our quiver of
tools and techniques in development planning. Not all prototyping needs
to be competitive. Prototyping should not be limited to the full system
level. There are cases where prototyping is best applied only to
address major platform integration risk areas, or to critical
subsystems. Again, I don't believe there is a ``one-size-fits-all''
solution to prototyping. But I do believe that at a big picture level
we are not doing enough prototyping in our development planning.
Reducing risk and developing domain expertise require that we build
things. M&S are of great value, and so are our computer based analysis
and design tools, but all of my experience tells me that we need to
build and test on a continuing basis to make mistakes, learn from those
mistakes, and apply that learning in the development of new
capabilities.
32. Senator Vitter. Dr. Kaminski, the F-16 program, lauded by many
as an example of how to acquire a capability, was competed to the point
where the Air Force actually flew competing planes before deciding on
which defense contractor won the award. Why have we gotten so far away
from that?
Dr. Kaminski. We haven't completely departed from that approach. We
did something very similar in the JSF program. But I agree that we are
not doing enough. We can do quite a bit more. I believe we can do so
without increasing costs by spending money upfront to reduce or avoid
the cost of significant overruns later.
JOINT REQUIREMENTS OVERSIGHT COUNCIL
33. Senator Vitter. Mr. Sullivan, the Vice Chiefs have important
jobs internally managing their respective Services. Do you believe that
JROC should instead be staffed with independent personnel who are not
advocates for a particular Service?
Mr. Sullivan. GAO has found DOD's requirements process to be too
Service-centric. The needs of the COCOMs should be given greater
consideration. There are a number of ways the requirements setting
process might be changed to address this issue. For example, the
combatant commanders might supplant the Service Vice Chiefs role in the
process or might be given an equal say in the process. Several outside
reviews have suggested such solutions:
The Beyond Goldwater-Nichols study team recommended a
more ``joint'' JROC in which the Service Vices are replaced by
COCOM Deputies and civilians responsible for requirements
policy.
The Defense Acquisition Performance Panel (DAPA)
recommended replacing the current requirements setting process
with a combatant commander-led requirements process in which
the Services and defense agencies compete to provide solutions.
Each of the combatant commanders would be tasked to prepare
extended planning annexes to each of their operational and
contingency plans (to be updated on a 2-year cycle) that would
provide a 15-year forecast of both capability gaps and excesses
relative to mission requirements.
Some members of the Defense Science Board Summer Study
on Transformation felt that the requirements process continues
to be dominated by the force providers and the Joint Staff and
that COCOM needs are under-represented. They proposed making
the COCOMs more equal partners with the force providers from
the beginning of the process, particularly when identifying
capabilities needed to carry out the Department's operational
missions.
We believe that these proposals deserve further consideration as a
means to help improve the Department's ability to balance joint
warfighting needs.
34. Senator Vitter. Mr. Sullivan, should we have a new, independent
JROC that could hear all advocates for all programs and then recommend
continuation or elimination of the program based on current and likely
future threats to the United States?
Mr. Sullivan. Because many major defense acquisition programs have
ambitious and lengthy product developments--in some cases, delivery of
a weapon system to the warfighter takes as long as 15 years--there can
be a significant amount of time that elapses between JROC reviews and
changes in the threat environment can occur. Before a major weapon
program is approved to begin system development, the JROC must validate
the capabilities to be developed, including the system's key
performance parameters. The JROC also validates the production elements
of a program before it is approved to start production. For programs
that have lengthy development cycles, more frequent reviews may be
necessary to assess whether programs are worth continuing in light of
current and likely future threats. In prior work we have conducted on
how successful companies manage their portfolios of product development
efforts, we found that the companies revisit their investment decisions
at multiple stages throughout product development to ensure products
are still of high value.
35. Senator Vitter. Mr. Sullivan, should the Services be required
to budget their programs to the cost estimates provided by a new
Director of Independent Cost Assessment?
Mr. Sullivan. Budgeting programs to an independent cost estimate
would be an improvement, however, for cost estimates to be effective
they must be based on a high degree of knowledge about requirements,
technology, design, and manufacturing.
A reliable cost estimate helps ensure a program's projected funding
needs are adequate to execute the program. Less than a quarter of the
48 programs in GAO's annual assessment of weapon system programs that
provided data used the estimate made by DOD's Cost Analysis Improvement
Group (CAIG) as a basis for the program's baseline, while almost 70
percent of the programs used the program office or Service cost
estimate. While cost estimates from the CAIG can underestimate a
program's costs by billions of dollars, GAO has previously found that
these independent estimates generally underestimate costs by a smaller
amount than program office and Service estimates.
Cost estimates are inaccurate in part because they are based on
limited knowledge about the requirements, technologies, design
maturity, and the time and funding needed to execute a program. GAO has
found, for example, that program Cost Analysis Requirements Documents--
used to build the program cost estimate--often lack sufficient detail
about planned program content for developing sound cost estimates.
Without this knowledge, cost estimators must rely heavily on parametric
analysis and assumptions about system requirements, technology, and
design maturity, and the time and funding needed. The assumptions used
in developing estimates also tend to be overly optimistic. Furthermore,
cost estimates that lack knowledge and rely on assumptions have
inherently high levels of risk and uncertainty that are not typically
communicated to decision makers.
36. Senator Vitter. Mr. Sullivan, with the annual Selected
Acquisition Reports and current Nunn-McCurdy reporting requirements, is
Congress suffering from a lack of oversight and reports from DOD?
Mr. Sullivan. In terms of the reports it receives from DOD,
Congress is suffering from a lack of insight into the risks on programs
that could lead to cost, schedule, and performance shortfalls. The
Selected Acquisition Reports and Nunn-McCurdy reporting that Congress
gets from DOD on its major defense acquisition programs primarily
report on program outcomes to date--including quantitative measures of
cost, schedule, and performance over time. While these are useful
indicators of the health of acquisition programs and whether they are
meeting their intended goals, they have limited predictive value. DOD
could improve the information it provides to Congress by reporting on
knowledge metrics that evaluate whether programs have attained certain
levels of technology, design, and manufacturing knowledge by key points
in the acquisition process. This reporting would facilitate the
identification of potential problems that could lead to cost, schedule,
and performance shortfalls before they occur.
37. Senator Vitter. Mr. Sullivan, is there any credence to the
concern that program managers have ``too many'' reports to file and are
not focusing on managing their programs?
Mr. Sullivan. Oversight and reporting requirements need to be
evaluated both from the perspective of the time they take and the value
they provide. In the past, we have reported on concerns in both of
those areas. In a November 2005 report, we reported that program
managers and program executive officers commented that they spend too
much time producing data that is not strategic or very useful to them.
In fact, more than 90 percent of 126 respondents to our survey of
program managers said that they spent either a moderate, great, or very
great extent of their time representing their program to outsiders and
developing and generating information about program progress.
We have also found that the reporting that is done does not include
key information that decisionmakers need to decide if a program is
ready to proceed into the next phase of the acquisition process. For
example, in transitioning from system integration to system
demonstration, we have recommended that DOD ensure the capture of
knowledge about the completion of engineering drawings; completion of
subsystem and system design reviews; agreement from all stakeholders
that the drawings are complete; and identification of critical
manufacturing processes, among other indicators. In the transition to
production, we recommended that DOD capture knowledge about production
and availability of representative prototypes along with statistical
process control data. Our 2005 report indicated that a relatively small
percentage of programs used these knowledge indicators. For example,
only 32 percent of 126 program managers who responded to our survey
said they used design drawing completion extensively to measure design
maturity; only 26 percent said they used production process controls to
a great extent. Even fewer program managers reported that their
immediate supervisor used these measures extensively to evaluate
progress. Broader use and reporting of these types of metrics would
help decisionmakers gauge progress and ensure that programs managers
are: (1) conducting activities to capture relevant product development
knowledge; (2) providing evidence that this knowledge has been
captured; and (3) holding decision reviews to determine that the
requisite knowledge has been captured before proceeding to the next
phase of the acquisition process.
[Whereupon, at 11:52 a.m., the committee adjourned.]
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
