Coast Guard: Progress Being Made on Deepwater Project, but Risks
Remain (02-MAY-01, GAO-01-564).
The Coast Guard is in the final stages of planning the largest
procurement project in its history-the modernization or
replacement of more than 90 cutters and 200 aircraft used for
missions beyond 50 miles from shore. This project, called the
Deepwater Capability Replacement Project, is expected to cost
more than $10 billion and take 20 years or longer to complete.
Congress and the Coast Guard are at a major crossroads with the
Deepwater Project. Planning for the project is essentially
complete, and Congress will soon be asked to commit to a
multibillion-dollar project that will define the way the Coast
Guard performs many of its missions for decades to come. The
deepwater acquisition strategy is unique and untried for a
project of this magnitude. It carries many risks which could
potentially cause significant schedule delays and cost increases.
The Deepwater Project faces risks in the following four areas:
(1) planning the project around annual funding levels far above
what the administration has told the Coast Guard it can expect to
receive, (2) keeping costs under control in the contract's later
years, (3) ensuring that procedures and personnel are in place
for managing and overseeing the contractor once the contract is
awarded, and (4) minimizing potential problems with developing
unproven technology. All of these risks can be mitigated to
varying degrees, but not without management attention.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-564
ACCNO: A00849
TITLE: Coast Guard: Progress Being Made on Deepwater Project,
but Risks Remain
DATE: 05/02/2001
SUBJECT: Aircraft
Contract administration
Cost control
Federal procurement
Ships
Coast Guard: Deepwater Capability
Replacement Project
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-564
Report to Congressional Requesters
United States General Accounting Office
GAO
May 2001 COAST GUARD Progress Being Made on Deepwater Project, but Risks
Remain
GAO- 01- 564
Page i GAO- 01- 564 Deepwater Project Letter 1
Appendix I Objectives, Scope, and Methodology 36
Appendix II Current Deepwater Cutters and Aircraft 39
Appendix III Estimated Service Life of the Coast Guard?s Deepwater Cutters
and Aircraft 40
Appendix IV List of the Members of Each Contracting Team 42
Appendix V Comments From the Department of Transportation 44
Appendix VI GAO Contacts and Staff Acknowledgements 51
Tables
Table 1: Areas of Risk and Overall Risk Levels for the Deepwater Project 5
Table 2: Office of Management Budget Targets for Coast Guard
Capital Budget, Fiscal Years 2002- 2006 11 Table 3: Estimated Service Life
of Deepwater Assets 40
Figures
Figure 1: Comparison of Expenditure Estimates for Deepwater Project, Fiscal
Years 2002- 2006 12 Contents
Page ii GAO- 01- 564 Deepwater Project Abbreviations
DOD Department of Defense DOT Department of Transportation FAA Federal
Aviation Administration IAGC Independent Analysis Government Contractor NASA
National Aeronautical and Space Administration OMB Office of Management and
Budget RFP request for proposals TRL technical readiness levels
Page 1 GAO- 01- 564 Deepwater Project
May 2, 2001 The Honorable Frank A. LoBiondo Chairman The Honorable Corrine
Brown Ranking Minority Member Subcommittee on Coast Guard and
Maritime Transportation Committee on Transportation and
Infrastructure House of Representatives
The Honorable Wayne T. Gilchrest The Honorable Bob Clement House of
Representatives
The Coast Guard is in the final stages of planning the largest procurement
project in its history- the modernization and/ or replacement of over 90
cutters and 200 aircraft used for missions beyond 50 miles from shore. 1
This project, called the Deepwater Capability Replacement Project, is
expected to cost over $10 billion (in 1998 dollars) and take 20 or more
years to complete. Through fiscal year 2001, the Congress has appropriated
$116 million to design the project. 2 The Coast Guard is currently
finalizing its acquisition plan and contracting approach and forming its
organization to oversee the project. During the latter part of 2001, the
Coast Guard will be evaluating proposals now being developed by three
contracting teams competing for a contract to build the deepwater system.
The Coast Guard plans to award a contract to one of these teams early in
2002.
Given the size and scope of the project and its importance to the Coast
Guard, you asked us to monitor the project closely and provide information
for the Congress, as it considers funding for this project. As agreed with
your office, this report focuses on the major risks facing the
1 Deepwater missions require extended on- scene presence of at least 45
days, long transit distance to reach the operating area, or forward
deployment of forces. 2 Of the $116 million, contractors developing
proposals for the system received $61 million, and the Coast Guard received
$55 million.
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 01- 564 Deepwater Project
Coast Guard on this project and the progress the agency has made in
addressing them.
Our assessments of risk are based primarily on federal acquisition criteria,
such as the Federal Acquisition Regulation and the Office of Management and
Budget?s (OMB) Circular No. A- 11 on Capital Programming; discussions with
officials of the Office of Federal Procurement Policy; interviews with
procurement experts; the experience gleaned from conducting a substantial
body of work performed on other acquisitions, especially those of the
Department of Defense (DOD), information technology projects across
government, and OMB and our reports on best practices in capital
decisionmaking and management of major acquisitions. Our assessments of the
progress the Coast Guard has made in addressing these risks are based on
reviews of many Coast Guard documents pertaining to the Deepwater Project
and capital planning in general, as well as numerous detailed discussions
with Coast Guard staff. Throughout this process, we brought our concerns to
the Coast Guard?s attention as soon as possible to increase the opportunity
for useful exchange of information and, where necessary, timely corrective
action. We conducted our work from April 2000 through April 2001 in
accordance with generally accepted government auditing standards. Additional
information on our scope and methodology appears in appendix I.
The Congress and the Coast Guard are at a major crossroads with the
Deepwater Project, in that planning for the project is essentially complete,
and the Congress will soon be asked to commit to a multibillion dollar
project that will define the way the Coast Guard performs many of its
missions for decades to come. The deepwater acquisition strategy is unique
and untried for a project of this magnitude, and it carries many risks which
could potentially cause significant schedule delays and cost increases. The
Deepwater Project faces risks in four key areas: (1) planning the project
around annual funding levels far above what the administration has told the
Coast Guard it can expect to receive; (2) keeping costs under control in the
contract?s later years; (3) ensuring that procedures and personnel are in
place for managing and overseeing the contractor, once the contract is
awarded; and (4) minimizing potential problems with developing unproven
technology. All of these risks can be mitigated to varying degrees, but not
without management attention. Results In Brief
Page 3 GAO- 01- 564 Deepwater Project
Affordability is the biggest risk for the Deepwater Project. The Coast Guard
has chosen a contract approach that depends on a sustained funding level of
about $500 million a year (in 1998 dollars) for the next 2 to 3 decades.
However, according to budget projections from OMB, the Coast Guard faces the
real possibility of a cumulative funding shortfall of almost half a billion
dollars for the project?s first 5 years. Also, the Coast Guard plans to
request additional funds at a time when the administration plans to continue
budget constraints on federal spending and when other Department of
Transportation (DOT) agencies, such as the Federal Aviation Administration
(FAA), are also undertaking major capital projects. The Coast Guard
acknowledges that the potential funding risks are high but believes it
should keep its current approach of developing the project around the
planned funding stream. However, the potential risks are substantial, and
another strategy appears warranted- one that involves a lower funding
scenario around which contractors can develop their proposals. Proceeding
with the project on the basis of unrealistic funding expectations may result
in unwanted consequences. If funding does not materialize as planned after
the contract award, the Coast Guard may have to alter the systems
integrator?s schedule for producing and delivering agreed to quantities and
types of deepwater assets. Altering the schedule would require renegotiating
prices in a sole- source environment and negotiating new cost and
performance guidelines. This would be costly for the Coast Guard and would
set off ripples affecting the acquisition of deepwater equipment for years
to come.
The Coast Guard has selected a novel contracting approach- one never tried
on a contract this large. It calls for procuring ships, aircraft, and
equipment through a single, prime contractor over 2 or more decades. Because
each of the three contractors now competing for the contract is developing
its proposal in conjunction with its own team of companies, it is likely
that the companies in each team will supply most of this equipment. This
approach allows the winning contractor to focus on integrating all of the
Coast Guard?s deepwater ships and aircraft, and the competition between
teams tends to control costs in the project?s early years. However, the
benefits of this competition will diminish after the first few years of the
project because of the Coast Guard?s likely reliance on a single contracting
team. We and others have raised concerns whether the Coast Guard?s planned
contracting approach will be able to control costs and still meet
performance requirements once the contract is awarded in 2002, particularly
since it was adopted without documentation that an in- depth analysis of
alternative contract approaches was done. The Dependence on Sustained
High Funding Levels Controlling Costs After the First Few Years
Page 4 GAO- 01- 564 Deepwater Project
Coast Guard has taken steps to consider this feedback, but it is still
evaluating what steps to take in response.
So far, the Deepwater Project has been in the planning phase. In most
respects, the Coast Guard?s management of this phase has been excellent. In
fact, the Coast Guard?s procedures and management structure for this phase
were among the best of the federal agencies we have evaluated. The Coast
Guard took many innovative steps, such as developing and using performance
specifications to encourage the contractors to develop innovative designs,
using state- of- the- art technology, and involving
?matrix? teams in which Coast Guard personnel representing different program
areas have been fully involved with each team of contractors. While its
management, during the planning phase, provides a solid foundation for the
project, the next phase presents considerably tougher challenges. The
challenges include recruiting and training enough staff to manage and
oversee the contract, determining how the oversight effort will be
structured, determining how to manage its relationship with subcontractors
as well as the prime contractor, ensuring useful segments of the project are
fully funded in advance of buying equipment, measuring the effects on
operations and total system costs as new equipment replaces existing ships
and aircraft, and developing contingency plans in case problems develop with
the performance of the prime contractor or subcontractors.
Our reviews of other acquisitions show that reliance on unproven technology
is a frequent contributor to escalated costs, schedule delays, and
compromised performance standards. As with contract oversight, the Coast
Guard?s initial steps in countering this risk have been very good. The Coast
Guard has told contractors to emphasize ?off- the- shelf? technology (i. e.,
technology that is commercially available and already in similar use). Our
review of 18 key technologies that contractors are proposing to use in the
first 7 years of the project showed that almost all should be sufficiently
mature by the time the contract is awarded. However, there is less certainty
about technology in later years. The Coast Guard needs a structured process
for assessing and monitoring this risk. So far, it has none.
Our overall assessment of the risk levels in the four areas is shown in
table 1. We are making recommendations to help the Coast Guard improve the
long- term success of the Deepwater Project, including one for DOT to have
contractors now competing for the deepwater contract to develop their final
proposals around a funding stream consistent with OMB budget Managing and
Overseeing
the Acquisition Phase of the Project
Minimizing Problems With Unproven Technology
Page 5 GAO- 01- 564 Deepwater Project
targets. In commenting on a draft of this report, DOT disagreed with this
recommendation, noting that out- year budget targets for the Coast Guard?s
capital projects have increased and will continue to increase to accommodate
deepwater funding needs. We believe that DOT?s position on this matter
leaves the Deepwater Project vulnerable to cost increases and schedule
slippages and could jeopardize the Coast Guard?s ability to achieve its
missions effectively if funding shortfalls occur. We are also raising an
issue for congressional consideration that could help the Congress in making
funding decisions on the project.
Table 1: Areas of Risk and Overall Risk Levels for the Deepwater Project
Area of risk Risk level Reasons for assigning this level of risk
Attaining a stable, sustained funding level High Several years of funding
substantially below planned funding levels can have adverse
consequences for the acquisition strategy. Budget constraints and other
budget priorities threaten the Coast Guard?s ability to achieve large,
sustained increases in its budget for capital spending. Controlling costs in
the contract?s later years Moderate to High The risks center on the
potential lack of future competition and reliance on a single
contractor to procure the entire system. The level of risk depends on the
effectiveness of provisions the Coast Guard designs and includes in the
contract to encourage or require subcontract competition and increase its
leverage in negotiating future contracts with the prime contractor.
Overseeing the acquisition Moderate Although there are many uncertainties
about contract management as the Coast
Guard increases the size of the administrative effort, the commendable start
and the ability to make specific changes lessen the degree of risk in this
area. Using unproven technology Low to moderate The steps needed to mitigate
this risk are relatively few and straightforward. The lack
of an assessment tool to measure technology maturity poses a short- term
(low) and long- term (moderate) risk.
Source: GAO analysis of risk areas.
Cutters, patrol boats, airplanes, and helicopters are all critical to
meeting the Coast Guard?s deepwater missions that are beyond the range of
shorebased small boats. 3 These missions include actions such as enforcing
fisheries laws, intercepting drug smugglers and illegal immigrants, and
conducting search and rescue missions far out at sea.
3 See app. II for a list of the various classes of the 93 ships and 206
aircraft currently assigned to deepwater missions. Background
Page 6 GAO- 01- 564 Deepwater Project
Many of the Coast Guard?s current cutters were built in the 1960s, and many
of the aircraft in the 1970s and 1980s. Although these ships and aircraft
have been upgraded in a number of ways since being acquired, the Coast Guard
has documented a number of performance and support problems, such as the
following:
poor sensors and night operations capability on both aircraft and cutters,
limited ability of cutters and aircraft to operate effectively together,
inadequate communications, and
high operating and maintenance costs. In a November 1995 mission analysis
report, the Coast Guard cited its rapidly aging deepwater fleet as a
justification to begin a project for acquiring new ships and aircraft. In
1998, we reported that the service life of the Coast Guard?s deepwater ships
and aircraft might be much longer than the Coast Guard originally estimated
in its 1995 analysis. 4 We recommended that the Coast Guard develop
additional information on the remaining service life of ships and aircraft.
In 1998, the Coast Guard determined that the service life of the various
aircraft classes could be extended by about 11 to 28 years over original
estimates, assuming that increased maintenance and upgrades occur. In
addition, by January 2001, the Coast Guard had issued an updated analysis
that extended the service life of two of the four ship classes by an
additional 5 years, assuming that increased maintenance and upgrades occur.
5 The Coast Guard provided this information to its contractors so that they
could use it in developing their proposals.
In December 1999, an interagency task force on the roles and missions of the
Coast Guard reported that recapitalization of the Coast Guard?s deepwater
capability is a near- term national priority and endorsed the Deepwater
Project?s process and timeline. 6 Although our earlier work took issue with
the Coast Guard?s initial analysis of how soon its deepwater assets would
need to be replaced, we do not now take issue with the Coast
4 Coast Guard?s Acquisition Management: Deepwater Project?s Justification
and Affordability Need to Be Addressed More Thoroughly (GAO/ RCED- 99- 6,
Oct. 26, 1998). 5 See app. III for the current estimated service life of
existing deepwater cutters and aircraft. 6 Interagency Task Force on Coast
Guard Roles and Missions, A Coast Guard for the Twenty First Century, Dec.
3, 1999. Why Does the Coast Guard
Need to Replace or Modernize Deepwater Ships and Aircraft?
Page 7 GAO- 01- 564 Deepwater Project
Guard?s position that it needs to modernize these assets, especially due to
the additional studies completed since our 1998 report. Our congressional
directive in this report has been to examine the acquisition approach.
The acquisition approach for the Deepwater Project is innovative. Rather
than using the traditional approach of replacing an individual class of
ships or aircraft, the Coast Guard has adopted a ?system- of- systems?
approach intended to integrate ships, aircraft, sensors, and communication
links together as a system to accomplish mission objectives more
effectively. The Coast Guard expects this approach will both improve the
effectiveness of deepwater operations and reduce operating costs.
The project has two basic phases- a design phase (called ?concept
exploration? and known as phase 1) and a final proposal preparation and
procurement phase (called ?demonstration and validation/ full- scale
development? and known as phase 2). Phase 1 began in March 1998. As part of
this phase, the Coast Guard contracted with three competing teams of
contractors to conceive and begin designing a proposed deepwater system. 7
Each proposal is to be based on meeting a set of performance specifications
developed by the Coast Guard. Each team was instructed to develop its
proposal on a funding stream of $300 million for the first year and $500
million annually until the project is completed. These amounts are in
constant 1998 dollars; actual funding would be higher to account for
inflation. Phase 1 ends with each team?s development of a proposed deepwater
concept, the functional design for which will be 80- percent complete.
In phase 2, which begins in June 2001, the Coast Guard plans to issue a
request for proposals (RFP) to the three industry teams to develop final
proposals. The current schedule calls for these proposals to be completed
and submitted to the Coast Guard during the last quarter of fiscal year
2001. The Coast Guard will evaluate which proposal provides best value for
the government as gauged mainly by a combination of improvements in
7 See app. IV for a complete list of participants from each contracting
team. The Acquisition Approach
for the Deepwater Project
Page 8 GAO- 01- 564 Deepwater Project
operational effectiveness 8 and minimizing total ownership costs. 9 Other
evaluation factors include the technical feasibility of the proposed design
and the management capability of the systems integrator.
When the deepwater contract is awarded early in 2002, the contract will
actually be between the Coast Guard and the prime contractor, known as the
?systems integrator,? of the winning contracting team. This systems
integrator will be responsible for ensuring that each ship, aircraft, or
other equipment is delivered on time and in accordance with agreed to
prices. The systems integrator will also be called on to deliver the
complete deepwater system in compliance with the Coast Guard?s system
performance specifications. The Coast Guard adopted this approach because it
does not believe it has the technical expertise or the resources to be a
systems integrator. Also, the Coast Guard believes that a team of
contractors led by a systems integrator would provide the best method of
acquiring a set of ships, aircraft, and other equipment and would optimize
improvements in operational effectiveness and total ownership costs. This
contracting approach could thus result in a long- term contractual
arrangement with a single contractor and its team of subcontractors.
The Coast Guard plans to have an initial 5- year contract with the systems
integrator. The systems integrator would receive a base award for management
and system integration services. Assuming the project proceeds as planned,
task and delivery orders for deepwater equipment would be issued by the
Coast Guard in accordance with the systems integrator?s implementation
schedule. 10 If the performance of the systems integrator is satisfactory
for each award- term contract, the Coast Guard plans to award follow- on,
award- term contracts (as many as five for successive 5- year, award- term
contracts) with the same systems integrator. The Coast Guard plans to
negotiate prices with the systems integrator on the follow- on contracts.
8 Operational effectiveness involves the Coast Guard?s ability to carry out
its deepwater missions. For example, it includes the number of lives saved,
the amount of drugs interdicted, and the number of immigrants interdicted.
9 Total ownership costs include acquisition, operating, maintenance, and
support costs for the deepwater system over a 40- year period. 10 The Coast
Guard will have some flexibility to delay issuing task and delivery orders
based on funding or other considerations. However, major deviations from the
minimum scheduled orders, as laid out in the systems integrator?s
implementation plan, will require rebaselining performance targets and
prices in a sole- source environment.
Page 9 GAO- 01- 564 Deepwater Project
The Congress is at a critical juncture with the Deepwater Project because
the success of the contracting approach rests heavily on the Coast Guard
being able to count on sustained funding of about $500 million (in 1998
dollars) for 20 years or more. The contracting approach the Coast Guard has
selected is not easily adaptable to lower levels of funding without
stretching the schedule and increasing costs. However, there are signs that
funding levels may be lower than the planned amount. Although the
administration?s budget request for the Deepwater Project for fiscal year
2002 will be about 10 percent less than the project?s planned first- year
funding, the average shortfall for fiscal years 2003 to 2006 is about 20
percent. 11 Moreover, much of the funding for fiscal year 2002 is from the
Western Hemisphere Drug Elimination Act (P. L. 105- 277), a source that will
not be available after this year unless the act is extended.
Capital funding for the Coast Guard is in competition with many other
potential uses of federal funds within the Coast Guard itself, DOT as a
whole, and other federal agencies. To accommodate the Deepwater Project, the
Coast Guard is proposing to limit spending on its other ongoing capital
projects to levels far below where they have been in decades. Given these
various budgetary pressures, it appears advisable to have contractors
develop their plans around a lesser amount. The Coast Guard?s approach,
however, is inextricably tied to the more optimistic option. In using a
lower, more realistic funding level aligned with OMB budget projections, the
Coast Guard could lessen the risk of future cost increases, schedule
stretch- outs, and low system performance levels.
The contract approach that the Coast Guard has decided to use for the
Deepwater Project depends on a large, sustained, and stable funding stream
over the next 2 to 3 decades. The approach is based on acquiring ships and
aircraft on the contractor?s proposed schedule so that they will form a
?system of systems.? Substantial funding shortfalls cannot only affect the
ships and aircraft scheduled for acquisition in the short term; but, they
can also set off ripples affecting the acquisition of deepwater equipment
for years to come. Adjustments that may be needed include
11 Until this point, the figures presented for the Deepwater Project have
been in constant 1998 dollars, the approach used in Deepwater Project
planning documents. However, the funding amounts that OMB has assigned to
the Coast Guard are in actual ?year- ofexpenditure dollars,? with amounts
for future years adjusted for inflation. To conform this discussion to the
OMB numbers, the amounts in this discussion are presented in currentyear
dollars. Viability of
Contracting Approach Depends on a Sustained High Level of Funding
Success of Contracting Approach Relies on Sustained High Funding
Page 10 GAO- 01- 564 Deepwater Project
revising the implementation plan for delivering equipment, renegotiating
prices for deepwater equipment, and negotiating new cost and performance
baselines with the systems integrator. Such adjustments would not only be
costly; but, they could also slow the schedule to the point that (1) total
ownership costs would rise and (2) advantages projected in the contractor?s
proposal, such as improvements in operational effectiveness, would not
materialize. At the extreme, funding shortfalls would affect the Coast
Guard?s ability to proceed with the contract as well as the agency?s ability
to perform its deepwater missions.
The decision on funding the Deepwater Project rests ultimately with the
Congress, but because this decision has yet to be made, we used the
administration?s budget proposal for the Coast Guard (as contained in the
budget documents prepared by OMB) as a starting point for analyzing the
funding issue. OMB?s budget targets for fiscal years 2002 through 2006 do
not propose specific amounts for the Deepwater Project; rather, they provide
a single amount for all Coast Guard capital projects. As table 1 shows, this
overall total ranges from $659 million (in- year- of expenditure dollars) in
fiscal year 2002 to $719 million in fiscal year 2006. Because the Coast
Guard has many other capital projects under way besides the Deepwater
Project, it must decide how this money will be allocated among them. After
receiving the budget targets from OMB in early March 2001, the Coast Guard
estimated that the amount available for the Deepwater Project would range
from $338 million (in year- of- expenditure dollars) in fiscal year 2002 to
$547 million in fiscal year 2006. Administration?s Funding
Proposals Signal Caution About Whether Projected Funding Will Be Available
Page 11 GAO- 01- 564 Deepwater Project
Table 2: OMB Budget Targets for Coast Guard Capital Budget, Fiscal Years
2002- 2006
Amounts in millions
Coast Guard?s planned distribution of budget target set by OMB
Fiscal year
OMB budget target for Coast
Guard?s capital projects
To be spent on Deepwater Project To be spent on
other projects
2002 $659 $338 $321 2003 673 396 277 2004 688 442 246 2005 704 508 196 2006
719 547 172
Note: Amounts are in current (year- of- expenditure) dollars. Source: U. S.
Coast Guard 5- year funding projection.
If the Coast Guard proceeds with its current plans in issuing the RFP,
contractors will be instructed to develop plans around a much higher funding
stream than is available under the OMB budget targets. For example, the
funding stream that the Coast Guard currently plans to use for the project
($ 350 million the first year and $525 million in subsequent years) is in
1998 dollars. 12 Adjusted for inflation, this figure becomes $373 million in
fiscal year 2002 (compared with OMB?s target of $338 million) and $569
million in fiscal year 2003 (compared with the target of $396 million). By
the end of fiscal year 2006, the cumulative gap will total $496 million.
While this shortfall may not seem so significant in the scheme of the
overall budget, this amount is significant in the context of DOT?s total
budget, especially given the competition among DOT agencies for available
funding- a point that we discuss in more detail below. Figure 1 shows that
the annual gap between planned funding and the amount available under OMB
budget targets ranges from $35 million to $173 million.
12 These amounts include $300 million for the systems integrator and $50
million for the Coast Guard to oversee the contract in the first year and
$500 million for the systems integrator and $25 million for the Coast Guard
in subsequent years.
Page 12 GAO- 01- 564 Deepwater Project
Figure 1: Comparison of Expenditure Estimates for Deepwater Project, Fiscal
Years 2002- 2006
Note: Amounts are in current (year- of- expenditure) dollars. Source: U. S.
Coast Guard?s 5- year funding projection and GAO analysis of deepwater
funding levels in year- of- expenditure dollars.
Although the Coast Guard may be able to begin the project as planned at the
level of funding provided for fiscal year 2002, the success of this first
year may provide a false sense of security about how easy it will be to
sustain projected funding levels. In fiscal year 2002, spending is
relatively low compared with later years. Coast Guard officials said they
plan to fully fund the contractor?s share of the planned amount and trim
their own administrative expenses related to the project. In subsequent
years, when planned payments to contractors rise much more steeply than
amounts available, the gap may be far less manageable. 13 In addition, the
project?s first- year funding comes mainly from a source that will be soon
exhausted. About $243 million of the amount proposed for the Deepwater
Project in
13 These amounts do not include any costs for engineering change proposals
or incentive payments for the systems integrator.
0 100
200 300
400 500
600 700
2002 2003 2004 2005 2006 Gap between available funding and projected funding
in Coast Guard's current proposal Projected funding for Deepwater under
Coast Guard's current proposal Millions of dollars
Annual gap between funding available and projected funding in Coast Guard?s
current proposal ranges from $35 million to $173 million
Page 13 GAO- 01- 564 Deepwater Project
fiscal year 2002 would come from funds authorized in the 1998 Western
Hemisphere Drug Elimination Act (P. L. 105- 277), which will expire this
year unless it is extended. OMB officials told us that it plans to request
additional appropriations under this act in fiscal years 2003 to 2006.
Another concern is the potential effect of the Deepwater Project on other
Coast Guard capital projects planned or under way. These other capital needs
include, for example, modernizing communication equipment used to support
search and rescue activities, and upgrading various shoreside facilities,
such as boat stations and housing. The overall amount the Coast Guard now
plans to spend on these projects is substantially less than the agency
indicated in plans just a year ago. For example, in 2000, the Coast Guard?s
planning documents proposed spending $475 million on nondeepwater projects
in 2005. However, under the current plan, that spending level would drop by
more than half, to about $196 million. In part, these proposals reflect the
fact that some of the other capital projects, such as the buoy tender
project, will be winding down, and the costs of other projects will be
absorbed as part of the Deepwater Project. Rather dramatic reductions in
other capital projects cannot be explained as easily. For example, estimated
spending for improved shore facilities in fiscal year 2005 dropped from $147
million in last year?s plan to $59 million in the current plan. If estimates
in the current plan hold true, fiscal year 2006 spending for nondeepwater
projects will be at its lowest level in decades and call into question the
validity of the agency?s estimates to maintain its current nondeepwater
infrastructure. The presence of these other capital needs cannot be
forgotten in assessing how ready the Coast Guard is to assume the risks of
the Deepwater Project.
The fiscal environment in which the Coast Guard must obtain funds for the
Deepwater Project and other capital needs is further complicated by
competition for funds with other DOT priorities. Obtaining additional
funding for the Coast Guard within the DOT budget is likely to be difficult
because of competition with other entities within the DOT appropriation,
such as the FAA and Amtrak, for available discretionary funding. For
example, recent action by the Congress limited FAA?s ability to use a
separate funding source (the Airport and Airway Trust Fund) to fund Future
Funding for
Deepwater Will Compete With Other Budget Priorities
Page 14 GAO- 01- 564 Deepwater Project
FAA?s operations. 14 As a result, funding for FAA?s operations now competes
for the same limited DOT dollars on which the Deepwater Project would rely.
FAA also expects its operating costs to increase to $7.4 billion by 2003, a
42- percent increase from 1998 levels. Similarly, Amtrak estimates that its
capital needs alone will amount to about $1.5 billion annually through
fiscal year 2020, part of which would come from the DOT budget.
Outside of DOT, the overall budget process is still driven by caps in
discretionary spending. 15 If these caps (which currently cover through
fiscal year 2002) are extended as the administration has proposed, funding
for the Deepwater Project would have to come from cuts in some other agency
or program. The percentage increase in the Coast Guard?s budget request is
among the largest of all federal agencies. However, the Coast Guard is
basing its plans for the Deepwater Project on another major boost in funding
beyond 2002. Thus, for all these reasons, sustaining the Deepwater Project
at the funding level the Coast Guard is currently planning to use in its RFP
appears to be a difficult task over a sustained period.
14 The Wendall H. Ford Aviation Investment and Reform Act for the 21st
Century (P. L. 106- 181), also known as AIR- 21, calls for FAA?s airport
improvement program and facilities and equipment accounts to be funded at
authorized levels before any revenues from the Airport and Airways Trust
Fund are allocated to FAA?s operations account. According to the DOT
Inspector General, barring a tax increase, Trust Fund receipts and interest
will clearly be inadequate to fund all of FAA?s operations costs.
15 Statutory limits on total discretionary spending have been in effect
since fiscal year 1991. The Balanced Budget and Emergency Deficit Control
Act of 1985 (the ?Deficit Control Act?) as amended by the Budget Enforcement
Act of 1990, the Omnibus Budget Reconciliation Act of 1993, and the Budget
Enforcement Act of 1997, established statutory limits on the federal
government?s discretionary spending for fiscal years 1991 through 2002.
Under these limits, outlays for discretionary spending will remain almost
constant in dollar terms from fiscal year 1998 through fiscal 2002.
Page 15 GAO- 01- 564 Deepwater Project
Our concern about this risk is not new. In several previous reports 16 on
the Coast Guard?s planning for the Deepwater Project, we expressed concern
about the Coast Guard overestimating the amount of funding that would be
available in the future for the project. The Coast Guard agrees that funding
for the Deepwater Project is high risk and that it provides limited funding
flexibility but believes it should keep its current approach of developing
the project around the planned funding stream. Coast Guard managers believe
that a deepwater system funded at planned levels provides the optimum system
to meet deepwater requirements. The agency also believes that OMB budget
targets could rise in the future and that the Congress could appropriate
more funds to meet the agency?s capital needs. However, the potential risks
are substantial and another strategy appears warranted. That strategy would
be to develop a lower funding scenario around which the contractors can
develop their proposals. If the project needs to be adjusted to a lower,
more realistic funding stream, the time to do so is before the contract
proposals are finished later this year.
Directing contractors to develop proposals around a lower funding scenario
aligned with OMB targets would have several advantages. 17 First, the Coast
Guard would have greater opportunity to evaluate which proposal will produce
the best value to the government within likely budget constraints. Second,
the agency would be in a better position to hold the contractor accountable
for delivering a system that meets original
16 Coast Guard?s Acquisition Management: Deepwater Project?s Justification
and Affordability Need to Be Addressed More Thoroughly (GAO/ RCED- 99- 66,
Oct. 26, 1998). Coast Guard: Key Budget Issues for Fiscal Years 1999 and
2000 (GAO/ T- RCED- 99- 83, Feb. 11, 1999). Coast Guard: Strategies for
Procuring New Ships, Aircraft, and Other Assets (GAO/ T- RCED- 99- 116, Mar.
16, 1999). Coast Guard: Budget Challenges for 2001 and Beyond (GAO/ T- RCED-
00- 103, Mar. 15, 2000). Performance and Accountability: Challenges Facing
the Department of Transportation (GAO- 0- 443T, Feb. 14, 2001). 17 DOD
provides a good example of how the linkage between the capital planning and
budgeting processes can be made and used to make budgeting decisions on
capital projects, such as the Deepwater Project, and to manage them
effectively. DOD links its capital planning process to its budget through
its Future Years? Defense Plan, which is updated each year to reflect
changing conditions. This plan is linked to OMB budget targets and used to
make programming and budgeting decisions over a 5- year budget cycle. It
identifies strategies for meeting budget targets, such as cost- savings in
operations that could be used to help fund capital requirements. In
addition, according to an OMB official, the plan identifies the funds needed
to complete projects and provides greater assurance that these funds will be
available, which can ultimately lead to better- managed capital projects.
The plan also allows the Congress to see where DOD is heading with its
capital projects. Developing Deepwater
Around Lower Funding Scenario Would Help Reduce Risk
Page 16 GAO- 01- 564 Deepwater Project
schedule and cost estimates if it selects the plan developed at the lower
funding level. Using realistic funding expectations will reduce the risk of
schedule stretch- outs and cost increases with the contractor in a
solesource environment after the contract is awarded- a situation in which
the government?s leverage is reduced because it does not have the benefit of
competition for obtaining a fair and reasonable price.
Any projection about likely funding levels for a project that lasts as long
as the Deepwater Project will involve an element of uncertainty and risk.
The Coast Guard?s current funding scenario exacerbates that risk. Because
the Coast Guard has not yet issued its request for contractors to submit
their best and final proposals, there is still time to mitigate the risk by
identifying a lower funding stream that contractors should use in developing
their proposals.
The deepwater contracting approach that the Coast Guard adopted has never
been tried on a contract this large extending over 20 or more years. At the
time it was adopted, there was little evidence that the Coast Guard had
analyzed whether the approach carried any inherent difficulties for ensuring
best value to the government and, if so, what to do about them. 18 We and
others who are involved in reviewing this approach, such as OMB and the
Office of Federal Procurement Policy, have expressed concerns about the
potential lack of competition during the project?s later years and the
reliance on a single contractor for procuring so much of the deepwater
equipment. The Coast Guard is still conducting this analysis on its approach
as it moves into phase 2 of the project and has delayed some of its key
milestones to consider these concerns.
When the Coast Guard selected the contract approach in May 2000, it had not
yet documented the risks involved or the degree to which this approach
provided better value than other approaches. Contracting officials within
the Coast Guard said their guidance from Coast Guard management had been to
develop an approach that would (1) allow a single systems integrator to
create a ?system of systems? approach and (2) achieve potential improvements
in operational effectiveness and minimize
18 For example, the Coast Guard?s risk- management plan did not include the
contracting strategy as an issue that required monitoring or mitigation
strategies. Ability to Control
Costs in Project?s Later Years Remains Uncertain
Little Evidence of Analysis Before Contract Approval Adopted
Page 17 GAO- 01- 564 Deepwater Project
total ownership costs. Contracting officials told us that with these
parameters in mind, they conducted a limited evaluation of several
contracting alternatives by meeting informally with government and private
sector officials about the Coast Guard?s proposed approach and meeting
internally to discuss possible strengths and weaknesses of three approaches.
19 Documentation detailing the basis for the decision- the depth of the
analysis performed, the factors considered, the expertise sought (people
contacted), and the compelling reasons why the current approach was chosen-
was not recorded prior to its approval by Coast Guard acquisition officials.
Without thorough documentation in this regard, the rigor of the Coast
Guard?s analysis of the approach is unknown.
When we initially reviewed the Deepwater Project proposed contracting
approach in March 2000, we expressed concerns about whether it could keep
costs from rising and ensure good performance once the contract is awarded.
20 We discussed the Coast Guard?s approach with contracting experts from
both the public and private sector who, in addition to their concern about
the Coast Guard?s ability to control costs, also raised concerns about
certain management- related issues, which we cover later in this report.
Presently, we focus on the cost- related issues of concern, namely the
potential absence of competition for subcontracts in the project?s later
years and the heavy reliance on a single- systems integrator to procure the
entire system.
OMB guidance recognizes the value of competition as a lever to keep contract
costs down. The benefits of competition are present in the contract?s early
years, as are other approaches for controlling costs. For the initial 5-
year award term contract, prices for equipment and software to be procured
are based on competition; and when the contract with the
19 The three approaches were (1) using a mix and match strategy that
involved contracting with a systems integrator and then competing
individuals classes of Deepwater ships, aircraft, systems or services in
accordance with the systems integrator?s deepwater design; (2) recompeting
the systems integrator?s contract every 5 years; and (3) awarding a single
contract to the systems integrator and acquiring all subsequent acquisitions
through task orders in accordance with the systems integrator?s design.
20 Coast Guard: Budget Challenges for 2001 and Beyond (GAO/ T- RCED- 00-
103, Mar. 15, 2000). Approach Has Limitations
for Ensuring That Costs Are Controlled
Potential Absence of Competition in the Project?s Later Years
Page 18 GAO- 01- 564 Deepwater Project
systems integrator is awarded, these prices will be fixed, according to
Coast Guard officials. 21 The Coast Guard also hopes to control costs by
encouraging the use of commercially available (nondevelopmental) equipment.
Prices for such equipment can be determined on the basis of previous orders
from other buyers and by the use of fixed- price contracts.
Beyond the first 5- year award term contract, however, the benefits of
competition are less certain. In a practical sense, the opportunity for
competition in the project?s out years is diminished because the systems
integrator will likely contract with those suppliers that were part of the
team putting together the offer rather than opening the contract to a wider
set of offerors. Coast Guard officials currently believe that a profit
motive could drive the systems integrator to open competition to a wider set
of offerors. Although this is possible, it would be easier to integrate
equipment or subsystems acquired from a team member since equipment will be
procured based on the plan developed by the team. A Coast Guard analysis of
the same issue draws this same conclusion. We believe that this potential
lack of competition reduces the normal marketplace control on price and
subjects the Coast Guard to situations in which the supplier could
potentially drive up project costs.
The Coast Guard is attempting to develop strategies for encouraging
competition among suppliers, and thereby controlling costs, in subsequent 5-
year award term contracts. One approach involves providing incentives for
the systems integrator to submit ?competitive proposals?- that is, proposals
that are reasonably priced-- beyond the first few years of the contract.
Contracting experts brought in by the Coast Guard discouraged this approach,
saying such incentives usually have limited effectiveness. As a result, the
Coast Guard now indicates it will evaluate the systems integrator?s
performance in minimizing total ownership costs as part of its decision of
whether to renew the systems integrator?s contract. By doing so, the Coast
Guard hopes that this will encourage the systems integrator to have
competition. At this point, it is not clear what effect this evaluation
would have.
A second approach the Coast Guard plans to take is to negotiate a ceiling on
the amount that will be paid for deepwater equipment in the 5- year
21 The Coast Guard expects the contract will have a provision allowing
adjustments to prices due to changes in funding or project requirements.
Page 19 GAO- 01- 564 Deepwater Project
period covered by a follow- on, award- term contract. This is a continuation
of the approach being taken for the first 5- year contract. However, the
ceiling could be waived if the project?s schedule or requirements are
changed. Given the funding- related concerns discussed earlier, the
potential for such changes cannot be easily dismissed. If such changes
occur, the Coast Guard will rely on the systems integrator to negotiate
prices with its vendors in a sole- source environment. Although doing so is
a valid alternative for pricing a contract, a sole- source environment
leaves little leverage in negotiations and therefore carries a higher risk
of goods being overpriced. This approach also carries the burden of
obtaining and reviewing cost and pricing data from suppliers and the systems
integrator.
Another cost- related concern involves dependence on the systems integrator
for a deepwater system that will take 20 or more years to acquire. This
dependence is both one of the main strengths of the approach and one of its
main weaknesses. On the positive side, if all aspects of the approach work
well, the systems integrator will form a partnership with the Coast Guard
and provide the technical expertise to assemble an integrated system and the
continuity needed to bring a longterm project to a successful conclusion.
However, the approach could establish the integrator as a monopoly supplier,
substantially constraining the Coast Guard?s options or leverage. The Coast
Guard could be in a weak position to negotiate aggressively on price because
of its reluctance to take on the risks of increased costs and other problems
associated with switching systems integrators. For example, if the systems
integrator?s performance is unsatisfactory, a new systems integrator will
have to step in to implement someone else?s partially completed design; or
the Coast Guard will have to adopt a more traditional approach of buying
individual classes of ships or aircraft, according to Coast Guard officials.
The learning curve and other complications involved in such a midcourse
adjustment could be dramatic and would probably be very costly.
As our work progressed, we expressed our concerns to the Coast Guard
immediately rather than waiting until the end of our review. As we raised
these concerns, the Coast Guard took additional steps to study them.
However, some of these efforts are still under way, and decisions have not
been made on all specific measures to be incorporated into the acquisition
plan and the RFP for the Deepwater Project.
In September 2000, we urged the Coast Guard to take several actions to deal
with the risks of the contracting strategy it had selected. We suggested
that the Coast Guard identify and evaluate all viable contracting Reliance
on Single Contractor
Steps to Mitigate Contracting Risks Still Under Way
Page 20 GAO- 01- 564 Deepwater Project
approaches, discuss the approaches with contracting experts, and document
the results. In particular, we stated that the Coast Guard should be open to
options that would maximize the benefits of competition in later years while
still maintaining the interoperability of the system. We also urged the
Coast Guard to convene an independent panel of contracting experts from the
government and private sector to review the proposed deepwater contracting
approach or whatever approach the Coast Guard selected. We felt that given
the contract?s uniqueness and the risks it poses, a rigorous review by a
widely represented panel of experts was essential to both validate the Coast
Guard?s approach, and recommend potential mitigating measures to strengthen
it.
In December 2000, the Coast Guard proposed a limited peer review- one
involving experts only from DOD and GAO and consisting of a 3- hour process
(a 1- hour presentation on the contracting approach that the Coast Guard
plans to use, followed by a 2- hour question- and- answer session). Because
of our concerns about the limited nature of this approach, the Coast Guard-
with our advice and assistance- expanded the panel of experts and adopted a
more extensive, structured format. OMB officials share our concerns about
the contracting approach, and they support the need for a peer review and a
careful consideration of issues raised before the RFP and acquisition plan
are finalized. 22 We also urged the Coast Guard to provide the experts with
key documentation, namely the acquisition plan and excerpts from the draft
RFP prior to the meeting. Doing so would better ensure that the members of
the panel have an objective basis for evaluating the Coast Guard?s
contracting approach. However, the Coast Guard decided not to provide such
documents to the panel members in advance, but instead provided them with
selected excerpts from the acquisition plan. The entire acquisition plan and
RFP was available for the panel upon request.
Subsequent to our discussions with Coast Guard management, the Coast Guard
contracted with two outside consultants to review the proposed contracting
approach for the Deepwater Project. One consultant 23 was tasked to develop
and recommend a contracting strategy for the deepwater system, given the
Coast Guard?s requirement for an integrated
?system of systems? solution. The consultant determined that the Coast 22
The Coast Guard plans to finalize its acquisition plan in early 2001. 23 The
Coast Guard contracted with Performance Management Consulting, Inc., to
review its contracting strategy. The firm submitted a summary of its
findings in January 2001.
Page 21 GAO- 01- 564 Deepwater Project
Guard should continue with the approach it selected. However, citing cost
increases and limited cost negotiation leverage as weaknesses, the
consultant identified risk mitigation strategies, such as including in the
RFP requirements for increasing competition over the mid to long term. A
second consultant 24 evaluated the draft RFP. He noted that this was one of
the most complex contracts he had ever seen and suggested that it be
simplified. For example, he suggested that the Coast Guard consider using
incentives as part of the provisions of the award- term contract rather than
as a separate item. He also observed that the success of the contracting
approach is dependent on the Coast Guard receiving the planned funding
stream.
To address the concerns raised by the consultants and to provide some time
to respond to additional concerns that might be raised by the peer review,
the Coast Guard has altered its planned date for issuing the deepwater RFP.
The Coast Guard now plans to release the RFP in June 2001, or about 2 months
later than its initial schedule. The Coast Guard is still responding to
comments from its consultants and industry. Making necessary revisions to
the RFP before giving it to contractors is important, because the RFP
represents the contractual basis upon which the Coast Guard and the
contractor will develop their relations. Also, changing the RFP after it has
been issued could result in contractors having to amend their offers.
At this point, we do not know what changes the Coast Guard might adopt.
Until adequate steps are in place to address concerns expressed by its
consultants and by members of the peer review, we believe the risk related
to cost control remains high. The Coast Guard?s success in this area also
rests on how well it develops other sound strategies and options for
managing potential problems. These strategies are discussed in the next
section.
24 The Coast Guard hired John Cibinic, Jr., Professor Emeritus of Law,
George Washington University, to review the draft RFP. He submitted a
summary of his findings in January 2001.
Page 22 GAO- 01- 564 Deepwater Project
Another area of potential risk involves the overall management and day-
today administration of the contract. In this regard, the Coast Guard?s
performance during the planning phase has been generally excellent. During
this phase, the Coast Guard took several innovative steps to establish and
communicate what it wanted contractors to do, and it had adequate processes
and trained staff in place to carry out the management tasks that needed to
be done. As the project moves into the procurement phase, these challenges
become more difficult, in large part, because the scope of work is so much
greater and the contracting approach is unique and untried. It is too early
to know if the Coast Guard can repeat the same strong performance on this
much larger scale, because plans for managing and administering the
deepwater contract are still being developed. The major challenges the Coast
Guard faces involve developing and implementing plans for (1) establishing
effective human capital practices, (2) having key management and oversight
processes and procedures, (3) forming close relationships with
subcontractors, (4) funding useful segments of the project, (5) tracking
data to measure contractor performance, and (6) having an exit strategy and
a contingency plan in the event of poor performance by the systems
integrator.
In the planning phase of the project, the Coast Guard applied a number of
?best practice? techniques recommended by OMB and others. 25 Among them are
the following:
The Coast Guard gave contracting teams mission- based performance
specifications, such as the ability to identify small objects in the ocean,
rather than asset- based performance specifications, such as how large a
cutter should be, and then it gave them leeway in deciding how to meet these
specifications. Specifying performance criteria is the more traditional
approach.
The Coast Guard established a management structure of Coast Guard and
contractor teams for rapidly communicating technical information. Among
other things, these teams assess each contractor?s evolving proposal to
determine if it will meet contractual requirements and identify issues that
could potentially have unacceptable effects. Communication mechanisms
include an Internet Web site.
25 Best practices are those that have been found to work well and that are
generally recommended by OMB and others. Overseeing the
Acquisition Phase of the Deepwater Project Poses New Challenges
Project Management During the Planning Phase Was Generally Excellent
Page 23 GAO- 01- 564 Deepwater Project
The Coast Guard highlighted the use of ?open- system architecture? and
emphasized the use of commercially supported products in the equipment to be
acquired. This means that communication and computer equipment can be more
easily replaced and upgraded without proprietary software or other unique
requirements.
The Coast Guard also had effective procedures and a management structure in
place for this phase of the project. Using a model developed by Carnegie
Mellon University, 26 we assessed the procedures and structure in eight key
areas- planning, solicitation, requirements development and management,
project management, contracting and oversight, evaluation, transition and
support, and risk management. Within these 8 areas, we examined 112 key
practices and found no significant weaknesses. 27 In fact, the Coast Guard?s
procedures and management structure for these eight areas were among the
best of all the federal agencies we have evaluated using this model. This
provides a good foundation for developing and implementing sound procedures
for the next phase of the project; however, in many ways, the challenges
will be more difficult.
As the project moves from the planning phase to the procurement phase, the
Coast Guard must ensure that it can perform project management and contract
administration activities at a high level, given the complexity and scope of
the contract and its uniqueness. Under the Coast Guard?s planned approach,
the systems integrator will be responsible for program management required
to implement the deepwater system, and the Coast Guard will continuously
monitor the integrator?s performance. The Coast Guard plans to implement, or
require the systems integrator to implement, many management processes and
procedures based on best practices, but these practices are not yet in
place. Because much work remains to be accomplished in this area, the full
effectiveness of the Coast Guard?s approach cannot be assessed in the short
term. The following are the key areas that will need to be addressed.
26 Carnegie Mellon University?s Software Engineering Institute?s Software
Acquisition Capability Maturity Model. ®The Software Engineering Institute
is a federally funded research and development center sponsored by the U. S.
Department of Defense under contract to Carnegie Mellon University.
27 Two of the 112 practices involved not having a plan for developing and
managing project requirements and lack of specific measures to assess how
requirements are being managed. In both cases, the weaknesses were not
significant given the other strengths and compensating controls in place.
Coast Guard Faces
Difficult Challenges During the Procurement Phase
Page 24 GAO- 01- 564 Deepwater Project
A critical element to the ultimate success of the project is having enough
trained and knowledgeable Coast Guard staff to conduct management and
oversight responsibilities. Project officials view this as a high- risk area
and one of the most important aspects of the project. The Coast Guard hopes
to have its full complement of staff needed for fiscal year 2002 by the time
the contract is awarded. Currently, the Coast Guard has 69 personnel devoted
to the Deepwater Project. According to project officials, the current
project staff is highly qualified- most have advanced degrees in management,
engineering- related, and other specialty fields. Moreover, the Coast Guard
has made a conscious effort to maintain the continuity of its project staff
by not rotating its military personnel on the project to new positions every
4 years as it normally does. In addition, the Coast Guard has assigned a
Project Executive Officer to head the project.
Project officials have identified the need for 62 additional positions to
manage the project beginning in fiscal year 2002. In addition, the officials
plan to hire civilians with acquisition and contracting experience and to
use support contractors for many activities. The Coast Guard is also in the
process of developing a training plan for its project staff; it hopes to
complete the plan later this year and ensure that all staff meet the
training requirements for their respective positions by the time the
contract is awarded.
Under its deepwater acquisition approach, the Coast Guard will rely heavily
on the systems integrator to establish a management organization and systems
necessary to manage the major subcontracts for deepwater equipment. The
systems integrator will be required to apply an integrated product and
process development approach, 28 including teams consisting of Coast Guard,
contractor, and major subcontractor personnel who are responsible for
specific areas of the program. Also, the systems integrator will be
responsible for developing key systems and processes, such as risk
28 An integrated product and process development approach, using integrated
project teams, is a best practice for managing major acquisition programs
cited in OMB?s Capital Programming Guide (Supplement to OMB Circular No. A-
11), Part 3: Planning, Budgeting, and Acquisition of Capital Assets.
Effective Human Capital
Practices Key Management and Oversight Processes and Procedures
Page 25 GAO- 01- 564 Deepwater Project
management, quality assurance, test and evaluation, and earned- value
management systems. 29
In addition, the Coast Guard is developing a program management plan to
oversee the systems integrator. The major components of this plan are
project planning; organization; and detailed planning documents, including
individual plans for contract management, information management, and
financial management. Although the Coast Guard plans to complete the program
management plan before the contract is awarded, project officials told us
that some individual plans, such as configuration and integrated logistics
support plans, are dependent upon the system selected and cannot be
finalized until after the contract is awarded.
Because the use of major subcontractors to provide high- value equipment
will be such an intricate part of the Deepwater Project, good relations and
communications between the Coast Guard, the systems integrator, and the
major subcontractors will be very important. Our past review of best
practices on this issue suggests that leading organizations establish
effective communications and feedback systems with their subcontractors to
continually assess and improve both their own and supplier performance. 30
These practices not only helped key subcontractors to fully understand the
firms? goals, priorities, and performance assessments, they also helped the
firms to understand subcontractors? ideas and concerns. Our experience in
evaluating DOD acquisition programs showed that it was important to
establish such relationships not only with prime contractors but with
subcontractors as well. For example, supplier relationships on one program
we reviewed reflected DOD?s traditional role of distancing itself from
subcontractors. This role was traced, in part, to the fact that DOD had not
articulated a particular subcontractor policy to guide program managers. We
recommended- and DOD agreed- that DOD establish a policy and incorporate it
into acquisition plans for major procurements.
29 Earned value is a management technique that relates resource planning to
schedules and to technical, cost, and schedule requirements. All work is
planned, budgeted, and scheduled in time- phased ?planned value? increments
constituting a cost and schedule measurement baseline. As work is performed,
it is measured against planned goals, and any difference is called a
variance.
30 Best Practices: DOD Can Help Suppliers Contribute More to Weapon System
Programs (GAO/ NSIAD- 98- 87, Mar. 17, 1998). Close Relationships With
Subcontractors
Page 26 GAO- 01- 564 Deepwater Project
The Coast Guard has developed no general policy on subcontractor
relationships. Major subcontractors will be part of the integrated product
and process development teams, and the Coast Guard plans to perform quality
assurance activities at subcontractors? facilities. However, according to
project officials, the program management and quality assurance plans have
not been completed, and it is not clear at this time what the quality and
nature of the Coast Guard?s relationship with subcontractors will be.
OMB Circular A- 11, Part 3, emphasizes that each useful segment (e. g. an
entire ship or an entire aircraft) of a capital project should be fully
funded in advance of incurring obligations. The Coast Guard has told its
contractors to develop their deepwater schedules by using full funding of
useful segments rather than incremental funding. Coast Guard contracting
officials have said that they plan to obtain full funding for a ship or
aircraft before proceeding with their procurement. However, if deepwater
plans need to be adjusted due to a shortfall in funding or changes in
program requirements, according to the officials, one option could be to
develop requests that fund only part of a ship or aircraft. We found in a
review of earlier Coast Guard budget justifications that the Coast Guard had
proceeded with some capital projects before the amount of full funding was
obtained. 31 According to OMB, proceeding with such incremental funding
could result in schedule delays or higher costs for capital projects. As the
Coast Guard proceeds with the Deepwater Project, it should ensure that its
budget requests are consistent with OMB guidelines on full funding of useful
segments to avoid attendant delays and increased costs.
The Coast Guard plans to award follow- on, award- term contracts on the
basis of factors such as improving operational effectiveness and minimizing
total ownership costs. To measure the performance of the systems integrator
in achieving these goals (as a basis for awarding the follow- on contracts),
the Coast Guard will use a simulation model to measure improvements in
operational effectiveness and will compare the contractor?s actual cost
reductions with their proposed costs. According to Coast Guard officials,
they will develop a new baseline for these factors on the basis of the
winning contractor?s plans and the most current
31 Budget Issues: Incremental Funding of Capital Asset Acquisitions (GAO-
01- 432R, Feb. 26, 2001). Full Funding in Advance of
Buying Equipment Accurate and Complete Data to Measure Contractor
Performance
Page 27 GAO- 01- 564 Deepwater Project
information on deepwater equipment after the contract is awarded in early
2002. 32
Coast Guard officials told us that they plan to use a subjective rating
system to assess the contractor?s performance rather than use database
benchmarks for improvements in operational effectiveness and total ownership
costs. According to Coast Guard officials, setting such benchmarks may be
difficult because performance data may reflect factors that did not result
from actions of the contractor. For example, improved intelligence on drug
smugglers could result in improvements in operational effectiveness. Also,
changes in fuel costs could cause operational costs to increase. Because a
host of factors could cause changes in these data, it will be important for
the Coast Guard to carefully track these measures and accurately identify
and segregate reasons for the changes that occur. Doing so would better show
the results of significant federal investments in ships and aircraft.
Given the Coast Guard?s heavy reliance on a single systems integrator for so
many facets of the Deepwater Project, the agency is at serious risk if- for
whatever reason- the systems integrator does not perform as expected or
decides to walk away from the project on its own. For example, if the
systems integrator?s performance falls short of expectations, the Coast
Guard will face numerous options, ranging from closer oversight to
termination of the contract. Faced with these options, having a carefully
thought- out contingency plan, which identifies and analyzes the implication
of potential actions, would solidify the Coast Guard?s ability to respond
effectively. Several high- level federal contracting officials echoed this
position, saying that given the circumstances for this particular project,
exit strategies and other means to deal with potential poor performance by
the systems integrator were highly desirable.
In the extreme case- where the contractual relation with the systems
integrator is terminated- an exit strategy identifying possible
alternatives, consequences, and transition issues would be important. In
this regard, contracting officials with the project told us that the
contract will provide
32 The current baselines for operational effectiveness and total ownership
costs are based on 1998 data. Since 1998, the Coast Guard has received funds
to buy additional ships and aircraft that are involved in deepwater
missions, and these need to be considered in developing a baseline for
operational effectiveness. In addition, it has more current cost data (e.
g., rising fuel costs) that need to be included in total ownership cost
estimates. Contingency Planning and Exit
Strategies
Page 28 GAO- 01- 564 Deepwater Project
several ?off- ramps? and that the Coast Guard has basically two options if
it were to terminate the systems integrator: (1) obtain a new systems
integrator and a new set of subcontractors as well and (2) revert to the
traditional ?stovepipe? procurement approach of procuring a single class of
vessels and aircraft at a time. These officials said that from a project
management standpoint, having a strategy to deal with options like these is
important; and the agency is currently documenting, with assistance from a
contractor, the pros and cons of each exit strategy. However, the officials
noted that specific, detailed plans to implement the options would not be
developed until it was known that the Coast Guard planned to terminate the
contract.
The risks associated with incorporating new unproven technology 33 into the
first part of the Deepwater Project are minimal, in part, because of the
Coast Guard?s emphasis that industry teams use technology that has already
been proven in similar applications. Our main concern is the absence of
criteria to measure the risk of the new technology that needs to be
developed, both now and in the project?s later years.
Too little assessment of the risks associated with developing new technology
has caused problems on many acquisition projects, both in government and the
private sector. OMB?s Capital Programming Guide (A11) states, ?Probably the
greatest risk factor to successful contract performance is the amount of
technology development that is planned for the procurement.? Minimizing a
technology?s unknowns and demonstrating that it can function as expected
significantly reduce such risk. We have found that leading commercial
companies use disciplined processes to demonstrate- before fully committing
to product engineering and development- that technological capability
matches project requirements. Waiting to resolve these problems can greatly
increase project costs- at least 10- fold if the problems are not resolved
until
33 For this report, we are using the term technology to denote assets,
systems, equipment, and components proposed for the Deepwater Project. Use
of Off- the- Shelf
Technology Minimizes Risks, but Effective Means to Assess Unproven
Technology Is Lacking
Coast Guard?s Approach Conforms With Best Practices for Technology
Development
Page 29 GAO- 01- 564 Deepwater Project
product development, and as much as 100- fold if they are not resolved until
after production begins. 34
The Coast Guard has taken steps to minimize these risks. One major step was
to emphasize in contracting documents to industry teams that, to the maximum
extent possible, proposed assets, systems, equipment, and components are to
be nondevelopmental or commercially available (offthe- shelf) items. Our
review showed that the teams? preliminary proposals included many commercial
off- the- shelf and nondevelopmental items currently operating in the
commercial or military environment. However, some proposed equipment
included developing technology that has not yet been proven. Generally,
these developing technologies are at the prototype level and are undergoing
performance testing and evaluation prior to contract award to commercial and
military customers.
The Coast Guard?s steps are helping to keep the risk of unproven nearterm
technology at a low level. We measured the maturity level for the project?s
most critical near- term technologies (those introduced in the first 7 years
of the project), using an approach developed by the National Aeronautical
and Space Administration (NASA). We applied this process, referred to as
technology readiness levels (TRL), to 18 technologies identified as critical
by the 3 contractor teams and the Coast Guard. We determined- and the Coast
Guard concurred- that by the time the contract is awarded, 16 of the 18 are
expected to be at a level of acceptable risk. 35 The remaining two
technologies will be slightly higher in risk; but in one case, an early
prototype is being tested, and in the other, a proven backup system has been
identified that, if needed, could replace the technology with no effect to
the project?s cost, schedule, or performance. Entering phase 2 of the
project with critical technologies at a high level of maturity or with
proven backup systems significantly
34 Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions (GAO/ T- NSIAD- 00- 137, Apr. 26, 2000). 35 TRL readiness
levels are measured on a scale of one to nine. Examples of the ratings are
as follows: a rating of one signifies that studies of the basic concept have
been done; a rating between three and six means that success has been
demonstrated to a degree in laboratory situations; and a rating of nine
means that the technology has been proven in operational mission conditions
and is in final form. To be considered acceptable for committing to a
contract award, a new technology or adopted system should be rated at seven
or higher. A rating of seven means that a system prototype has been
demonstrated in the operational environment.
Page 30 GAO- 01- 564 Deepwater Project
lowers risk and the likelihood of delays, which in turn helps to control
program costs.
Although technological risks appear minimal in the near term, the Coast
Guard lacks criteria for assessing the maturity of technology in the longer
term. 36 The Coast Guard has a risk- management plan in place, as well as a
process to identify, continuously monitor, and assess technology risks; and
the resources the Coast Guard expects to commit to the task during phase 2
appear to be adequate. What the process lacks, however, is uniform and
systematic criteria for judging the level of technology maturity and risk,
such as the TRL ratings in the approach we adopted from NASA. In contrast,
since January 2001, DOD has required the use of TRL criteria as a tool for
measuring the technology readiness of its procurement projects.
Such criteria are important for monitoring both continued development of the
technologies we examined and the development of other technologies that will
not be used until later in the project. As of July 2000, when we completed
our TRL assessment, half of the 18 deepwater key technologies we reviewed
were still below the maturity level considered an acceptable risk for
entering production. Before the contract is awarded, the Coast Guard must
assess the readiness of these technologies. In addition, the industry team
proposals include numerous technologies that are planned for deepwater
system introduction from 2009 to 2020- well after contract award. Many of
these future technologies will not be proven at contract award and will need
to be assessed for technology risk before acceptance. The Coast Guard plans
to have a test and evaluation master plan in place by June 2001, but it is
not planning to include a requirement for using TRL criteria to measure
technology readiness in that plan.
The Coast Guard?s acquisition strategy and contracting approach for the
Deepwater Project are innovative. The agency plans to use the full
flexibility provided by congressional reforms of the federal acquisition
process to avoid the all too frequent failures of major federal acquisitions
in the past. Despite the numerous commendable innovations during the concept
development phase, we remain concerned that considerable risks
36 According to the Coast Guard, the draft RFP includes a requirement for
assessing the maturity of technology at the time of delivery. Criteria to
Adequately
Monitor and Assess Technology Maturity Are Lacking
Conclusions
Page 31 GAO- 01- 564 Deepwater Project
remain with its chosen approach for the procurement phase of the
acquisition.
The Coast Guard?s contracting approach for the production phase of the
deepwater acquisition is unique- relying on a single contractor to manage,
build and integrate the modernization of its entire deepwater fleet over a
period likely to exceed over 2 decades. The key promise of the approach is
achievement of a fully integrated system that both maximizes improvements in
operational effectiveness and minimizes total ownership costs (including not
only the acquisition, but operation, maintenance, and support costs of the
entire system over a 40- year period).
While we recognize the merit of exploring innovative and even unique
approaches, we believe the selected approach puts at risk precisely the
purported benefits of the approach- that is, maximizing operational
effectiveness and minimizing operational costs. Development of this unique
and untried approach on such a large scale and for an acquisition so
critical to the Coast Guard?s ability to perform every aspect of its
deepwater mission puts a heavy burden on the Coast Guard. Not only would it
be reasonable to expect a rigorous effort to identify and mitigate all the
major potential risks associated with a totally new approach, but the Coast
Guard would also need to ensure that other approaches were fully evaluated.
Unfortunately, we found that the Coast Guard has yet to accomplish either.
At our urging, the Coast Guard has only recently sought to set up a
systematic effort to identify and mitigate risks associated with its chosen
approach and the evaluation of alternative approaches remains limited and
poorly documented.
We remain concerned that the Coast Guard will soon be making critical
decisions regarding the Deepwater Project, namely issuing the RFP in less
than 2 months and awarding a contract to procure deepwater equipment in less
than a year. Yet, significant risks still exist, and the Coast Guard has not
completed actions to fully address them. The unique contracting approach of
relying on a single systems integrator to manage, acquire, and integrate all
Deepwater assets and capabilities poses two major risks, both of which still
remain. First, the agency?s choice of a contracting approach is now
inextricably tied to a projected deepwater funding level of over $500
million annually for the next 2 to 3 decades. Attaining sustained funding
for the project at this level looms as the major potential problem. By
choosing to proceed with a funding scenario that appears to be
unrealistically high in the face of budget projections that are
substantially less, the Coast Guard is increasing the risk that the project
will incur future cost increases and schedule stretch- outs. Second, the
Coast Guard?s
Page 32 GAO- 01- 564 Deepwater Project
reliance on a single contracting team raises serious questions regarding the
Coast Guard?s ability to control costs and ensure performance once the
contract is awarded. Their strategy for adequately controlling costs in the
project?s later years is still being worked out and requires careful
attention before the RFP is issued. Similarly, the Coast Guard is still
developing plans for managing the contract, and much remains to be done.
These are risks that need to be well understood and resolved before the RFP
is issued. Moving ahead before addressing the major risks and evaluating
options for addressing them, potentially including an evaluation of
alternative approaches, would be unwise.
The Coast Guard?s acquisition approach for the Deepwater Project- and its
reliance on a large and sustained funding level over a long period- makes
the Congress? next decision on the project crucial as well. This decision
goes far beyond deciding what Coast Guard equipment needs to be replaced or
modernized. The Congress is in effect being asked to provide the first
installment based on the Coast Guard?s spending plan for the project, which
is essentially dependent on a continuous funding stream in excess of $500
million annually for decades. Allowing the project to proceed as planned and
then later reducing that funding level significantly would result in higher
system costs and reduced system performance. We think this is the central
risk posed by the current approach, and that the Congress needs to make its
decision about providing funding for the project this year with clear
knowledge that the Coast Guard?s chosen contracting strategy depends heavily
on a sustained high level of funding for at least the next 20 years.
We recommend that before the Coast Guard issues the RFP for the Deepwater
Project, the Secretary of Transportation should
ensure that a realistic level of funding, based on OMB budget targets, the
Coast Guard?s capital planning process, and congressional guidance is
incorporated into the RFP and used by contractors as the basis for designing
their proposed systems; and
direct the Commandant of the Coast Guard to carefully consider and
incorporate recommendations, if any, made by the peer review panel into the
deepwater acquisition plan and RFP or if the peer review panel finds serious
and unmitigated risks in the Coast Guard?s approach, evaluate alternative
contracting strategies that could address the risks.
Before the Coast Guard signs a contract with the systems integrator for the
Deepwater Project, we recommend that the Secretary of Recommendations for
Executive Action
Page 33 GAO- 01- 564 Deepwater Project
Transportation should direct the Commandant of the Coast Guard to address
the following issues:
complete development of the Program Management Plan, including plans and
procedures to (1) facilitate relations with subcontractors, (2) ensure that
the project is adequately staffed and that the staff is properly trained to
perform their respective project management responsibilities, and (3) cover
actions to be taken in the event that the Coast Guard decides not to
continue its contract with the systems integrator;
complete plans for ensuring that annual budget requests for the Deepwater
Project are for useful segments and that a mechanism is in place for
reporting to OMB and the Congress, as part of its annual budget submission,
the progress that is made in achieving baseline goals of minimizing costs
and improving operations due to investments in funding the Deepwater
Project; and
select a process, such as the technology readiness levels approach, for
assessing the technology readiness of equipment and major systems to be
delivered.
The success of the contracting approach the Coast Guard selected for the
Deepwater Project relies heavily on the Coast Guard being able to sustain
the funding level around which the contractor?s proposal is based.
Substantial and prolonged funding below that level will lead not only to
cost increases and schedule slippages, but also to situations in which the
Coast Guard?s ability to achieve its missions may be jeopardized. To avoid
these situations, the Congress should have the opportunity to weigh in on
the affordability of the project before the contract is awarded. Therefore,
the Congress may wish to direct the Secretary of Transportation to (1)
ensure that any funding scenario included in the RFP is based on OMB budget
targets as well as discussions with appropriate congressional committees,
(2) submit a report to Congress justifying the funding scenario and
explaining any variations from the funding projections of OMB, and (3) wait
30 calendar days from submission of the report before issuing the RFP.
We provided a draft of this report to the Department of Transportation and
the Office of Management and Budget for their review and comment.
In commenting on our draft report, DOT disagreed with our recommendation to
incorporate more realistic levels of funding for the project into the RFP
based on OMB?s budget targets. In support of its position, DOT noted that
OMB out- year funding targets have been Matters for
Congressional Consideration
Agency Comments and Our Evaluation
Page 34 GAO- 01- 564 Deepwater Project
converging with estimated project requirements during the last year, and it
believes that OMB targets will change in the future to better match project
requirements of $500 million annually. DOT?s position in this regard is
counter to good capital planning and OMB guidance that says that agencies
should plan projects within available funding levels. As noted in the
report, the Coast Guard faces the real possibility of a cumulative funding
shortfall of almost half a billion dollars, or over 20 percent of the total
funding needs for the project?s first 5 years. Ultimately, by the Coast
Guard?s own admission, funding levels significantly below project
requirements would most likely lead to cost increases and schedule slippages
and jeopardize the agency?s ability to achieve its missions.
DOT agreed with two recommendations and did not comment on two others. The
agency agreed to evaluate and incorporate into the RFP as appropriate
recommendations from the peer review panel on its contracting approach for
the Deepwater Project. Also, the agency agreed to complete development of
the Program Management Plan prior to awarding the contract for phase 2. DOT
had no comment on two other recommendations, which focused on (1) ensuring
that its annual budget requests are for useful segments and that a mechanism
is in place for reporting to OMB and the Congress on the progress in
achieving baseline project goals and (2) selecting a process for assessing
the technology readiness of equipment and major systems to be delivered.
DOT?s written comments and our response are in appendix V.
We met with officials from OMB, including the Chief, Transportation Branch.
OMB concurred with our recommendations but believed that additional actions
may be warranted. OMB has concerns about the deepwater acquisition strategy
and believes that a broader evaluation of alternative strategies is needed.
The agency indicated that the Coast Guard has chosen an approach that relies
on a required funding level each year, and OMB has the same concerns that we
do about the potential impact on the project if funding does not materialize
as expected. OMB is also concerned that this approach sets up a situation
where the administration and the Congress would have to fund the project in
later years at the planned level, regardless of other competing priorities.
Essentially, OMB believes that the deepwater funding strategy transfers the
risk of program failure to external sources, such as the Congress. According
to OMB, future funding levels cannot be guaranteed, and it would be
inappropriate for the Coast Guard to use funding levels in the RFP that are
not consistent with OMB?s targets. Under the current acquisition approach,
if sustained funding is substantially less than planned, the Coast Guard
would have to rebaseline the project in a sole- source environment, a
Page 35 GAO- 01- 564 Deepwater Project
situation that could increase project costs even further. Finally, OMB
raised these concerns at the peer review panel meeting; however, OMB is not
optimistic that the Coast Guard will sufficiently recognize and adequately
address its concerns prior to issuing the RFP. We generally share OMB?s
concerns and have made many of the same points throughout our report.
We plan to provide copies of this report to the Honorable Norman Y. Mineta,
Secretary of Transportation; Admiral James M. Loy, Commandant of the Coast
Guard; and the Honorable Mitchell E. Daniels, Jr., Director, Office of
Management and Budget. We will also send copies to others upon request.
If you or your staff have any questions about this report, please call me at
(202) 512- 2834 or Randall Williamson at (206) 287- 4860. Other key
contributors to this report are listed in appendix VI.
JayEtta Z. Hecker Director, Physical Infrastructure Issues
Appendix I: Objectives, Scope, and Methodology
Page 36 GAO- 01- 564 Deepwater Project
This report examined the major risks associated with the Deepwater Project
and the progress the Coast Guard has made in addressing them. Our work
focused on four risks: (1) planning the project around annual funding levels
far above what the administration has told the Coast Guard it can expect to
receive, (2) keeping costs under control in the contract?s later years, (3)
ensuring that procedures and personnel are in place for managing and
overseeing the contractor once the contract is awarded, and (4) minimizing
potential with developing unproven technology.
To assess the funding risk, we reviewed OMB Circular No. A- 11, Part 3
(Planning, Budgeting, and Acquisition of Capital Assets); OMB?s Capital
Programming Guide, the Coast Guard?s 5- year capital plan; and the agency?s
past budget requests for capital projects. We also reviewed various
deepwater planning documents, including the risk management plan, the draft
acquisition plan, and requests for proposal. We reviewed prior Coast Guard
appropriations and pertinent laws affecting the Coast Guard?s budget. We
also reviewed DOT Inspector General reports on the Deepwater Project. We
interviewed DOT and Coast Guard officials involved in forming the Coast
Guard?s budget, including the Coast Guard?s Director of Resources. We also
interviewed OMB budget officials and officials from the Office of Federal
Procurement Policy.
To assess the risks of controlling costs, we reviewed the Federal
Acquisition Regulation; OMB Circular A- 11, Part 3, (Planning, Budgeting,
and Acquisition of Capital Assets); and OMB?s Best Practices for Multiple
Award Task and Delivery Order Contracting. We reviewed the Coast Guard?s
draft acquisition plan and RFP for the deepwater phase 2 contract, reports
that the Coast Guard received from consultants it hired to evaluate its
acquisition strategy, and numerous Coast Guard documents regarding how the
agency planned to acquire deepwater assets. We interviewed numerous
contracting officials, including, Coast Guard contracting officials,
officials from OMB?s Office of Federal Procurement Policy, the former Deputy
Under Secretary of Defense for Acquisition Reform, the Deputy Director for
Defense Procurement, the Chief of the Internal Revenue Service?s Contracting
Branch, and an official of a private consulting firm. We also drew from our
extensive agencywide contracting experience in reviewing DOD and other
agency procurements.
To determine the risk involved in managing the contract, we assessed the
Coast Guard?s project management during the planning phase of the Deepwater
Project and identified challenges the Coast Guard will face during the
procurement phase. To identify best practices in contract management and
administration, we reviewed OMB Circular No. A- 11, Part Appendix I:
Objectives, Scope, and
Methodology
Appendix I: Objectives, Scope, and Methodology
Page 37 GAO- 01- 564 Deepwater Project
3, and drew from our extensive agencywide contacting experience in reviewing
DOD and other agency procurements. We reviewed the Coast Guard?s Project
Management Plan, Risk Management Plan, and other management plans to
identify the Deepwater Project?s organizational structure and key management
procedures used during the planning phase. We assessed the effectiveness of
these procedures and structure using Carnegie Mellon University?s Software
Engineering Institute?s Software Acquisition Capability Maturity Model® and
its Software Capability Evaluation method. Although the model is
specifically designed to determine software acquisition process maturity,
its application can be used for the acquisition of any type of asset (ships,
aircraft, etc.). The model ranks organizational maturity according to five
levels. Maturity levels 2 through 5 require the verifiable existence and use
of certain acquisition processes, known as key process areas. Satisfying the
requirements of maturity level 2 demonstrates that an organization has the
policies needed to manage a project and the procedures needed to implement
those policies. We evaluated the acquisition processes for two Deepwater
Project matrix product teams against all seven level- 2 areas (planning,
solicitation, requirements development and management, project management,
contracting and oversight, evaluation, and transition and support) and one
level- 3 area (risk management). Within these 8 key process areas, we
examined 112 key practices to determine their strengths or weaknesses. We
reviewed the Coast Guard?s draft acquisition plan and RFP for the Deepwater
Project phase 2, comments that the Coast Guard received from the consultants
it hired to evaluate its acquisition strategy, and other documents to
identify how the agency plans to manage and administer the procurement phase
of the Deepwater Project. We discussed these management plans with Coast
Guard contract and Deepwater Project officials, the DOD Deputy Director for
Defense Procurement, the Chief of the Internal Revenue Service?s Contracting
Branch, and a representative from a private consulting firm.
To assess the risk of using new technologies, we asked each of the three
competing deepwater contracting teams to first develop a list of the most
critical technology and keystone systems being proposed as ?near- term?
deepwater contract deliverables to be introduced during the first 7 years
(2002 through 2008) after contract award. Eighteen technologies and systems
were identified, including assets and components representing deepwater
aviation, surface, and command, control, communications, computers,
intelligence, surveillance, and reconnaissance concept solutions. We then
asked the contracting teams to assess ?technology
readiness? for each of the items they identified on their lists using NASA?s
technology readiness level (TRL) criteria. TRLs provide a standard
Appendix I: Objectives, Scope, and Methodology
Page 38 GAO- 01- 564 Deepwater Project
definition of nine levels of technology maturity that can be used to measure
technology readiness, regarding the type of demonstration that must be
achieved; the scale (form, fit, and function) of the asset; and the
operational environment in which demonstration is performed. We asked the
contracting teams to score technology readiness at three points in the
deepwater acquisition process- July 2000, April 2001, and January 2002. We
focused our analysis on the technology readiness level at the date of
contract award, January 2002. We independently met with program managers
from each of the three industry teams to discuss the status of each
technology/ keystone system, identify the rationale for the initial TRL
score assessment, and determine whether adjustments in the TRL score were
necessary. On the basis of these discussions, we made adjustments to the
initial TRL scores that the competing contractors agreed were consistent
with the TRL criteria. We then crosswalked the TRL scores to project risk
criteria established by the Air Force Research Laboratory that predicts
project risk on the basis of technology readiness at program decision
points. Specifically, the Laboratory established that a technology/ key
system should be at TRL 7 at the time the decision is made for a program to
enter the Engineering and Manufacturing Development Phase- a phase we
believe is comparable to the January 2002 deepwater contract award for
?near- term? technology and keystone systems.
Appendix II: Current Deepwater Cutters and Aircraft
Page 39 GAO- 01- 564 Deepwater Project
Type of asset Number Description Cutters
378- foot high- endurance cutter 12 This is the largest multipurpose cutter
in the fleet. It has a planned crew size of 167, a speed of 29 knots, and a
cruising range of 14, 000 nautical miles. The Coast Guard operates each
cutter for about 185 days a year, and it can support helicopter operations.
270- foot medium- endurance cutter 13 This cutter has a planned crew size of
100, a speed of 19.5 knots, and a cruising
range of 10, 250 nautical miles. The Coast Guard operates each cutter for
about 185 days a year, and it can support helicopter operations. 210- foot
medium- endurance cutter 16 This cutter has a planned crew size of 75, a
speed of 18 knots, and a cruising
range of 6,100 nautical miles. The Coast Guard operates each cutter for
about 185 days a year, and it can support operations of short- range
recovery helicopters. 110- foot patrol boat 49 This patrol boat has a crew
size of 16, a speed of 29 knots, and a cruising range
of 3,928 nautical miles. The Coast Guard operates each for about 1,800 hours
a year. Mature class cutters (282- foot, 230- foot, and 213- foot cutters) 3
The 213- foot medium- endurance cutter, commissioned in 1944, has a planned
crew size of 64. The 230- foot medium- endurance cutter, commissioned in
1942, has a planned crew size of 106. The 282- foot medium- endurance
cutter, commissioned in 1971, has a planned crew size of 99.
Total 93 Aircraft
HC- 130 long- range surveillance airplane 30 This is the largest aircraft in
the Coast Guard?s fleet. It has a planned crew size
of seven, a speed of 290 knots, and an operating range of about 2,600
nautical miles. The Coast Guard operates each of these aircraft for about
800 hours every year. HU- 25 medium- range surveillance plane 41 This is the
fastest aircraft in the Coast Guard?s fleet. It has a planned crew size
of five, a speed of 410 knots, and an operating range of 2,045 nautical
miles. The Coast Guard operates each for about 800 hours a year. HH- 60J
medium- range recovery helicopter 42 This helicopter is capable of flying
300 nautical miles off shore, remaining on
scene for 45 minutes, hoisting six people on board, and returning to its
point of origin. The Coast Guard operates each for about 700 hours a year.
It has a planned crew size of four, a maximum speed of 160 knots, and a
maximum range of 700 nautical miles. HH- 65 short- range recovery helicopter
93 This helicopter is capable of flying 150 nautical miles off shore. It has
a crew
allowance of three, a maximum speed of 165 knots, a maximum range of 400
nautical miles, and a maximum endurance of 3.5 hours. The Coast Guard
operates each for about 645 hours a year.
Total 206
Source: U. S. Coast Guard.
Appendix II: Current Deepwater Cutters and Aircraft
Appendix III: Estimated Service Life of the Coast Guard?s Deepwater Cutters
and Aircraft
Page 40 GAO- 01- 564 Deepwater Project
In our 1998 study on the Deepwater Project, we found that the Coast Guard
had substantially understated the service life of its aircraft and, to a
lesser extent, its ships. For example, in its project justification prepared
in 1995, the Coast Guard estimated that its aircraft would need to be phased
out starting in 1998. However, in 1998, a Coast Guard study showed that with
proper maintenance and upgrades, its aircraft would be capable of operating
until at least 2012 and beyond. 1 Also, a September 1999 study revised
earlier estimates and concluded that the Coast Guard?s deepwater cutters
have a service life until 2007 and beyond, assuming that adequate funds
remain available for maintenance support and service life upgrades. 2 Shown
below are the differences in the service life of its deepwater ships
and aircraft between the initial estimates (in the1995 justification) and
later studies.
Table 3: Estimated Service Life of Deepwater Assets Type of asset
Years Acquired
Service life shown in the initial justification
Service life shown in recent studies Ship
378- foot high- endurance cutter 1967- 1972 2003- 2007 2008- 2012 270- foot
medium- endurance cutter 1983-
1991 2012- 2020 2013- 2021 210- foot medium- endurance cutter 1964-
1969 2002- 2011 2007- 2018 110- foot patrol boat 1986-
1992 2005- 2012 2000- 2012
Aircraft a HC- 130 long- range surveillance airplane 1972-
1987 1998- 2008 2012- 2027 HU- 25 medium- range surveillance airplane 1982-
1984 2002- 2004 2030- 2032 HH- 60J medium- range recovery helicopter 1990-
1996 2010- 2015 2023- 2030 HH- 65 short- range recovery helicopter 1984-
1989 2005- 2009 2015- 2019
1 Aviation Near- Term Support Strategy, Office of Aeronautical Engineering,
U. S. Coast Guard, (updated Oct. 28, 1998). 2 USCG WHEC/ WMEC Fleet
Condition and Remaining Service Life Study, Nichols Advanced Marine, (Sept.
30, 1999). Appendix III: Estimated Service Life of the
Coast Guard?s Deepwater Cutters and Aircraft
Appendix III: Estimated Service Life of the Coast Guard?s Deepwater Cutters
and Aircraft
Page 41 GAO- 01- 564 Deepwater Project
a The service life estimates shown in recent studies for aircraft are based
on investing $150 million for upgrades. Sources: The Coast Guard?s 1995
Deepwater Mission Analysis Report, the Coast Guard?s 2000 Deepwater Mission:
Current and Projected Requirements and Capabilities Report, and the U. S.
Coast Guard?s WPB Fleet Condition and Remaining Service Life Study (Jan. 31,
2001).
Appendix IV: List of the Members of Each Contracting Team
Page 42 GAO- 01- 564 Deepwater Project
During Phase 1 of the Deepwater Project 1
Litton/ Avondale Industries (Systems Integrator) Boeing- McDonnell Douglas
Corporation John J. McMullen & Associates, Inc. DAI, Inc. Raytheon Systems
Company
Lockheed Martin Naval Electronics and Surveillance Systems
(Systems Integrator) Lockheed Martin Aeronautical Systems Lockheed Martin
Electronics Platform Integration - Oswego, NY Lockheed Martin Global
Telecommunications Lockheed Martin Management and Data Systems Sanders, A
Lockheed Martin Company Litton/ Ingalls Shipbuilding Litton/ PRC M.
Rosenblatt & Son Bell Helicopter Textron, Inc. Halter- Bollinger Joint
Venture, L. L. C. Acquisition Logistics Engineering L- 3 Communications East
PROSOFT Whitney, Bradley and Brown, Inc.
Science Applications International Corporation (Systems Integrator)
Marinette Marine Corporation Sikorsky Aircraft Corporation Soza & Company,
Ltd. Bath Iron Works AMSEC Fuentez Systems Concepts, Inc. Gibbs & Cox, Inc.
1 In keeping with the acquisition strategy delineated in the phase 1
proposal, the Center for Naval Analyses, and later Booz Allen & Hamilton,
were selected to serve as the Independent Analysis Government Contractors
(IAGC). The IAGC, like the industry teams, performed an analysis of
deepwater equipment and the deepwater operating environment and developed an
integrated deepwater system concept. Unlike the industry teams, however, the
IAGC is prohibited from participating in the actual production; therefore,
it is uniquely positioned to perform an impartial and objective analysis of
relevant technologies, platforms, and implementation strategies and
scenarios. Appendix IV: List of the Members of Each
Contracting Team
Appendix IV: List of the Members of Each Contracting Team
Page 43 GAO- 01- 564 Deepwater Project
Clark Atlanta University Rockwell Collins
Appendix V: Comments From the Department of Transportation
Page 44 GAO- 01- 564 Deepwater Project
Appendix V: Comments From the Department of Transportation
Note: GAO comments supplementing those in the report text appear at the end
of this appendix.
See comment 3. See comment 2
See comment 1.
Appendix V: Comments From the Department of Transportation
Page 45 GAO- 01- 564 Deepwater Project
See comment 6. See comment 5.
See comment 4.
Appendix V: Comments From the Department of Transportation
Page 46 GAO- 01- 564 Deepwater Project
See comment 8. See comment 7.
Appendix V: Comments From the Department of Transportation
Page 47 GAO- 01- 564 Deepwater Project
The following are GAO?s comments on DOT?s letter dated April 19, 2001. 1.
Our report notes that the Coast Guard took many innovative steps and
recognizes that the agency?s procedures and management structure for the
planning phase of the Deepwater project were excellent. While its management
during the planning phase provides a solid foundation for the project, the
acquisition phase presents considerably tougher challenges. By almost
everyone?s assessment, the acquisition strategy is a high risk, untried
approach to procure deepwater assets. Whether the Coast Guard has adequately
addressed these risks will not be known for years to come. Furthermore, the
Coast Guard?s handling of remarks and suggestions made by members of the
peer review are largely unknown at this point.
2. The contracting approach lacks flexibility in several key areas. First,
it requires sustained funding at planned levels of more than $500 million
for 2 or more decades. Second, it offers no true means to ensure competition
for major components as a lever to minimize costs. Third, if planned funding
levels are not realized, it opens the door to added costs because the Coast
Guard would have to renegotiate costs and delivery dates- all in a sole
source environment. Finally, changing the systems integrator after the
contract is awarded- while doable- would likely be costly both in terms of
dollars and delays in the project.
3. The added dollars expected from the Western Hemisphere Drug Elimination
Act has allowed the budget targets to increase substantially from prior year
(fiscal year 2001) targets. However, OMB told us that had it not been for
the act, the large increase in the Coast Guard?s targets for capital
projects would have been difficult to achieve given the budgetary
environment. While targets may increase somewhat in future years, any large
increases would require new funding sources or shifts in funding from other
entities, such as FAA and Amtrak, which also have critical capital needs.
Already, the funding requirements for the project are almost half a billion
dollars more than OMB budget targets through 2006. Given the uncertainty of
future funding, it would be unwise and fiscally imprudent for the Coast
Guard to blindly proceed with an RFP that contains a planned funding stream
of $500 million, hoping that funding at planned levels will materialize
later. OMB echoed our position on this issue. GAO Comments
Appendix V: Comments From the Department of Transportation
Page 48 GAO- 01- 564 Deepwater Project
4. While the Coast Guard has the flexibility to alter project plans based on
reduced funding levels in future years, the Coast Guard would likely pay
dearly for this. The Coast Guard recognizes this but steadfastly opposes
including a lower, more realistic funding level in its RFP. It has
essentially rejected our concerns and those of OMB in this area and has
adopted a position that runs counter to sound ?best practices? for capital
planning that are based on widely- accepted OMB guidance.
5. The Coast Guard?s characterization of the peer review panel?s
deliberations and findings are overly optimistic and overstate the positive
results from the panel. Our review of the transcript of the panel?s
deliberations showed that there was not the unanimous consensus among panel
members on the efficacy of the acquisition approach that was portrayed by
the Coast Guard. For example, the panel member from the Office of Federal
Procurement Policy voiced numerous concerns about whether a thorough and
honest risk analysis of the acquisition approach had been done and whether
adequate mitigation and management plans are in place. Another member also
echoed this position, while another remarked that much work is needed before
the RFP should be issued to contractors. We believe that such concerns by
panel members do indeed refer to potentially serious and unmitigated risks
that should not be dismissed lightly. In addition, given that panel members
were not given the RFP or the acquisition plan prior to the panel meeting,
we question the thoroughness of the panel?s results and the depth to which
it explored key questions. It is evident from the transcript of the panel
discussions and our observations of the proceedings that panel members may
not have had a good grasp or understanding of many issues in the depth
necessary to make informed observations and suggestions. For example, one
panel member remarked in his summary at the end of the panel discussion that
there was an information void on some issues when the panel began
discussions, and having more detailed information on the acquisition
strategy ahead of time would have been useful. OMB officials who observed
the peer review session told us that they felt the same way. Moreover, the
panel members were not asked to determine whether this approach represents
the ?best
approach among all possible alternatives,? nor were panel members given the
time or the information necessary to make such a determination.
6. To provide funding for the Deepwater Project, the Coast Guard will likely
have to keep funding for other capital projects at levels which would be
substantially lower than levels experienced over the last
Appendix V: Comments From the Department of Transportation
Page 49 GAO- 01- 564 Deepwater Project
decade or more. It is unrealistic to believe that other non- deepwater
capital needs will be minimal for the entire duration of the Deepwater
Project. The DOT Office of Inspector General, for example, has recently
identified millions of dollars of potential capital projects associated with
the Coast Guard?s search and rescue program. Also, in its current fiscal
year 2002 capital plan, the Coast Guard may have significantly understated
amounts needed for information technology and other projects. For example,
the current plan projects information technology funding needs of only $3
million in 2005; its capital plan of just a year ago cited information
technology project needs of $31.4 million in 2005. Similarly, estimates of
funding needs for shore facilities were $128.8 million in the 2001 plan and
only $58.7 million in the fiscal year 2002 plan. Either the Coast Guard
grossly overstated its non- deepwater needs in the fiscal year 2001 plan or
it cut deeply into these projects for the fiscal year 2002 plan to
accommodate funding for the Deepwater Project. Regardless, this leaves
serious questions about whether the Coast Guard is understating funding
needs for nondeepwater projects to give the appearance that the Deepwater
Project funding needs can be met in the next 5 years.
7. While the Coast Guard has provisions in the RFP that allow it to exit the
contract if price or performance is unsatisfactory, the practical reality is
that changing the systems integrator will be costly, and there is a natural
reluctance for an agency to do so. Members of the peer review panel remarked
similarly on this issue. In addition, complete, reliable data on total
ownership costs and operational effectiveness may be absent, especially in
the project?s early years, making those measures less effective as a means
to evaluate contractor performance. Members of the peer review panel made
this point as well. Also, the inclusion of contract incentives does not
guarantee competition will exist among subcontractors. The panel did not
reach unanimous consensus that such incentives would necessarily be
effective in this regard, as the Coast Guard contends.
8. The Coast Guard did not comment on two recommendations that need to be
addressed. Developing an effective assessment tool to evaluate the
technology maturity of major equipment and components is critical to keep a
tight rein on costs. Also, ensuring that future budget requests for
deepwater components are for useful segments is essential. OMB strongly
concurred with our view on these issues. Finally, keeping the Congress
appraised of progress being made in achieving the baseline goals of
minimizing costs and improving operations is vital as a basis
Appendix V: Comments From the Department of Transportation
Page 50 GAO- 01- 564 Deepwater Project
for holding the Coast Guard accountable to the Congress and the
administration for the significant investment in the project.
Appendix VI: GAO Contacts and Staff Acknowledgements
Page 51 GAO- 01- 564 Deepwater Project
JayEtta Z. Hecker (202) 512- 2834 Randall B. Williamson (206) 287- 4860
In addition to those named above, Marie Ahearn, Neil Asaba, Naba Barkakati,
Alan Belkin, Christine Bonham, Sue Burns, John Christian, Tom Collis, Ralph
Dawn, Paul Francis, David Hooper, Richard Hung, Matt Lea, Sterling
Leibenguth, Lynn Musser, Madhav Panwar, Colleen Phillips, David Robinson,
Katherine Schinasi, Stanley Stenersen, Mike Sullivan, and William Woods made
key contributions to this report. Appendix VI: GAO Contacts and Staff
Acknowledgements GAO Contacts Acknowledgements
(348225)
The first copy of each GAO report is free. Additional copies of reports are
$2 each. A check or money order should be made out to the Superintendent of
Documents. VISA and MasterCard credit cards are also accepted.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail:
U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
Orders by visiting:
Room 1100 700 4 th St., NW (corner of 4 th and G Sts. NW) Washington, DC
20013
Orders by phone:
(202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.
Orders by Internet
For information on how to access GAO reports on the Internet, send an email
message with ?info? in the body to:
Info@ www. gao. gov or visit GAO?s World Wide Web home page at: http:// www.
gao. gov
Contact one:
Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
E- mail: fraudnet@ gao. gov
1- 800- 424- 5454 (automated answering system) Ordering Information
To Report Fraud, Waste, and Abuse in Federal Programs
*** End of document ***
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
