Coast Guard: Actions Needed to Mitigate Deepwater Project Risks
(03-MAY-01, GAO-01-659T).
The Coast Guard is in the final stages of planning the largest
procurement project in its history--the Deepwater Capability
Replacement Project. This project will likely cost over $10
billion or more and take 20 years or more to complete. The Coast
Guard has already spent about $116 million on the project's
design and is asking for $338 million this year to begin the
acquisition phase. This testimony discusses the major risks
associated with the project. GAO found four major areas in which
the project is vulnerable. They are (1) planning the project
around annual funding levels far above what the administration
has told the Coast Guard it can expect to receive, (2) keeping
costs under control in the contract's later years, (3) ensuring
that procedures and personnel are in place for managing and
overseeing the contractor once the contract is awarded, and (4)
minimizing potential problems with developing unproven
technology. GAO also identified the following key areas that will
need to be addressed. The Coast Guard needs (1) effective human
capital practices, (2) a systems integrator to establish a
management organization and systems necessary to manage the major
subcontracts for deepwater equipment; (3) to establish a general
policy on subcontractor relationships, (4) to ensure that its
budget requests are consistent with Office or Management and
Budget guidelines on full funding of useful segments, (5) to
implement a subjective rating system to measure contractor
performance and (6) to develop and implement a carefully
thought-out contingency plan which identifies and analyzes the
implication of potential actions should the systems integrator
not perform as expected or walk away from the project. This
testimony summarizes the May report, GAO-01-564.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-01-659T
ACCNO: A00948
TITLE: Coast Guard: Actions Needed to Mitigate Deepwater Project
Risks
DATE: 05/03/2001
SUBJECT: Aircraft
Contract costs
Cost control
Cost overruns
Federal procurement
Personnel recruiting
Risk management
Ships
Coast Guard Deepwater Capability
Replacement Project
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-659T
COAST GUARD Actions Needed to Mitigate Deepwater Project Risks Statement of
JayEtta Hecker, Director, Physical Infrastructure
United States General Accounting Office
GAO Testimony Before the Subcommittee on Coast Guard and Maritime
Transportation, Committee on Transportation and Infrastructure, House of
Representatives
For Release on Delivery Expected at 10: 00 a. m. EST Thursday May 3, 2001
GAO- 01- 659T
1
Mr. Chairman and Members of the Subcommittee: We appreciate the opportunity
to testify on the Deepwater Capability Replacement Project, which was
initiated by the Coast Guard to replace and modernize its aging fleet of
over 90 cutters and 200 aircraft used beyond 50 miles from shore. This
project, the largest acquisition ever attempted by the Coast Guard, will
likely cost over $10 billion or more and will not be completed for 2 to 3
decades. Already, the Coast Guard has spent about $116 million on the
project?s design, and this year is asking for $338 million to begin the
acquisition phase.
The Congress and the Coast Guard are now at a major crossroads with the
Deepwater Project, in that the planning is essentially complete, and the
Congress is now being asked to commit to a multibillion dollar project that
will define the way the Coast Guard performs many of its missions for
decades to come. The acquisition strategy the Coast Guard has chosen for the
Deepwater Project is unique and untried for a project of this magnitude. It
carries many risks that could potentially cause significant schedule delays
and cost increases.
Since 1998, we have reviewed numerous aspects of this project at the request
of this Subcommittee and others. Most recently, we evaluated the major risks
associated with the project, and our testimony today is based on a report 1
released earlier this week. We will discuss risks the project faces in four
major areas: (1) planning the project around annual funding levels far above
what the administration has told the Coast Guard it can expect to receive,
(2) keeping costs under control in the contract?s later years, (3) ensuring
that procedures and personnel are in place for managing and overseeing the
contractor once the contract is awarded, and (4) minimizing potential
problems with developing unproven technology.
1 Coast Guard: Progress Being Made on Deepwater Project, but Risks Remain
(GAO-01-564, May 2, 2001).
2
In summary: Affordability is a major risk for the Deepwater Project. The
Coast Guard has chosen a contracting approach that depends on a sustained
funding stream of over $500 million each year (in 1998 dollars) for the next
20 years or more. Contractors now competing for the right to acquire the
entire deepwater system have been told to plan their entire proposals around
this level of funding. According to budget projections from the Office of
Management and Budget (OMB), the Coast Guard faces the real possibility that
$500 million annually will not be available for the project and that the
cumulative budget shortfall may be as much as half a billion dollars for the
project?s first 5 years. The Coast Guard knows that any significant
shortfall could lead to dire consequences, including cost increases,
schedule stretch- outs, and degradation of system performance once the
contract is awarded. In addition,
?best practices? for capital planning strongly suggest that agencies should
plan capital projects within available funding levels. Despite these
factors, the Coast Guard plans to enter the acquisition phase, basing the
deepwater procurement around the $500 million funding stream.
The Coast Guard has selected a novel contracting approach- one never tried
on a contract this large. It calls for procuring ships, aircraft, and
equipment through a single, prime contractor. Before it was adopted, there
was little evidence that the Coast Guard had analyzed whether the approach
had any inherent difficulties for ensuring best value for the government
and, if so, what to do about them. We and others who are involved in
reviewing this approach, such as OMB, have expressed concerns with it,
particularly about the potential lack of competition during the project?s
later years and the reliance on a single contractor for procuring so much of
the deepwater equipment. In part, at our suggestion, the Coast Guard has
taken a number of steps to mitigate these concerns. Still, as the project
moves ever closer the acquisition phase, the
3
Coast Guard is conducting an analysis of its approach and has delayed some
of its key milestones to consider these concerns more fully.
In most respects, the Coast Guard has managed the planning phase very well.
In fact, the Coast Guard?s procedures and management structure thus far have
been among the best of federal agencies we have evaluated. It faces tougher
challenges during the acquisition phase, however, and much is left to do
before it is ready to move to the next phase. It still must recruit and
train enough staff to manage and oversee the contract, determine how to best
manage its relationship with its subcontractors, ensure that useful segments
are fully funded in advance of buying equipment, implement an effective
means to accurately measure the effects on operations and total system costs
as new equipment replaces existing ships and aircraft, and develop
contingency plans in the event that problems develop with the performance of
the prime contractor or subcontractors.
Our reviews of other acquisitions governmentwide show that reliance on
unproven technology is a frequent contributor to cost escalation, schedule
delays, and compromised performance standards. As with contract management,
the Coast Guard?s initial steps in countering this risk have been very good.
The Coast Guard has encouraged contractors to include off- theshelf
technology in their proposals. Our review of key technologies that
contractors are proposing for the first few years of the project showed that
almost all should be sufficiently mature by the time the contract is
awarded. However, there is less certainty in later years. The Coast Guard
needs a structured process for assessing and monitoring this risk. So far,
it has none.
Our overall assessment of the risk levels is shown in table 1. In the
report, we make several recommendations to help the Coast Guard and the
Congress improve the long- term success of the Deepwater Project.
4
Table 1: Areas of Risk and Overall Risk Levels for the Deepwater Project
Area of risk Risk level Reasons for assigning this level of risk
Attaining a stable, sustained funding level
High Several years of funding substantially below planned funding levels can
have adverse consequences for the acquisition strategy. Budget constraints
and other budget priorities threaten the Coast Guard?s ability to achieve
large, sustained increases in its budget for capital spending. Controlling
costs in the contract?s later years
Moderate to High
The risks center on the potential lack of future competition and reliance on
a single contractor to procure the entire system. The level of risk depends
on the effectiveness of provisions the Coast Guard designs and includes in
the contract to encourage or require subcontract competition and increase
its leverage in negotiating future contracts with the prime contractor.
Overseeing the acquisition
Moderate Although there are many uncertainties about contract management as
the Coast Guard increases the size of the administrative effort, the
commendable start and the ability to make specific changes lessen the degree
of risk in this area. Using unproven technology
Low to Moderate
The steps needed to mitigate this risk are relatively few and
straightforward. The lack of an assessment tool to measure technology
maturity poses short term (low) and long- term (moderate) risk.
Source: GAO analysis of risk areas.
As the Coast Guard has attempted to mitigate these risks during the
project?s planning phase, we have assisted agency officials and brought our
concerns to their attention as soon as possible to increase the opportunity
for useful exchange of information and, where necessary, timely corrective
action. To its credit, the Coast Guard has listened to us and made many
changes to improve the project and mitigate major areas of risk.
Nonetheless, the Coast Guard still has much left to do in this regard before
it proceeds into the acquisition phase.
Background
Many of the Coast Guard?s cutters were built in the 1960s, and many of the
aircraft in the 1970s and 1980s. Although these assets have been upgraded
since being acquired, they are aging and have serious performance and
support problems. Because of these problems, the Coast Guard began planning
for the modernization of its deepwater fleet in 1995.
The acquisition approach the Coast Guard chose for the Deepwater Project is
innovative. Rather than using the traditional approach of replacing an
individual class of ships or aircraft, the Coast Guard has adopted a
?system- of- systems?
5
approach intended to integrate ships, aircraft, sensors, and communication
links together as a system to accomplish mission objectives more
effectively. The Coast Guard expects this approach will both improve the
effectiveness of deepwater operations and reduce operating costs.
In 1998, the Coast Guard contracted with three competing teams of
contractors to conceive and begin designing a proposed deepwater system.
Each team is made up of aircraft manufacturers, shipbuilders, and
manufacturers of electronic, communication, and other equipment needed for
the deepwater system. Later this year, the Coast Guard will ask each team to
submit a final proposal, which the Coast Guard will evaluate as a basis for
selecting one team to build the entire system. The Coast Guard plans to
award the deepwater contract in early 2002, based largely on which team
proposes a system that provides the best value in terms of improvements in
operational effectiveness and minimizing total ownership costs. 2
When the deepwater contract is awarded in 2002, the contract will actually
be between the Coast Guard and a prime contractor, known as the ?systems
integrator,? of the winning team. The systems integrator will be responsible
for ensuring that each ship, aircraft, or other equipment is delivered on
time, in accordance with agreed to prices, and in compliance with the Coast
Guard?s performance specifications. Because each of the three system
integrators now competing for the contract is developing its proposal in
conjunction with its own team of companies, it is likely that the companies
in each team will supply most of the equipment. The deepwater contracting
approach could thus result in a longterm contractual arrangement and working
relationship with a single contractor and its team of contractors.
2 Operational effectiveness involves the Coast Guard?s ability to carry out
its deepwater missions. For example, it involves the number of lives saved,
the amount of drugs interdicted, and the number of illegal immigrants
interdicted. Total ownership costs include acquisition, operating,
maintenance, and support costs for the deepwater system over a 40-year
period.
6
Initially, the Coast Guard plans to have a 5- year contract with the systems
integrator. The systems integrator would receive a base award for management
and system integration services. Task and delivery orders for deepwater
equipment would be issued by the Coast Guard in accordance with the systems
integrator?s implementation schedule. If the performance of the systems
integrator is satisfactory for each award- term contract, the Coast Guard
plans to award follow- on, award- term contracts (as many as five for
successive 5- year award- term contracts) with the same systems integrator.
Viability of Contracting Approach Depends On a Sustained High Level of
Funding
Securing sustained funding for any major acquisition is difficult,
especially in the constrained budget environment that currently exists. In
the case of the Deepwater Project, two factors- locked in a collision
course- exacerbate the funding difficulties and jeopardize the viability of
the project. First, the Coast Guard has planned the entire project around an
expectation that it will get funding of about $350 million during the first
year and $525 million (in 1998 dollars) each year thereafter for the life of
the project. (This would include $500 million for the contractor and $25
million for the Coast Guard.) Adding inflation factors would increase this
amount substantially throughout the life of the project. Second, the
administration has told the Coast Guard to plan for considerably less-
perhaps half a billion dollars less for the first 5 years of the project.
Congress will make the final decision on this issue.
Success of Contracting Approach Relies on Sustained High Funding The
contracting approach chosen by the Coast Guard depends on a sustained level
of funding at planned levels over the life of the project. Any significant,
sustained deviation from the planned funding levels would cause the Coast
Guard to alter the system integrator?s schedule for producing and delivering
agreed to quantities and types of deepwater assets. Altering the schedule
after the contract
7
is awarded would require renegotiating prices in a sole source environment
and negotiating new cost and performance guidelines. This would be costly
for the Coast Guard in the short- term and would set off ripples affecting
the acquisition of deepwater equipment for years to come. Significant
shortfalls would likely result in increased costs, late delivery of
equipment, and even degradation of the performance of deepwater assets.
Projections of Available Funding for the Project Fall Short of Expectations
OMB?s budget targets for the Coast Guard?s capital projects have sent a
strong signal that planned deepwater funding levels for fiscal years 2002
through 2006 may be unattainable. Given the OMB budget targets, the Coast
Guard estimates that funds available for the Deepwater Project will be about
$2.2 billion (in 1998 dollars) through fiscal year 2006. Funding required
under current plans is about $2.5 billion, or a shortfall of about $300
million. However, adding inflation- which is what the Coast Guard has
instructed the contractors to do in their final proposals- would result in a
shortfall for the 5- year period of $496 million.
Administration?s Budget Projections Suggest the Need for a Lower Planning
Estimate
OMB guidelines for planning capital projects say that agencies should plan
new projects within available funding levels. If the Coast Guard were to
follow these guidelines for the Deepwater Project, it would align the
planned funding stream to OMB?s budget targets and tell the contracting
teams to develop their proposals accordingly. This would reduce the risk
later that deviations would have to be made from the system integrator?s
implementation plan due to funding shortfalls. However, the Coast Guard is
reluctant to lower the planned funding stream because it believes that (1)
the $525 million planning figure represents the level needed to optimize
operating effectiveness and efficiencies and (2) the administration will
provide more money for the project in future years. However, according to
OMB officials, future funding levels cannot be guaranteed; and it
8
would be inappropriate for the Coast Guard to tell the contractors to use
higher funding levels for the project that were not consistent with the
administration?s budget targets.
In our report, we recommended that the Department of Transportation (DOT)
align the planned funding levels for the Deepwater Project with the
administration?s budget targets. However, in commenting on our report, DOT
disagreed, saying that they plan to proceed with the hope that future
funding will materialize. In our opinion, this is unwise and fiscally
irresponsible unless the Congress sends a strong signal to the Coast Guard
that ample funding will be available for the project.
Ability to Control Cost?s in Project?s Later Years Remains Uncertain
When we initially reviewed the proposed contracting approach for the
Deepwater Project, we expressed concerns to the Coast Guard about whether it
could keep costs from rising and ensure good performance once the contract
is awarded. We were particularly concerned about the potential absence of
competition for subcontracts in the project?s later years and the heavy
reliance on a single systems integrator to procure the entire system.
Several other factors heightened our concerns. First, the contracting
approach had never been tried on a contract this large, extending over 20
years or more. There were no models to help guide the Coast Guard in
developing its approach. Second, when the Coast Guard selected the contract
in May 2000, it had little documented evidence to support the depth of its
analysis of risks with the approach, the factors considered, or the degree
to which this approach provided better value than other approaches. Finally,
we discussed the Coast Guard?s approach with contracting experts from the
public and private sector who echoed our concerns with the approach. Based
on these discussions, we asked the Coast Guard to undertake a more rigorous
analysis and seek outside expertise in validating its contracting strategy.
9
Potential Absence of Competition in the Project?s Later Years OMB guidance 3
on capital planning recognizes the value of competition as a lever to keep
contract costs down. Given that contract teams are competing for the initial
deepwater contract, the benefits of competition are present in the project?s
early years. Prices for deepwater equipment for this 5- year contract will
be pretty much fixed when the contract is awarded in 2002. Beyond the first
5- year contract, however, the benefits of competition are less certain. In
a practical sense, the opportunity for competition in the project?s out
years is diminished because the systems integrator will likely contract with
those suppliers that were part of the team putting together the proposal
rather than opening the contract to a wider set of offerors. We believe that
this potential lack of competition reduces the normal marketplace control on
price and subjects the Coast Guard to situations in which the supplier could
potentially drive up project costs.
The Coast Guard is attempting to develop strategies for encouraging
competition among suppliers. For example, the Coast Guard has included an
evaluation factor- for how well the integrator fosters competition- in its
criteria for evaluating the systems integrator?s performance and awarding
follow- on contracts. By doing so, the Coast Guard hopes that this will
encourage the systems integrator to have competition. At this point, it is
not clear what effect this evaluation would have.
Reliance on Single Contractor The dependence on a systems integrator to
acquire and integrate the deepwater systems is both one of the contracting
approach?s biggest strengths and one of its main weaknesses. On the positive
side, if all aspects of the approach work well, the systems integrator will
form a long- term partnership with the Coast Guard and provide technical
expertise to assemble an integrated system and the continuity
3 OMB?s Capital Planning Guide (Supplement to OMB Circular No. A-11).
10
needed to bring the project to a successful conclusion. However, the
approach could establish the integrator as a monopoly supplier,
substantially constraining the Coast Guard?s options or leverage. The Coast
Guard could be in a weak position to negotiate aggressively on price because
of its reluctance to take on the risks of increased costs and other problems
associated with switching systems integrators. For example, if the systems
integrator?s performance is marginal or unsatisfactory and the Coast Guard
is considering replacing the integrator, a new systems integrator will have
to step in to implement someone else?s partially completed design. The
learning curve and other complications involved in such a midcourse
adjustment could be dramatic and would probably be very costly.
Steps to Mitigate the Contracting Risks As our work progressed, we expressed
concerns to the Coast Guard immediately, rather than waiting until the end
of our review. As we raised concerns, the Coast Guard took additional steps
to study them. In September 2000, we urged the Coast Guard to take a number
of steps to deal with the risks of the contracting strategy, the most
substantive being to convene an independent panel of contracting experts
from the government and the private sector to review the contracting
approach. The Coast Guard agreed and formed such a panel, which met in April
2001. The panel identified additional items the Coast Guard needs to do
before it asks the contracting teams to submit a final proposal for the
deepwater system. At this point, we do not know what actions the Coast Guard
plans to take as a result of the panel discussions. However, we still
believe that outstanding issues remain to deal with competition and the
contracting approach before the agency proceeds much further.
Overseeing the Acquisition Phase of the Project Poses New Challenges
Another area of potential risk involves the overall management and day- to-
day administration of the contract. In this regard, the Coast Guard?s
performance
11
during the planning phase has been generally excellent. The acquisition
phase is a much tougher challenge, and the Coast Guard has much to do before
it is ready to award the deepwater contract.
Project Management During the Planning Phase Was Generally Excellent In the
planning phase of the project, the Coast Guard applied a number of ?best
practice? techniques recommended by OMB and others. 4 For example, the Coast
Guard gave contracting teams mission- based performance specifications, such
as the ability to identify small objects in the ocean, rather than asset-
based specifications, such as how large a cutter should be. Along with this,
the Coast Guard highlighted the use of ?open- system architecture? and
emphasized the use of commercially supported products in the equipment to be
acquired. In addition, the Coast Guard established a management structure of
Coast Guard and contractor teams for rapidly communicating technical
information.
The Coast Guard also had effective procedures and a management structure in
place for this phase of the project. Using a widely recognized management
model, we assessed procedures and structure in several key areas and found
no significant weaknesses. In fact, the Coast Guard?s procedures and
management structure for these areas were among the best of all the federal
agencies we have evaluated using this model.
Coast Guard Faces Difficult Challenges During the Procurement Phase As the
project moves from the planning phase to the procurement phase, the Coast
Guard must ensure that it can perform project management and contract
administration activities at a high level, given the complexity and scope of
the contract and its uniqueness. Under the Coast Guard?s planned approach,
the
4 Best practices are those that have been found to work well and that are
generally recommended by OMB and others.
12
systems integrator will be responsible for program management required to
implement the deepwater system, and the Coast Guard will continuously
monitor the integrator?s performance. The Coast Guard plans to implement, or
require the systems integrator to implement, many management processes and
procedures based on best practices; but these practices are not yet in
place. Because much work remains to be accomplished in this area, the full
effectiveness of the Coast Guard?s approach cannot be assessed in the short
term. The following are the key areas that will need to be addressed.
Effective human capital practices. A critical element to the ultimate
success of the project is having enough trained and knowledgeable Coast
Guard staff to conduct management and oversight responsibilities. Project
officials view this as a high- risk area and one of the most important
aspects of the project. The Coast Guard needs additional capabilities in
several critical areas and hopes to have its full complement of staff needed
for fiscal year 2002 by the time the contract is awarded.
Key management and oversight processes and procedures. Under its deepwater
acquisition approach, the Coast Guard will rely heavily on the systems
integrator to establish a management organization and systems necessary to
manage the major subcontracts for deepwater equipment. The systems
integrator will be responsible for developing key systems and processes,
such as risk management, quality assurance, and test and evaluation. In
addition, the Coast Guard is developing a program management plan to oversee
the systems integrator.
Close relationships with subcontractors. Because the use of major
subcontractors to provide high- value equipment will be such an intricate
part of the Deepwater Project, good relations and communications between the
Coast Guard, the systems integrator, and the major subcontractors will be
very important. Our past review of best practices on this issue suggests
that leading organizations establish effective communications and feedback
systems with their subcontractors to continually assess and improve both
their
13
own and supplier performance. 5 The Coast Guard has developed no general
policy on subcontractor relationships. The program management and quality
assurance plans have not been completed, and it is not clear, at this time,
what the quality and nature of the Coast Guard?s relationship with
subcontractors will be.
Full funding in advance of buying equipment. OMB Circular A- 11, Part 3,
emphasizes that each useful segment (e. g., an entire ship) of a capital
project should be fully funded in advance of incurring obligations. We found
in a review of earlier Coast Guard budget justifications that the Coast
Guard had proceeded with some capital projects before the amount of full
funding was identified. 6 As the Coast Guard proceeds with the Deepwater
Project, it needs to ensure that its budget requests are consistent with OMB
guidelines on full funding of useful segments.
Accurate and complete data to measure contractor performance. Coast Guard
officials told us that they plan to use a subjective rating system to assess
the contractor?s performance rather than use database benchmarks for
improvements in operational effectiveness and total ownership costs.
According to Coast Guard officials, setting such benchmarks may be difficult
because performance data may reflect factors that did not result from
actions of the contractor. For example, improved intelligence on drug
smugglers could result in improvements in operational effectiveness. Also,
changes in fuel costs could cause operational costs to increase. Because a
host of factors could cause changes in these data, it will be important for
the Coast Guard to carefully track these measures and accurately identify
and segregate reasons for the changes that occur. Doing so would better show
the results of significant federal investments in ships and aircraft.
Contingency planning and exit strategies. Given the Coast Guard?s heavy
reliance on a single systems integrator for so many facets of the Deepwater
5 Best Practices: DOD Can Help Suppliers Contribute More to Weapon System
Programs (GAO/NSIAD- 98-87, Mar. 17, 1998). 6 Budget Issues: Incremental
Funding of Capital Asset Acquisitions (GAO-01-432R, Feb. 26, 2001).
14
Project, the agency is at serious risk if- for whatever reason- the systems
integrator does not perform as expected or decides to walk away from the
project on its own. Faced with these options, having a carefully thought-
out contingency plan, which identifies and analyzes the implication of
potential actions, would solidify the Coast Guard?s ability to respond
effectively. In the extreme case- where the contractual relationship with
the systems integrator is terminated- an exit strategy identifying possible
alternatives, consequences, and transition issues would be important.
Use of Off- the- Shelf Technology Minimizes Risks, but Effective Means to
Assess Unproven Technology Is Lacking
The risks associated with incorporating new unproven technology 7 into the
first part of the Deepwater Project are minimal, in part, because of the
Coast Guard?s emphasis that industry teams use technology that has already
been proven in similar applications. Our main concern is the absence of
criteria to measure the risk of the new technology that does need to be
developed, both now and in the project?s later years.
Coast Guard?s Approach Conforms With Best Practices Too little assessment of
the risks associated with developing new technology has caused problems on
many acquisition projects, both in government and the private sector.
Minimizing a technology?s unknowns and demonstrating that it can function as
expected significantly reduce such risk. We have found that leading
commercial companies use disciplined processes to demonstrate- before fully
committing to product engineering and development- that technological
capability matches project requirements. Waiting to resolve these problems
can greatly increase project costs- at least 10- fold if the problems are
not resolved
7 We are using the term technology to denote assets, systems, equipment, and
components proposed for the Deepwater Project.
15
until product development, and as much as 100- fold if they are not resolved
until after production begins. 8
The Coast Guard has taken steps to minimize these risks. One major step was
to emphasize in contracting documents to industry teams that-- to the
maximum extent possible-- proposed assets, systems, equipment and components
are to be nondevelopmental or commercially available (off- the- shelf)
items. Our review showed that the teams? preliminary proposals included many
commercial off- theshelf and nondevelopmental items currently operating in
the commercial or military environment. However, some proposed equipment
included developing technology that has not yet been proven. Generally,
these developing technologies are at the prototype level and are undergoing
performance testing and evaluation prior to contract award to commercial and
military customers.
The Coast Guard?s steps are helping to keep the risk of unproven near- term
technology at a low level. We measured the maturity level for the project?s
most critical near- term technologies (those introduced in the first 7 years
of the project), using an approach developed by the National Aeronautical
and Space Administration (NASA). We applied this process, referred to as
technology readiness levels (TRL), to 18 technologies identified as critical
by the three contractor teams and the Coast Guard. We determined- and the
Coast Guard concurred- that by the time the contract is awarded, 16 of the
18 are expected to be at a level of acceptable risk. 9 The remaining two
technologies will be slightly higher in risk; but in one case, an early
prototype is being tested; and in the other, a proven backup system has been
identified that, if needed, could replace the
8 Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions (GAO/T- NSIAD-00-137, Apr. 26, 2000). 9 TRL readiness
levels are measured on a scale of one to nine. Examples of the ratings are
as follows: a rating of one signifies that studies of the basic concept have
been done; a rating between three and six means that success has been
demonstrated to a degree in laboratory situations; and a rating of nine
means that the technology has been proven in operational mission conditions
and is in final form. To be considered acceptable for committing to a
contract award, a new technology or adopted system should be rated at seven
or higher. A rating of seven means that a system prototype has been
demonstrated in the operational environment.
16
technology with no effect to the project?s cost, schedule, or performance.
Entering phase 2 of the project with critical technologies at a high level
of maturity or with proven backup systems significantly lowers risk and the
likelihood of delays, which in turn helps to control program costs.
Coast Guard Lacks Criteria to Assess Technology Maturity Although
technological risks appear minimal in the near term, the Coast Guard lacks
criteria for assessing the maturity of technology in the longer term. The
Coast Guard has a risk- management plan in place, as well as a process to
identify, continuously monitor, and assess technology risks; and the
resources the Coast Guard expects to commit to the task during phase 2
appear to be adequate. What the process lacks, however, is uniform and
systematic criteria for judging the level of technology maturity and risk,
such as the TRL ratings in the approach we adopted from NASA. In contrast,
since January 2001, DOD has required the use of TRL criteria as a tool for
measuring the technology readiness of its procurement projects.
Such criteria are important for monitoring both continued development of the
technologies we examined and the development of other technologies that will
not be used until later in the project. As of July 2000 when we completed
our TRL assessment, half of the 18 deepwater key technologies we reviewed
were still below the maturity level considered an acceptable risk for
entering production. Before the contract is awarded, the Coast Guard must
assess the readiness of these technologies. In addition, the industry team
proposals include numerous technologies that are planned for deepwater
system introduction from 2009 to 2020- well after contract award. Many of
these future technologies will not be proven at contract award and will need
to be assessed for technology risk before acceptance. The Coast Guard plans
to have a test and evaluation master plan in place by June 2001, but it is
not planning to include a requirement for using TRL criteria to measure
technology readiness in that plan.
17
Conclusions
Many critical issues must be addressed and resolved before the Coast Guard
is ready to procure deepwater equipment. Two issues, however, loom large as
needing more attention in the near term. Affordability is perhaps the
biggest issue and one on which the Congress will ultimately have to decide.
The question is,
?Should the Coast Guard plan the Deepwater Project around a much higher
funding stream than the administration estimates will be available for the
project?? Continuing down the funding path it is currently on is risky for
the Coast Guard and could lead to adverse consequences if significant
funding shortfalls for the project occur. The Coast Guard is reluctant to
back away from its $500 million funding mark because it believes that this
funding level is needed to provide the optimum deepwater system. So, the
Congress has an opportunity now to help the Coast Guard answer this question
by sending a clear message about its preferences on this matter.
The other critical issue centers around whether the Coast Guard can control
project costs effectively. The actions the agency has taken to date to
mitigate risks in this area are noteworthy, and it is still pondering other
means to keep costs in check. Much will depend on how well the Coast Guard
oversees and manages the contract as well. But, again, the Congress has an
opportunity to weigh in on the progress of the project as it proceeds. If
the Congress chooses to fund the project at levels that the Coast Guard
wants, we think that the Coast Guard should be accountable- through annual
report to the Congress- not only for managing the program well, but also for
documenting improvements in the operating efficiency and effectiveness of
its deepwater assets.
This concludes my statement. I would be pleased to answer any questions you
or other Members of the Subcommittee might have.
(394006)
*** End of document. ***
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
