Military

New Attack Submarine: More Knowledge Needed to Understand Impact of Design Changes (Letter Report, 04/30/98, GAO/NSIAD-98-38)

GAO reviewed: (1) the status of the new attack submarine (NSSN)
development program; (2) current information on the antisubmarine
warfare threat; and (3) the Navy's plans to model the NSSN's
survivability.
GAO noted that: (1) since modeling the NSSN's survivability in 1995, the
Navy, because of technical and funding limitations, has modified the
design for some subsystems that reduce performance below the optimal
levels used to model the 1995 baseline design; (2) other systems also
have developmental problems; (3) at the same time, Navy threat
assessments have reported that the open ocean antisubmarine warfare
threat has improved, resulting in a more capable threat than previously
projected; (4) the Navy tester's 1997 assessment report concluded that
the NSSN could potentially be operationally effective and suitable, but
noted a number of significant changes and risks in the development
program; (5) the report also noted several technological advances in the
open ocean antisubmarine warfare threat; (5) in addition, the report
stated that budgetary pressures resulted in tradeoffs in some of the
performance modeled in the NSSN milestone II cost and operational
effectiveness analysis and the tester's 1995 early operational
assessment; (6) as of November 1997, the Navy program manager planned no
additional survivability modeling to test the NSSN with its potential
for reduced performance against the improved threat; (7) however, as a
result of its 1997 assessment, the Navy tester recommended that the Navy
develop a new modeling baseline that reflects the reduced performance of
some subsystems and that this new design baseline be evaluated against
the increased threat; and (8) without such modeling, the Department of
Defense and Navy program officials appear to have little basis for their
confidence that the currently designed submarine will perform as
expected.
--------------------------- Indexing Terms -----------------------------
 REPORTNUM:  NSIAD-98-38
     TITLE:  New Attack Submarine: More Knowledge Needed to Understand 
             Impact of Design Changes
      DATE:  04/30/98
   SUBJECT:  Submarines
             Military cost control
             Naval warfare
             Military budgets
             Combat readiness
             Computer modeling
             Systems design
             Antisubmarine warfare
             Advanced weapons systems
             Defense capabilities
IDENTIFIER:  Navy New Attack Submarine Program
             TB-29 Towed Array System
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                    <info@www.gao.gov>                        **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************
Cover
================================================================ COVER
Report to the Secretary of Defense
April 1998
NEW ATTACK SUBMARINE - MORE
KNOWLEDGE NEEDED TO UNDERSTAND
IMPACT OF DESIGN CHANGES
GAO/NSIAD-98-38
New Attack Submarine
(707217)
Abbreviations
=============================================================== ABBREV
  DOD - Department of Defense
  NSSN - new attack submarine
Letter
=============================================================== LETTER
B-277780
April 30, 1998
The Honorable William S.  Cohen
The Secretary of Defense
Dear Mr.  Secretary: 
The Navy plans to spend $64 billion to acquire 30 New Attack
Submarines (NSSN) over the next 18 years.  According to the Navy,
these purchases will allow it to maintain its force structure goals
and the current submarine industrial base.  As part of our efforts to
assist in the oversight of major weapon systems acquisition programs,
we reviewed (1) the status of the NSSN development program, (2)
current information on the antisubmarine warfare threat, and (3) the
Navy's plans to model the NSSN's survivability.  We are providing the
results of our review for your use in oversight of the NSSN program. 
   BACKGROUND
------------------------------------------------------------ Letter :1
The NSSN program is intended to address the Joint Chiefs of Staff
requirement for 10 to 12 new attack submarines with Seawolf level
quieting by the year 2012 and to maintain future force structure
goals.  In funding the NSSN program, Congress expected the Navy to
deliver a less costly submarine than its predecessor, the Seawolf,
without compromising military utility.  The NSSN is expected to be a
highly effective multimission platform capable of performing
antisubmarine and antisurface ship missions and land attack strikes
as well as mine missions, special operations, battle group support,
and surveillance.  The NSSN is also expected to be as quiet as the
Seawolf, include a vertical launch system, and have improved
surveillance as well as special operations characteristics to enhance
littoral warfare capability.  While the NSSN is expected to perform
effectively against the most capable, open ocean, nuclear attack
submarine threat, it will be slower and less capable in diving depth
and arctic operations and will carry fewer weapons than the Seawolf. 
The Navy's fiscal year 1999 budget request contained about $1.5
billion for procurement of the second NSSN and $504.7 million for
advanced procurement of the third authorized NSSN.  The Navy also
requested about $219 million for continued research and development
activities.  Public Law 105-56\1 appropriated funds and Public Law
105-85\2 provided authorization for the contractor teaming
arrangement to build the first four new attack submarines.  The Navy
has established performance levels to ensure that the NSSN will have
the capabilities to successfully conduct its missions.  Operational
requirements documents are required for the ship and its major
subsystems.  These documents establish the optimal (objective) and
minimal (threshold) requirements related to the submarine's
performance.  For the most part, according to the NSSN program
manager, the NSSN is being designed to meet a cost-effective balance
at a performance level that meets or exceeds minimum requirements. 
The Navy is also establishing detailed technical specifications for
the design of individual subsystems. 
To gain assurance that the designs of the submarine and its
subsystems will result in the submarine successfully performing its
various missions, the Navy requires that the Program Manager use
computer simulations as a principal tool to model the NSSN's
capabilities against existing and potential threats.  An example is
the modeling performed for the June 1995 NSSN milestone II cost and
operational effectiveness analysis.  Based on the results, both the
Department of Defense (DOD) and the Navy believe the baseline NSSN
design satisfies military requirements.  The Navy also seeks
assurance by requiring that weapon systems be tested and evaluated in
their anticipated operational environment and against the anticipated
threat.  This mission is performed by the Operational Test and
Evaluation Force, which was established by the Secretary of the Navy
to be the Navy's sole independent agency for these activities. 
--------------------
\1 The Department of Defense Appropriations Act, 1998. 
\2 The National Defense Authorization Act for Fiscal Year 1998. 
   RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
Since modeling the NSSN survivability in 1995, the Navy, because of
technical and funding limitations, has modified the design for some
subsystems that reduce performance below the optimal levels used to
model the 1995 baseline design.  Other systems also have
developmental problems.  At the same time, Navy threat assessments
have reported that the open ocean, antisubmarine warfare threat has
improved, resulting in a more capable threat than previously
projected. 
The Navy tester's 1997 assessment report concluded that the NSSN
could potentially be operationally effective and suitable, but noted
a number of significant changes and risks in the development program. 
The report also noted several technological advances in the open
ocean antisubmarine warfare threat.  In addition, the report stated
that budgetary pressures resulted in trade offs in some of the
performance modeled in the NSSN milestone II cost and operational
effectiveness analysis and the tester's 1995 early operational
assessment. 
As of November 1997, the Navy program manager planned no additional
survivability modeling to test the NSSN with its potential for
reduced performance against the improved threat.  However, as a
result of its 1997 assessment, the Navy tester recommended that the
Navy develop a new modeling baseline that reflects the reduced
performance of some subsystems and that this new design baseline be
evaluated against the increased threat.  Without such modeling, DOD
and Navy program officials appear to have little basis for their
confidence that the currently designed submarine will perform as
expected. 
   CHANGES MADE IN APPROVED NSSN
   DESIGN AND POTENTIAL FOR MORE
------------------------------------------------------------ Letter :3
Since the Navy modeled the NSSN in 1995, a number of subsystems in
development have encountered financial constraints and developmental
problems.  These financial constraints resulted in modifying the
design requirements for some of the subsystems to reduce the
performance capabilities.  Significant development risks are also
present in other subsystems that could further affect planned
performance.  The Navy's tester noted that many of the potential
risks are the result of program restructuring to mitigate the effects
of internally directed funding cuts.  He expressed concern that the
combined effects of the reductions in performance and developmental
risks may affect the NSSN's operational effectiveness. 
      REDUCED CAPABILITY
      SUBSYSTEMS
---------------------------------------------------------- Letter :3.1
The Navy has restructured two key NSSN subsystems--electronics
warfare and acoustic intercept.  The Navy has also reduced or will
reduce some operational performance requirements to the minimum
acceptable levels for the NSSN to successfully complete assigned
missions. 
The electronics warfare system enables the NSSN to covertly monitor
intelligence targets and record electronics data.  Because of
internally directed fiscal year 1998 funding cuts, some system
capability was removed.  The reduced capability system will not meet
the optimal performance levels modeled in the 1995 assessment, but it
is projected to meet minimum levels.  The Navy has established the
detailed technical specifications that will be important to meeting
those projections but has not approved all of the operational
requirements documents. 
Public Law 105-56 provided increased funding to restore some of the
critical elements of the electronics warfare subsystem--such as
specific emitter identification, full implementation of precision
radar band direction finding, and interception of frequency-hopping
communications.  Public Law 105-85 authorized the increase. 
The acoustic intercept system provides defensive capability for the
submarine and according to a Navy official, is critical to its
survival.  Like the electronics warfare system, the acoustic
intercept system was restructured because of fiscal year 1998
internal funding cuts.  Although the restructured system will have
fewer capabilities than the original one, limited computer modeling
indicates that if the restructured system performs as expected, there
is no statistical difference in performance.  The question is whether
the restructured system will perform as expected.  In the June 1997
operational assessment of this system, the Navy tester noted several
deficiencies in achieving required performance.  (Detailed
information on these deficiencies is classified.) As a result, the
Navy tester recommended approval for only a single unit for backfit
testing on 688I class submarines and only one unit for release to
support the first NSSN contingent upon resolution of these issues. 
      SUBSYSTEMS EXPERIENCING
      DEVELOPMENTAL PROBLEMS
---------------------------------------------------------- Letter :3.2
The submarine's propulsor and external communications systems are
experiencing development problems.  These problems, although not
unusual at this stage in a weapon program, present significant risks
in meeting performance requirements.  Also, the design for the lower
cost alternative to the present towed array has not been approved,
nor has a contractor been selected. 
The propulsor provides thrust to move the submarine through the
water.  Cavitation\3 noise from the propulsor is critical to the
ability of enemy submarines or surface ships to detect the submarine
and, consequently, has a major impact on a submarine's survivability
and operational effectiveness.  Currently, there is no cavitation
performance requirement in the NSSN operational requirements
document, but there are program office cavitation design goals. 
The Navy, through large-scale vehicle testing, determined that an
interim propulsor design did not meet the program office's cavitation
design goals.  As a result, it has developed two alternative designs
that it began to test in March 1998.  To meet the lead ship NSSN
production schedule, the Navy must select from these alternatives
during the one remaining large-scale vehicle test before a propulsor
for the lead ship is produced.  If the alternative designs do not
meet cavitation goals, the Navy plans to backfit another redesigned
propulsor on the lead ship. 
The external communications system was restructured in August 1996 to
provide a cost-effective means of introducing commercial hardware and
software technologies in order to meet the NSSN development schedule
and operational requirements.  This system consists of several
components such as the submarine high data rate antenna system,
various radio frequency receivers, imagery and teleconference video
capability, and internal data distribution systems. 
Improvements in the data rate capability of the external
communications system depend on the high data rate antenna system and
the amount of satellite resources allocated to submarine platforms. 
As currently designed, with a 17-inch antenna, the Navy tester noted
that the submarine's system will only be able to process the required
amounts of data if all of the Navy's current satellite resources are
allocated to support submarine communications.  The Navy is
attempting to establish a concept of operations among satellite
scheduling units that will allocate appropriate resources to the
deployed submarine.  Program office officials said the Navy has
alternative ways to provide the required satellite resources such as
using different frequencies on satellites or leasing commercial
satellites. 
In addition, the Navy has not completed an overall operational
requirements document for submarine external communication systems. 
As such, the NSSN external communications system design has not been
finalized.  These documents are required to ensure that the system
configuration is properly designed to meet minimum performance
requirements. 
--------------------
\3 The noise generated by the movement of the submarine's propulsor. 
      STATUS OF TB-29 TOWED ARRAY
---------------------------------------------------------- Letter :3.3
The TB-29 towed array and its processing system are critical to NSSN
operations in detecting, tracking, and, if required, attacking a
threat submarine.  This system enables the NSSN to hear acoustic
noises made by threat submarines.  However, the Navy has determined
that the current TB-29 system is too expensive.  Also, the contract
for the current TB-29 expired in fiscal year 1997. 
The Navy is looking for a comparable system at a lower cost than the
TB-29 array.  Navy officials told us that required technology is
available and that it is a matter of selecting a design and a
contractor to produce the system.  They believe there is sufficient
time to develop and procure a new system to meet the delivery of the
first NSSN.  However, there is no approved design for the new system. 
Some developmental funding has been specifically identified.  Navy
officials said the Chief of Naval Operations has fully supported
completing the TB-29 follow-on development and procurement in future
years' budget submissions. 
According to the program manager, a request for proposal for the
design of a new array will be issued early in fiscal year 1998.  The
Navy expects to award a contract for the development and production
of the new array in the third quarter of fiscal year 1998. 
   A MORE CAPABLE THREAT HAS BEEN
   DEFINED
------------------------------------------------------------ Letter :4
In April 1996, the Office of Naval Intelligence revised its
classified underseas threat assessment and noted several
technological advances in the open-ocean, antisubmarine warfare
threat.  Several improvements resulting in a more capable threat were
noted over the previous threat of record, which the Navy used to
model the survivability of the NSSN design in the 1995 assessment. 
(Details of these improvements are classified.) Facing a more capable
threat, and without an increase in submarine capability, the risk to
the NSSN's survivability is likely to increase. 
   NAVY DOES NOT PLAN
   SURVIVABILITY MODELING OF
   CHANGES IN DESIGN AND THREAT
------------------------------------------------------------ Letter :5
The Commander, Operational Test and Evaluation Force, conducted NSSN
operational assessments in April 1995 and again in January 1997. 
(Detailed results of these assessments are classified.) The 1995
assessment was conducted using computer simulated modeling of the
baseline NSSN design against the threat projected at that time.  As a
result of the 1995 assessment, the Navy tester expressed concern that
if the NSSN were just to meet minimum requirements for survivability,
the NSSN may not be operationally effective against the most capable
threat that the Navy was projecting at that time. 
The 1997 assessment was based on a more limited amount of
information, such as changes outlined in budgetary documents, and did
not include an in-depth survivability modeling as was done for the
1995 assessment.  The Navy tester's report noted reduced performance
of several subsystems and developmental problems in others that also
will result in reductions in planned performance.  The report pointed
out that many of the affected subsystems, such as the acoustic
intercept system and the propulsor, are necessary to support the
NSSN's operational effectiveness and survivability. 
The Navy tester concluded that the NSSN could potentially be
operationally effective and suitable.  However, he recommended that a
new NSSN modeling baseline be established to reflect more current
information, because the performance of some subsystems had been
reduced below the performance modeled in the 1995 NSSN milestone II
cost and operational effectiveness analysis and the April 1995 early
operational assessment.  The tester also recommended that this new
design baseline be evaluated against the currently projected threat. 
Navy program officials are cognizant of the Navy tester's report but
have indicated that there are no plans to perform an updated
survivability modeling of the total system against the new threat. 
Navy program officials told us that they have modeled, or plan to
model, the performance of individual subsystems instead.  Program
officials also stated that even at the current reduced performance
levels, the subsystems discussed will still meet NSSN minimum
requirements.  However, the submarine's survivability has only been
assessed using performance levels above the minimum requirements. 
   CONCLUSIONS AND RECOMMENDATION
------------------------------------------------------------ Letter :6
The combined effects of a more capable threat, the reduction of some
system performance requirements, and the risks inherent in new
development could affect the NSSN's operational effectiveness. 
Without an evaluation that reflects current conditions, DOD and Navy
program officials appear to have little basis for their confidence in
how the submarine, with its design changes, will perform.  Given the
complexities and uncertainties in weapon system acquisitions,
encountering performance problems during the development phase is not
unusual.  At this point in the NSSN program, using modeling tools to
identify and correct problems that could affect the system's
survivability, such as those described in this report, would allow
changes to be made in development schedules and funding profiles at a
much lower cost than if problems were identified later. 
To avoid spending funds on construction from a design that may
require costly modifications to meet requirements, we recommend that
the Secretary of Defense require the Secretary of the Navy to conduct
survivability modeling to assess the impact that reduced capabilities
of various subsystems have on ship survivability when integrated into
the overall NSSN design.  Available research and development funding
could be used for this modeling.  Further, we recommend that the
Secretary of Defense take steps to ensure that the results are used
in making fiscal year spending decisions on the program. 
   AGENCY COMMENTS
------------------------------------------------------------ Letter :7
DOD provided written comments on a draft of this report, which are
reprinted in appendix I.  DOD stated that it agreed with the
recommendation in our draft report to conduct sufficient
survivability modeling to assess the extent to which the NSSN will be
fully capable of countering the threat and meeting all its mission
requirements.  In its comments, DOD acknowledged that the performance
of some subsystems was reduced below that used to model the
survivability of the NSSN during the
milestone II cost and operational assessment and the 1995 early
operational assessment.  DOD laid out the process by which it makes
decisions on what testing is needed and how the test results are
used.  DOD offered, as an example, that design changes to the
Acoustic Intercept Receiver and to the Electronic Warfare Support
Measures suites were assessed and determined to have reduced
performance.  The program's management concluded that the reduced
performance of these subsystems would not compromise ship
survivability and, therefore, no higher level modeling was required. 
DOD also stated that operational assessments, already scheduled for
fiscal year 2000 on an interim basis and fiscal
year 2002 for a final report, will assess the impact on overall NSSN
performance of changes to the design, validated threat projection,
and demonstrated subsystem performance. 
The intent of our recommendation, however, was to have DOD conduct
survivability modeling.  As we point out in the report, until the
cumulative effect of subsystem changes, including reduced
performance, on overall ship survivability is modeled, it will not be
known if the NSSN will perform as intended.  For example, while
performance modeling indicates that the restructured acoustic
intercept system may perform as expected, this does not answer the
question of what impact the system's reduced capabilities have on
ship survivability when integrated into the overall NSSN design. 
Therefore, although important, individual assessments of subsystem
performance such as those conducted in the Janaury 1997 operational
assessment, do not provide information on overall survivability when
they are integrated into the overall submarine design. 
Likewise, the second phase of operational testing discussed in our
report and scheduled to be reported on in fiscal year 2000 will not
include an assessment of the overall survivability of the NSSN at
reduced levels of subsystem performance, unless explicitly requested
and paid for by the program sponsor.  Program officials have no plans
to do so.  As we note, the Navy rejected the recommendation in the
January 1997 operational assessment that a new NSSN baseline be
established to reflect more current information and be evaluated
against the currently projected threat.  Based on our discussions
with Navy officials, there is no indication that tests scheduled for
fiscal year 2000 will include an assessment of overall survivability
nor that the results of the tests will be used to make modifications
to the program.  If the combined reduction of subsystem performance
is subsequently found to affect overall ship survivability, the NSSN
program could face expensive modifications or reduced capability. 
Therefore, we have modified our draft report recommendation to
clarify what we meant by sufficient survivability modeling. 
DOD concurred with our recommendation that the Secretary of Defense
take steps to ensure that the modeling results are used in making
fiscal year spending decisions on the program.  DOD officials have
stated that it now plans to conduct comprehensive annual reviews of
the NSSN program. 
   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :8
We analyzed Navy and DOD documents and studies such as the NSSN cost-
and operational effectiveness analysis and discussed the status of
the NSSN's acquisition with Navy program officials in Washington,
D.C.; at the Naval Undersea Warfare Center, Newport, Rhode Island;
and the Naval Surface Warfare Center, Carderock Division.  We held
additional discussions with officials from the offices of the Chief
of Naval Operations; the Assistant Secretary of the Navy for
Research, Development, and Acquisition; the Secretary of Defense; and
the Program Executive Office for Submarines.  We also discussed
program acquisition status with (1) representatives from Electric
Boat Corporation, Groton, Connecticut, and Newport News Shipbuilding
and Drydock Company, Newport News, Virginia; (2) the Supervisors of
Shipbuilding at these respective shipyards; and (3) representatives
from Lockheed Martin Federal Systems, Manassas, Virginia.  In
addition, we analyzed the threat modeling and other testing results
contained in the NSSN's operational assessments and discussed the
results with representatives of the Commander, Operational Test and
Evaluation Force, Norfolk, Virginia.  Discussions on the capabilities
of the projected submarine threat were held with representatives of
the Office of Naval Intelligence, the Defense Intelligence Agency,
and the Central Intelligence Agency. 
We conducted our review between December 1996 and March 1998 in
accordance with generally accepted government auditing standards. 
We are sending copies of this report to the four congressional
Defense committees, the Secretary of the Navy, and the Assistant
Secretary of the Navy for Research, Development, and Acquisition. 
Upon request, we will make copies available to other interested
parties. 
Please contact me on (202) 512-4841 if you or your staff have any
questions concerning this report.  Major contributors to this report
are in appendix II. 
Sincerely yours,
Katherine V.  Schinasi
Associate Director, Defense
 Acquisitions Issues
(See figure in printed edition.)Appendix I
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
============================================================== Letter 
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
NATIONAL SECURITY AND
INTERNATIONAL AFFAIRS DIVISION,
WASHINGTON, D.C. 
Richard J.  Price, Assistant Director
Jack G.  Perrigo, Jr., Senior Evaluator
NEW YORK /BOSTON FIELD OFFICE
Richard E.  Silveira, Evaluator-in-Charge
Joseph Rizzo, Jr., Evaluator
OFFICE OF THE GENERAL COUNSEL
William T.  Woods, Assistant General Counsel
*** End of document. ***