• Military Menu                       

• Introduction
• Systems
• Facilities
• Agencies
• Industry
• Operations
• Countries
• Hot Documents
• News
• Reports
• Policy
• Budget
• Congress
• Links

• WMD
• Intelligence
• Homeland Security
• Space

	Director, Operational Test & Evaluation
	FY98 Annual Report

INTRODUCTION

In accordance with our statutory responsibility to "review and make recommendations to the Secretary of Defense on all budgetary and financial matters relating to operational test and evaluation, including operational test facilities and equipment," we have reviewed the state of test and evaluation (T&E) in the Department of Defense.

There is much positive news related to T&E. Given the dramatic reductions in defense spending overall, T&E workload is surprisingly robust. T&E workload is a function of the number and variety of different systems under test, not procurement quantities, and many programs are now undergoing test and evaluation.

On the other hand, a decade of downsizing is taking a toll. The T&E infrastructure is showing signs of significant deterioration, and our ability to meet T&E requirements in the future is getting farther out of reach.

We take pride in supporting our warfighters, both through effective T&E and by absorbing a fair share of the downsizing burden. Since the peak defense budget of 1985, the T&E infrastructure has been reduced about 35 percent, and the total acquisition infrastructure, including T&E, has been reduced about 18 percent. Since 1995, T&E and the total acquisition infrastructure have both been reduced about 15 percent. In FY99, T&E operating and investment funding will be $1.1 billion less than it was in 1990. Yet, T&E did not benefit from the acquisition build-up of the 1980s.

We should all be concerned about our ability to meet future T&E requirements. The workload at all the Service Operational Test Agencies is up dramatically from just a few years ago and is climbing. While the need for operational test and evaluation increases, the test and evaluation infrastructure on which we depend to meet that need continues to deteriorate. The deterioration is the result of significant reductions in manpower and operating and investment funding that outpace any decrease in workload they may be experiencing. Where improvements are being introduced, they do not begin to compensate for inadequate maintenance and operating funds, loss of personnel and expertise, and loss of productivity due to aging, technologically outdated facilities. Contributing to this erosion of capability is a lack of new investment to address the most urgent long-term needs and inadequate test and evaluation operating budgets to meet current workload. The latter is a direct result of an exaggerated perception of excess capacity, discussed below.

In the future, weapon systems will be even more complex; they will have to be interoperable and function as a system of systems; they will be highly dependent on information and digitization; and they will be deployed under a broader and more dynamic set of conditions. Clearly we must have a more capable and robust T&E infrastructure if we are to provide our warfighters with weapons that work.
 
 

WHY IS A STRONGER T&E INFRASTRUCTURE IMPORTANT?

T&E Plays a Critical Role in the Acquisition Process

The importance of adequate test and evaluation during all phases of the acquisition process cannot be overstated. The purpose of test and evaluation is to provide essential information to decision makers, assess whether weapon system technical performance parameters have been attained, and determine whether systems are operationally effective, suitable, and survivable for their intended use. T&E programs are structured to integrate all aspects of testing to include developmental T&E (DT&E), operational T&E (OT&E), contractor testing, and live fire testing. To support testing, the T&E community uses increasingly representative modeling and simulation as well as sophisticated hardware-in-the-loop facilities and highly instrumented open land, air, and sea ranges. Through early involvement in the acquisition process, T&E expertise is focused on early detection of design and development problems and on building quality and excellence into programs from the start.

OT&E Workload Is Growing

Many factors contribute to the expected growth in OT&E workload in the future. First of all, both Procurement and Research, Development, Test and Evaluation (RDT&E) funding are expected to increase over the next few years as the modernization programs resulting from the Quadrennial Defense Review (QDR) are undertaken. To implement the QDR initiatives, the Secretary of Defense and the Under Secretary of Defense (Acquisition and Technology) are committed to increasing the annual acquisition budget to over $68 billion, an increase in real terms of more than 37 percent over the low point in 1997. In addition to these important modernization programs, a number of very large acquisition programs are currently underway. Some of these key programs are listed in the box below.

The Revolution in Military Affairs (RMA) and Joint Vision 2010 concepts¾ including the need to validate models and simulations and increase joint operational testing, and training¾ are also contributing to growth in OT&E workload. Joint Vision 2010 requires "systems of systems" testing, which is more complex and test resource intensive. In addition, Secretary of Defense direction requires involvement of operational test and evaluation earlier in programs, expanded use of modeling and simulation, combining OT&E with developmental test and evaluation, leveraging of testing and training exercises, and greater OT&E involvement in Advanced Concept Technology Demonstration (ACTD) programs.

Involving operational testers early in a program, results in early operational insight and learning that can reduce program risk and acquisition cycle-times. This early involvement is not restricted to the major Acquisition Category (ACAT) 1D programs but includes smaller programs and ACTD programs as well. This enables the operational tester to introduce operational realism early in program testing, which contributes strongly to program success.

Other factors contributing to the increase in the operational test and evaluation workload include involvement in Joint Experimentation programs and battle lab exercises for the CINCs, Rapid Acquisition Programs, and an increasing number of non-major and upgrade programs. The battle laboratories and Joint Experiments provide early information on the operational effectiveness of proposed weapon systems. Early involvement by operational T&E personnel supports operational users in assessing concepts for new systems, tactics, or doctrine.

Operational test workload is expected to increase for the foreseeable future. Figure 1 shows Service trends in the number of OT&E programs. The Navy's operational test agency, OPTEVFOR, has more OT&E programs now than at any other time in its 54-year history.

MRTFB Workload Remains Robust

Although its composition has shifted over time, Figure 2 illustrates that Major Range and Test Facility Base (MRTFB) workload has remained relatively steady in terms of user funding for the past ten years. Despite fluctuations from 1987-1998 in the number of new weapons development programs, a significant amount of the workload continues to be related to engineering and manufacturing development. Workload associated with engineering and manufacturing development will increase in the future due to the QDR initiatives, as noted above. Workload associated with product improvements to operational systems has remained relatively steady as the Department continues to improve already fielded systems. In addition, workload associated with quality assurance and aging and surveillance has grown as fielded systems age, and consequently require testing to ensure continued high confidence in their potential performance. Taken together, these elements have kept the workload at the MRTFB relatively steady in terms of user funding over the past decade.

An alternative measure of workload is the number of workyears expended at the major T&E centers. Figure 3 illustrates the trend in workyears funded by users as well as those funded institutionally. Overall, the workyears for T&E activities paid for by users declined by about 10 percent between FY87-FY99. However, a portion of this decline can be attributed to efficiency gains made possible by investment and modernization programs. For example, new instrumentation systems require fewer people to operate than the old systems they replaced. The institutionally funded workyears declined about three times as fast as the user funded workyears. This represents a significant achievement in revolutionizing business affairs to reduce the overhead burden within our T&E infrastructure.

WHERE DOES OUR T&E INFRASTRUCTURE STAND TODAY?

An Infrastructure Made of People, Processes and Facilities

To remain sound, the T&E infrastructure must contain a balanced mix of people, processes, and facilities. Together, these three components enable the Department to conduct quality T&E to ensure we buy weapons that work. People in sufficient numbers with appropriate skills are always needed, and adequate personnel development programs must be in place to meet projected needs. Key business processes must reflect the best business practices, and facilities must be modern and efficient. To some extent, we can offset weaknesses in one of these components by investing more heavily in another. For example, process improvements may reduce the number of people needed to operate a particular facility or piece of equipment. However, continued reductions in people and funding have brought us to a point where process improvements alone are no longer enough. Now, facility modernization and investment funding is needed so T&E infrastructure can support future demands.

People

Manpower Reductions Have Been Drastic to Match Budgets Cuts

People are a key element of our T&E infrastructure. Our T&E professional workforce is both our greatest asset and our biggest expense. Figure 4 shows that, between FY 1987-FY 1999, the MRTFB workforce has declined by over 9,200 people (about 22 percent). This reduction is roughly equivalent to the reduction in workforce caused by eight base closures.

Military Personnel Reductions in T&E Have Been Severe

Not only has the T&E workforce been cut, but the number of military personnel available for participation in test and evaluation has also been reduced. Since FY90, we have seen a 39 percent decrease in the number of military personnel at MRTFB activities. Removal of military personnel from the test and evaluation community will have grave effects on both developmental and operational test and evaluation. Active military participation in testing is key to understanding how a system will actually be used in combat. The Army, to return as many personnel as possible to "warrior" status, is virtually eliminating military personnel from working in the MRTFB. In 1990, the Army had 762 soldiers directly supporting T&E programs. Their numbers have already been reduced by 98 percent. By the year 2001, the Army will have only five military personnel directly supporting T&E. White Sands Missile Range, NM, will have none. The indirect numbers are equally startling. In 1990, the Army had 504 institutional military personnel in T&E. In the year 2001, they will have 37. White Sands Missile Range will have two of those 37.

Severe reductions in military personnel in T&E are being seen in the Navy and Air Force as well. Military personnel funding is down 40 percent at Navy and 35 percent at Air Force MRTFB activities. Each of the Services needs to commit to a stronger military role in T&E to ensure military operational insights and perspectives are preserved.

Civilian Workforce Is Aging

Recent years have seen a gradual decline in the numbers and an increase in the average age of highly skilled government civilian workers. Several factors have contributed to this decline. First, many high-quality people left because of downsizing and lack of advancement opportunities. At the same time, there has been little hiring in the past ten years to ensure the workforce remains balanced.

In addition, younger members of the workforce are being forced out because of cut backs that affect less senior people. Also, positions for which younger personnel can be hired are scarce. For example, the Aberdeen Test Center in Maryland has had only one new-hire in six years. The loss of high-quality people combined with the lack of ability to hire new people has gradually degraded T&E capability.

Loosing a large portion of our most experienced civilian workers due to retirement is a constant management challenge. A high percentage of our aging civilian workforce is within 10 to 15 years of retirement. At Army Test and Evaluation Command MRTFB activities, the average age of government civilian employees is 46 to 48 years old. At Air Force MRTFB activities, 54 percent of civilian employees are at least 46 years old, while only 14 percent are under the age of 36. At Navy MRTFB activities, 24 percent of government civilian employees are eligible to retire. The average age of the workforce increases nearly one year every year. We need to begin stabilizing this workforce today and work toward invigorating it over time.

T&E Is Significantly Outsourced

In FY98, contractors accounted for approximately 53 percent of the MRTFB workforce. As Figure 5 shows, about three-fourths of MRTFB funding is allocated for purchased services from commercial sources. Only one-fourth of MRTFB funding is for government personnel. Furthermore, only one-fourth of the T&E costs of major acquisition programs goes toward government T&E at the MRTFB. The majority of funds go toward testing performed by the development contractor.

Over the years significant savings have accrued from contracting out functions such as base operations support, and aircraft, vehicle, and facility maintenance. Additional opportunities exist, and a number of studies underway at MRTFB activities are expected to lead to additional, mainly non-core functions, being contracted out. However, these efforts can result in poor morale and disrupt operations. In addition, the savings derived from such efforts can be much less than expected. When savings from outsourcing fail to materialize, MRTFB activities are being forced to make up for those savings from other already strained resources.

Processes

Revolution In Business Affairs

The second element of infrastructure is process. Changes in processes have helped us to compensate for the reductions in T&E infrastructure funding and personnel. Major process improvement initiatives include those to: leverage technology to improve the efficiency and productivity of our facilities; partner with other government agencies, industry and our allies to leverage each others facilities; and reengineer our business processes to improve performance and to provide more affordable testing through better business practices.

Examples of initiatives in each of these areas are given in the following subsections.

Leveraging Technology

Significant advancements in technology, especially in the area of high-performance computational systems, have provided opportunities for the T&E community to improve test processes. Some of these improvements involve the use of modeling and simulation to augment traditional types of live testing. Modeling and simulation can substantially reduce the time, resources, and risk associated with the acquisition process. The use of modeling and simulation throughout the requirements and acquisition phases will allow early operational insight into programs and identify corrective actions before metal is bent or software rewritten. As a program's design starts to mature, appropriate OT&E modeling and simulation can generate operationally realistic scenarios to verify critical components in hardware-in-the-loop facilities and test operational software. Before any field test, modeling and simulation can be used to optimize test conditions, to maximize the payoff from each test event, and to generate pre-test predictions. The value-added of each test is well understood and data collection plans focus on the most important items needed for system development.

For some programs, such as those in the Ballistic Missile Defense program, the use of modeling and simulation technology is an absolute necessity. Operational requirements are written at the theater level. Operational testing at these levels is precluded not only by test range limitations and safety considerations but also by cost. As it is, each field test of these systems, which is limited to "one on one" and "few on few" can cost several million dollars. For all of the Ballistic Missile Defense programs, over $2 billion will be spent on flight tests. The determination of lethality for these programs is also dependent on modeling and simulation technology, both at the point of intercept and for ground and collateral effects.

The introduction of additional capabilities and facilities based on modeling and simulation has also enabled us to improve test processes. One example is the use of the Aerial Cable Test Facility at White Sands Missile Range, NM. During the course of 662 individual tests, it saved projects over $20 million in its first year of operation. Another example is the use of the Air Combat Environmental T&E Facility at the Naval Air Warfare Center Aircraft Division, Patuxent River, MD. By testing equipment on the ground, installed in the EA-6B aircraft, the facility has reduced flight test hours and cost by one-third.

While we are proud of the initiatives we have undertaken in the area of leveraging technology, we are concerned that future efforts may be stymied by a lack of investment and modernization funds. While modeling and simulation offer a tool to supplement more expensive test and evaluation, it nevertheless requires significant resources to develop and use. Use of modeling and simulation for test and evaluation requires: (1) development to design code and acquire a hardware environment; (2) verification, validation, and accreditation; (3) hardware-in-the-loop capability, as appropriate; and (4) documentation for reuse.

The broad spectrum of 21^st century military operations demands environments that are flexible and able to keep pace with varied test and training requirements. These requirements present numerous opportunities for testers and trainers to work together to solve mutual problems at the lowest cost with improved effectiveness. Processes need to be developed that will improve our capability to augment the live test and training environment by capturing benefits from the rapid improvements in simulations and stimulations, an area of significant shortfall for both operational testing and training.

Four foundation programs are supporting the use of modeling and simulation to its full potential for supporting test and evaluation and training activities for Joint Vision 2010 among ranges, facilities, digital models, and virtual simulations. These four programs-Test and Training Enabling Architecture; the Common Display, Analysis, and Processing System; the Virtual Test and Training Range; and the Joint Regional Range Complex-are currently funded under the Central Test and Evaluation Investment Program (CTEIP) as the Foundation Initiative 2010. They embody the critical technology needed to enable the DoD and contractor ranges and simulations to interact so that they can test interoperability and interdependence among weapon systems in the widest variety of test conditions with the most realism.

Developing Partnerships

By partnering with those outside the DoD T&E community, we have been able to leverage each other's capabilities, investments, and operations to conduct quality T&E in a resource constrained environment. The T&E community is partnering with the training community to leverage training ranges and exercises to conduct cost-effective operational test and evaluation under realistic conditions. DoD has partnered with the National Aeronautics and Space Administration (NASA), and the Departments of Energy and Transportation through memorandums of agreement, project orders, and other means to support DoD test and evaluation programs. For example, the DoD relies on NASA at Langley, VA, for wind tunnel support. However, reductions in NASA budgets have contributed to lack of investments to revitalize these facilities to the level of readiness needed to support DoD requirements. Some DoD test and evaluation requirements can only be satisfied by the use of facilities and ranges outside of the United States. For this reason, DoD has entered into a number of partnerships through memorandums of agreement, data exchange agreements, and other arrangements with international allies. DoD maintains an active program to catalog international capabilities and opportunities and works with a number of nations to standardize test procedures so that test results are interchangeable. Of particular note has been the Canada/U.S. Test and Evaluation Program (CANUSTEP). While a majority of the programs supported under this agreement are Canadian programs using U.S. ranges and facilities, Canadian geographic features offer capabilities for some of the larger U.S. programs not available elsewhere. This partnership prevents the United States from having to develop similar test range capabilities.

Business Process Reengineering

By reengineering our business processes, we have been able to achieve economies while improving the efficiency of our operations. One example can be found at the Air Force's Arnold Engineering Development Center in Tennessee, where they have reengineered processes to reduce wind tunnel cycle time. Test productivity has been increased by 40 percent, lost test time has been reduced from 20 percent to less than 5 percent, and model installation time has been reduced from 2 days to less than 20 hours.

Another example is our effort to standardize and reuse threat representations. Standard threat representation models are key to improving not only the test and evaluation process, but the acquisition process as well. Contractors will be able to design and test against the same threats as will be used in government-conducted test and evaluation. And all weapons can be tested against common threat representations, thus providing consistent evaluation that can be correlated among systems-of-systems.

A third example of business process reengineering is the Army's efforts to consolidate their operational and developmental test and evaluation functions under one organization. This will afford more opportunities to combine operational and developmental tests. It will also permit the use of the same instrumentation and the reuse of data for different purposes resulting in shorter and less expensive tests.

Facilities

T&E Ranges and Facilities

The third element of T&E infrastructure is facilities. Figure 7 shows the locations of DoD's major T&E ranges and facilities. They comprise the Major Range and Test Facility Base (MRTFB) and are operated and maintained under uniform guidelines to provide T&E support to DoD components responsible for developing or operating defense materiel and weapons systems. MRTFB ranges are sized to enable DoD to safely and effectively test the capabilities of a variety of advanced and highly capable weapon systems in environments representative of conditions found around the world. The MRTFB is a national asset available for multi-Service use. For example, the Army's White Sands Missile Range conducts about 60 percent of its T&E work in support of other Services. The Naval Air Warfare Center Weapons Division at China Lake, CA, provides test support to other Services in areas such as tactical missile and rocket systems testing. The Air Force's Arnold Engineering Development Center in Tennessee supports the Navy's aircraft engine testing. The MRTFB is comprised of essential, often one of a kind, test facilities and ranges.

Some training ranges are also used for T&E and experimentation. Training ranges offer the T&E community a cost-effective opportunity to conduct OT&E under realistic conditions. Training events capitalize on the interplay between actual warriors (as opposed to test specialists or engineers), the system being tested, and other operational systems. They add a degree of realism that is difficult and costly to achieve otherwise. Employing a system being tested as part of training exercise also affords operational users early insight into proposed systems for use in assessing the value they add. This is valuable to the test community, and T&E is conducted so as not to interfere with the training objectives of such exercises.

BRAC Actions and Downsizing Have Reduced the Number of T&E Facilities

A number of T&E facilities and installations have been closed, consolidated, or realigned under the Base Closure and Realignment (referred to as BRAC) process and other downsizing initiatives. In addition to the BRAC actions, reductions in T&E have produced savings equivalent to another 16 BRAC base closures. The major closures, consolidations, and realignments undertaken in the T&E community between 1988 and 1998 are listed below.
 
  Closures

• Jefferson Proving Ground, IN


• Fort Hunter-Liggett, CA


• Naval Air Propulsion Center, Trenton, NJ


• Open Water Test Facility, Oreland, PA

• Army restructured from nine Major Test Centers down to six:
• Tropic Test Center and Cold Regions Test Center consolidated as Test Directorates under Yuma Proving Ground
• Electronic Proving Ground consolidated as a Test Directorate under White Sands Missile Range
• Army Operational Test and Evaluation Command formed by consolidating:
• Operational Test and Evaluation Agency (OTEA)
• Test and Experimentation Command (TEXCOM)
• Operational Threat Support Activity (OTSA)
• Navy consolidated technical activities into a combined RDT&E infrastructure:
• 13 RDT&E sites closed
• 27 RDT&E tenant activities closed
• 34 Commands associated with Department of Navy technical efforts eliminated
• Navy closed the RDT&E Center at White Oak, MD, and consolidated management of required assets under the Air Force's Arnold Engineering Development Center
• Air Force reduced test aircraft inventory by 50 percent
• Air Force consolidated test assets:
• Dissolved 4950th Test Wing and consolidated residual assets at Edwards AFB
• Disestablished Electromagnetic Test Environment (EMTE) and will consolidate required test assets at Edwards AFB
• Closed REDCAP and will provide required test assets and necessary support equipment at Edwards AFB
• Air Force transferred management of the Utah Test and Training Range to the Air Combat Command

Reductions in Funding for MRTFB Take Their Toll

As shown in Figure 8, reductions in total MRTFB operating and investment funding between FY90-FY99 were approximately $5.8 billion. For the most part, the Military Departments made these reductions in reduced requests to Congress for RDT&E funding for T&E. Since FY90, the Military Departments have reduced their RDT&E budget requests for test and evaluation support and investment by a cumulative total of almost $4 billion. Congress further reduced the requests by less than $100 million cumulatively over the same period. Additional reductions were made in the military personnel accounts supporting T&E, as discussed earlier.

The MRTFB operating and investment budgets in FY99 are about $1.1 billion less than they were in FY90, a 34 percent reduction. This reduction will result in savings to the DoD of approximately $1.1 billion per year¾ the equivalent dollar saving realized from sixteen base closures within the BRAC process. Further, these reductions did not come with the up-front costs associated with BRAC closures. The typical BRAC closure of a single base saves about $60 million after up-front costs of approximately $210 million.

Some of these savings were generated by closing or mothballing T&E capabilities while others were enabled by efficiency gains resulting from investment in improved capabilities with lower operating costs. However, others had to be absorbed by dramatically reducing funding for remaining facilities and ranges. This has made it difficult at best to keep our remaining test and evaluation facilities viable.

T&E Facilities Are Aging

The average age of our T&E facilities is well over 40 years. Most of these facilities were built in the early stages of the Cold War. More than two-thirds of them are over 30 years old, as shown in Figure 9.

During the last 20 years, DoD's investment rate for T&E facilities has been less than one-third the rate of investment in private industry and an order of magnitude below the investment rate for high-technology industries. Military Construction funding for T&E facilities at the MRTFB is down 65 percent since 1990. Our current investment level for military construction equates to a replacement rate of 500 years compared to industry rates of 20 to 40 years. Overall, investment funding is down by 39 percent since FY90.

In real terms, that means that T&E facilities at places like Arnold Engineering Development Center in Tennessee face declining availability and maintainability. For example, 35 percent of Arnold's aging infrastructure experiences significant loss of capability in a given year. Another 43 percent experiences limited loss of mission capability, while only 22 percent experiences only marginal to no loss of mission capability. At Arnold, two old and inefficient electric induction motors driving the 16-foot transonic and supersonic tunnels' compressors are the Achilles-heel of the Propulsion Wind Tunnel facility's operations. These motors, designed almost 50 years ago, are not environmentally friendly and tunnel operations must be halted whenever they fail.

Kwajalein Missile Range is another example of a facility deteriorating because of inadequate operating and maintenance funds. The picture on the left (below) depicts the large hole that developed between the first and second floors of the installation hospital as a result of a small crack in the floor going unrepaired. The picture on the right (below) depicts the collapse of a 60-foot ledge on one of the range buildings. In addition to the problems pictured, living quarters (trailers) obtained during the 1960s have passed their useful life and must be replaced. They no longer provide safe, acceptable housing.

Unique Facilities Are Threatened Because of Inadequate Resources

In accordance with DoD policy, T&E managers have entered into "Reliance" agreements that commit them to maintaining a unique capability for DoD-wide use. "Reliance" agreements were designed to enable the Services to eliminate duplication while ensuring the availability of critical T&E capabilities. Yet, dramatic reductions in funding are making it difficult for the Services to support some of these T&E capabilities.

Resource reductions are forcing T&E managers to make difficult and, sometimes, unwise decisions to close or reduce funding for facilities that lack constituency, primarily support other Services, or have lower utilization levels. We have already mentioned the physical state that Kwajalein Missile Range is in because of huge funding reductions. The Army has reduced operating and investment funding for the range by 40 percent¾ a level that may jeopardize its future viability. No significant improvement and modernization funding is currently programmed by the Army for the range. As a result, the range's Target Resolution and Discrimination Experiment (TRADEX) will be shut down, a move that will adversely affect new foreign launch and space surveillance missions and pose new risks to the Ballistic Missile Defense Organization's test programs. Kwajalein Missile Range is our only test range compliant with the Anti-Ballistic Missile Treaty, and we cannot afford to lose the capabilities it provides for operational testing of long-range missile defense systems. Reduced capability at Kwajalein will adversely affect testing operations originating at Vandenberg Air Force Base and the Naval Air Warfare Center Weapons Division in California. Likewise, the mission of the Space and Missile Defense Command in Huntsville, AL, would also suffer.

The Army's reductions in funding forced them to recommend closure of the Army Pulse Radiation Facility at Aberdeen Test Center, MD. The Army recommendation to close this truly unique facility was based on the facility's low utilization. While the utilization level may have been low, the facility is considered to be a national asset and is critical to ascertaining the hardness of systems to combined nuclear effects. Replacing the facility would require an investment of over $100 million for construction and several years of lead-time to obtain environmental assessments and state and local permits.

Plans for Kwajalein Missile Range and Aberdeen Pulsed Reactor Facility are but two examples of the difficult decisions T&E managers must make due to resource constraints and internal Service pressures.

My office is particularly concerned about two other test ranges whose unique environmental testing capabilities must remain available to support future T&E activities.

The first is the Cold Regions Test Center located at Fort Greely, AK. As a result of 1995 Base Realignment and Closure decisions, the Cold Regions Test Center at Fort Greely, AK, will be available in the future only on a safari basis. Fort Greely is unique, having foliage like North Korea in the summer, extended cold climate conditions for dynamic operations in the winter, and the ability to conduct long-range indirect fire testing throughout the year.

The second is the Tropic Test Center in Panama. With the termination of the Panama Canal treaty, the Tropic Test Center is being closed. The requirement remains for realistic testing in a tropic environment with concurrent heat, humidity, solar radiation, rain, and salt in a jungle environment containing fungus, bacteria, and insects. We will monitor the Army's progress in reconstituting this capability at a new site.
 
 

WILL THE INFRASTRUCTURE BE ABLE TO GET THE JOB DONE?

Continuing Resource Reductions Adversely Affect T&E's Ability to Support Acquisition Programs

The declines in T&E funding and the T&E workforce and consequently T&E capacity have been severe at a time when workload has remained relatively stable. The short-term impacts of such declines are not easy to quantify because they are not always immediately apparent. An increase in weapon system program cost, for example, might be caused by schedule delays in the test program, or an increase in program risk might be caused by the lack of availability of a test capability or technology. The impact is there whether immediately apparent or not, and the damage is cumulative. The longer-term impacts effect not only the T&E community but also the warfighter.

In the end, it is the warfighter who loses by being issued inadequately tested weapons systems. The lives and effectiveness of our combat forces are too important for DoD to risk having complex weapon systems that have not been thoroughly tested. DoD must have T&E capability that is unquestionably effective. In addition, it must have the optimal capacity and capability to support all development programs, and their increasingly complex requirements. Reductions in operating funds can be sustained only if funds are provided for new investments and process improvements to improve efficiency and lower costs. If funds for investments and new processes are also reduced, our ability to support acquisition programs will slowly erode.

Our inability to modernize the test facilities and ranges in an affordable and timely manner and adequately fund institutional budgets affect acquisition programs in the following ways:

• The cost to test customers increases, forcing them to limit the scope or number of tests thereby increasing risk. Funding and manpower reductions that cannot be absorbed through efficiency gains and reduced cost force MRTFB activities to increase customer charges during the fiscal year when budgets are fixed.
• The time it takes to test systems increases, resulting in cycle time impacts on programs. Access to and availability of older facilities that are more difficult to maintain introduces delays in all programs competing for access to these facilities.
• The overall cost of tests increases because of the lack of investment to implement cost-effective T&E methodologies and to improve the productivity of aging facilities.
• The amount of risk increases because test and measurement capabilities lag behind the technologies being tested. Inadequate capabilities to measure and produce data in advanced weapon programs increase the risks associated with determining the effectiveness, suitability, and vulnerability of that system.

A second example is the High-Speed Sled Test Track at Holloman Air Force Base in New Mexico, which has experienced a number of rail failures over the past few months. In addition to the recent failures, the sled track facility has had several failures over the last few years that were attributed to aging and fatigue. The facility was closed for significant maintenance about a year ago. Over the past 12 months both the PAC-3 and Navy Theater Ballistic Missile Defense programs have suffered costly delays due to problems at the sled track facility. The Holloman sled track is the only facility with the capability to test the high velocities required to estimate lethality for theater ballistic missiles. Each ballistic missile sled test stresses the track to its maximum capability, thus accelerating normal wear and tear on the facility.

A third example involves Arnold Engineering Development Center. In a recent incident, tests on the F119 engine for the F-22 aircraft and the F110 engine for the F-15 aircraft were delayed because no spare parts were available to repair exhauster electric motor failures. At Arnold, equipment failures frequently delay critical development schedules. A $65 million sustainment program is needed to correct a number of deficiencies there.

Another example involves the VX-9 Squadron from the Naval Air Warfare Center Aircraft Division at Patuxent River, MD. The squadron was unable to support all of its programs concurrently, including the F/A-18E/F, because of inadequate funding for spare parts and personnel. As a result, support to the Standoff Land Attack Missile- Expanded Response (SLAM-ER) and the ALR-67(V3) was delayed for one and a half months thereby delaying those acquisition programs.

The Navy has been forced to slow development and fielding of T&E capability to support new weapons programs because of an approximate 60 percent reduction in Navy T&E Improvement and Modernization funding. In addition, the Navy's 40 percent reduction in target funding has led to rationing of targets and less realism in representation of threat systems. The Army's reduction of more than 70 percent in threat simulator funding is resulting in less threat-representative targets and threat simulations and more risk that weapon systems will not perform in combat as envisioned.

Lack of sufficient institutional funding to keep the doors of all of our test facilities open has resulted in shut downs and mothballing of many test capabilities that are no longer available even for surge use. Backscatter measurement capabilities at the Air Force's Radar Target Scatter (RATSCAT) test facility located on the White Sands Missile Range, New Mexico, and various radar and optics systems at Eglin AFB, Florida, have been shut down. The Air Force has successfully cut the number of test support aircraft in half. The remaining test support aircraft, which support acquisition programs at both Edwards AFB, California and Eglin AFB, Florida, provide little elasticity in responding to acquisition program requirements. Users of these facilities will now incur additional costs and time delays while needed capabilities are replaced or taken out of mothball status and restored to operational status.

T&E Capacity Is Misunderstood

Despite significant closures, consolidations, and realignments of T&E facilities over the last 20 years, the incorrect perception persists that there is still excessive T&E capacity. Three years ago it was reported that the excess capacity for T&E was 52 percent, and many have inferred from this that T&E infrastructure still should be reduced. This percentage is not valid and was derived from the 1995 BRAC Cross-Service Analysis estimate for air vehicles, electronic combat, and armaments/weapons testing. The BRAC '95, excess capacity estimate for T&E centers was based on two determinants: (1) maximum capacity that was experienced between 1985 and 1993; and (2) the projection that workload would decrease by 28 percent by 1998. This workload reduction did not occur; rather, the workload has only been moderately reduced from the peak acquisition budget periods in the 1980s.

In some cases, the maximum capacity for a test facility used in the study included twenty 8-hour shifts per week and in most cases depended upon a high manning level to support this peak workload. By 1999, T&E manpower reductions will result in 25 percent fewer people than were available at the peak periods used in the study. Since the capacity of our T&E facilities is largely driven by the number of people available to operate these facilities, the capacity estimate used to determine excess capacity in the BRAC '95 study was overstated.

We should be optimizing the overall acquisition process not just T&E capacity. If our customers have to wait, their cost is high compared to the cost of maintaining the T&E facilities. We must measure our capacity on the basis of how well we meet the demands of our customers. We should instead use the kinds of measures used by other service-oriented organizations such as gas stations, public transportation services, and fire and public safety services.

For example, gas stations do not compute the maximum amount of gas that can be pumped from a pump in a 24-hour period and divide that number into the amount of expected gas sales to determine the number of pumps required. That would result in lost profit for the gas stations because customers do not arrive evenly over the 24 hours. Instead, gas station owners determine the amount of capacity necessary to satisfy their customers and their customers time and schedules.

For service organizations such as these, the consequences of not meeting workload demands are significant. The same is true for T&E. T&E users, like buyers of gas, and riders of public transportation, view their time as valuable-more valuable than the service provider's time. For that reason, we cannot be in the mode of optimizing T&E operations; we must instead optimize cost and schedule in the overall acquisition process.

T&E Infrastructure Is the Subject of Continuous Study

For the past ten years, the DoD T&E infrastructure has been the subject of a number of studies to determine whether further reductions, consolidations, streamlining, or business process reengineering could be achieved. For such a small proportion (1.6 percent) of the DoD infrastructure costs, it has received much attention. Figure 10 shows the breakout of the annual $115.5 billion DoD infrastructure. T&E infrastructure represents approximately 20 percent of the $10.5 billion acquisition infrastructure costs and only 1.6 percent of total DoD infrastructure costs.

Some of the more significant T&E infrastructure studies are identified in Figure 11

The Defense Management Review Decision (DMRD) 922 reduced T&E and laboratory funding by $3.4 billion over four years and led to T&E and Laboratory Reliance. The Joint Chiefs of Staff Role and Missions study led to the creation of the T&E Executive Agent-Board of Directors. Other studies include the Commission on Roles and Missions, Quadrennial Defense Review, Defense Reform Initiative, Vision 21, and Section 912 Defense Science Board Task Force on Acquisition Workforce, to name a few. Currently, T&E is the subject of three follow-on studies to the Secretary of Defense Report to Congress: Actions to Accelerate the Movement to the New Workforce Vision, dated April 1, 1998: (1) the Defense Science Board Task Force on Integrated Test and Evaluation; (2) Streamlining the RDT&E Infrastructure; and (3) the Defense Science Board Task Force for Non-Traditional Sources for RDT&E.

In addition, there are several standing organizations that look at T&E infrastructure. The T&E Executive Agent Board of Directors, a corporate board at the military department level, provides central oversight of Service T&E programs to ensure efficiency and inhibit unnecessary duplication while maintaining critical T&E capability. The Range Commanders Council, a corporate structure at the range or activity level, continuously looks for opportunities to reduce cost, increase efficiency, and benefit from improvements and "lessons learned" across the ranges.
 
 

WHAT DO WE NEED TO DO NOW?

Modernize Our Remaining Infrastructure

In addition to the problems we face today-sustaining the T&E mission with fewer resources and a withering infrastructure-we are also concerned about our ability to test future weapons systems. Weapons of the future will draw on new, more complex technologies and we will need new, more advanced equipment to test them. Current levels of investment will be insufficient to meet future requirements.

Our test capabilities will need to support future acquisition programs involving such technologies and capabilities as: advanced sensors, real-time data processing, unmanned operations, urban and terrorist warfighting, detection and destruction of weapons of mass destruction in hardened targets with minimum collateral effects, massive communications and data handling, and advanced aircraft and munitions.

Present T&E equipment and facilities are inadequate to accommodate testing of such technologies. Given present plans to operate with less funding and manpower, investment in modernization is critical. Efficient, cost-effective facilities are needed to replace those facilities that are still labor-intensive to operate. State-of the art instrumentation is needed to replace outdated test equipment. In addition, test facilities that satisfy joint-Service needs are needed.

Support the T&E Corporate Investment Strategy

In coordination with the Military Departments and the Director of Test, Systems Engineering, and Evaluation, we published "A Corporate Strategy for Test and Evaluation Resources and Investments." It includes initiatives for strengthening the T&E infrastructure, integrating the OSD and Military Department investment processes, and forming strategic partnerships with other government and industry communities to share technology and resources. "Investing as a corporation" is seen as key to stretching investment budgets.

One important source of investment is the Central Test and Evaluation Investment Program (CTEIP). While funding for investments by the Military Departments has been decreasing, the OSD-administered CTEIP has relieved some of the investment shortfalls. CTEIP supports the development of new test and evaluation capabilities to correct deficiencies and to improve productivity. It focuses primarily on long-lead-time capabilities requiring advances in technology that meet multi-Service requirements, and promote interoperability and interconnectivity among test centers and facilities. With approximately 60 CTEIP projects underway or completing this year, this program has proved to be an important source of investments in the T&E infrastructure.

An example of a successful past CTEIP investment is the Aerial Cable Test Facility, located at White Sands Missile Range. It continues to provide significant payback for conducting operational test and evaluation. The facility uses a cable suspended in a canyon to provide open-sky backgrounds for targets with threat-representative signatures for a fraction of the cost of flying an aerial target to obtain the same signature. It is also used to test missile warning and countermeasures systems.

Another successful CTEIP investment was the Common Airborne Instrumentation System (CAIS). This program is already reducing the need for aircraft-unique instrumentation and can be used at various test ranges without having to acquire different instrumentation for each.

The CTEIP also supported the development of the Smart Munitions Test Suite (SMTS). The SMTS is deployable to various ranges as needed to provide timely space-position information on multiple missiles, munitions, and sub-munitions involved in a test. SMTS is especially useful in tracking large numbers of objects simultaneously as required in the ATACMS Block 2 BAT testing.

Present CTEIP investment programs are expected to provide extensive T&E capabilities and large paybacks. One example is a hardened subminiature telemetry and sensor system development for immediate application to artillery rounds fired at Yuma Proving Ground, AZ illustrated in Figure 12. Another is a podded digital recorder that will be able to collect and store data from 1553 avionics data buses to permit users to accurately assess the electronic warfare suite testing of threat warning, jamming, and other countermeasures in a near real-world environment at multiple test and training ranges.

CTEIP also funds a project titled Resource Enhancement Program which responds to critical operational test capability shortfalls in the near-term (less than three years). Six Resource Enhancement Program projects were completed with FY98 funding. The Aerial Target launch ship, one of these projects, will provide a platform for mobile, unmanned launch of aerial targets on sea test ranges. The Global Positioning System Jammers, another of these projects will produce a GPS Jammer simulator that simulates jamming of both L1 and L2 satellite downlinks without disruption to civilian users or range instrumentation. Other projects completed with FY98 funding include: Large Scale ECM Aircraft Modification, Missile on a Mountain, Modeling and Simulation Support to OT&E, and Vulnerability Assessment. Nine continuing projects are scheduled to receive funding in fiscal year 1999 including: Advanced Threat Instrumentation, Advanced Missile Instrumentation Package, Ultraviolet Stimulator, Laser Observation Test and Evaluation Capability, and Realistic Operational Communications Scenarios.

We must continue to support these CTEIP projects and other programs that integrate the strategy of "investing as a corporation."

Invest in Technologies for Joint Vision 2010 Requirements

CTEIP investment funds are being applied in areas with the most immediate deficiencies. The next step is to begin investment in capabilities that address future requirements of Joint Vision 2010. We need the support of Office of the Secretary of Defense, the Military Departments, and Congress to do that.

The following paragraphs give examples of needed capabilities that are not currently funded and test capability development programs that are insufficiently funded.

Large-scale, integrated Command, Control, Communications, Computer, Intelligence, Surveillance, and Reconnaissance (C4ISR) systems testing. A requirement exists for more advanced capability to conduct large-scale integrated C4ISR systems testing. The required capability includes additional models and simulations to provide realistic test environments, instrumentation, and data systems to collect and measure performance.

Individual testing instrumentation for real-time data relay and casualty assessments. Embedded, non-intrusive instrumentation for the individual soldier or marine is needed that provides real-time data that also can be used for real-time casualty assessments and battle damage computations. The size, weight, and battery life of current ground troop instrumentation affects test realism, especially when added to the equipment the troops are already required to carry.

Large-missile, reconfigurable, installed-systems hardware-in-the-loop test facility. A reconfigurable hardware-in-the-loop facility is needed to support all phases of missile testing. The facility would need scene- and flight-generation modeling and simulation capability; a large missile flight table; anechoic chamber; and certified live-round component and system level testing, live fire test in launch, flight and transport vibrations/shock and temperature environments.

Radio frequency spectrum. CTEIP is currently funding a project, Advanced Range Telemetry, to begin advancing technology to compensate for loss of frequency spectrum. In addition, real-time selective data retrieval capability from on-board data recording equipment is needed to optimize frequency spectrum use. Parts of the radio frequency spectrum used by the DoD T&E infrastructure have been sold to the private sector. Since 1989, 83 megahertz of the frequencies used for test and evaluation-13 megahertz in the past year-have been either eliminated or restricted in emitted power. This loss of frequency spectrum is reducing our ability to conduct simultaneous test missions as well as restricting some of the larger test missions.

Submarine quieting demonstrations and testing. An upgrade to test facilities at the Acoustic Research Detachment at Lake Pend Oreille in Bayview, ID, is needed to keep pace with submarine stealth technologies. Existing facilities and planned upgrades will not be adequate to measure the low acoustic levels and higher frequencies needed to fully evaluate future technologies for advanced submarine quieting. Network connectivity and software tools are also inadequate for processing the expected quantities of data in 2001 and later.

Accurate measurements of miss distances. A continuing deficiency exists in obtaining accurate miss distance measurements of non-cooperative air-to-air and surface-to-air missiles and their targets. Endgame vector scoring of vehicles with closing velocities up to 20,000 feet/second and altitudes from sea level to the exoatmosphere remains a critical need for applications in theater missile defense, precision guided missiles, and electronic warfare.

Fly-out model validation. The use of fly-out models to represent threat missile launches and trajectory performance can help avoid cost and time delays in weapon system development programs and early operational assessments. However, most existing models have not been validated against actual threat performance because of the expense of performing the validation. Before these models can be relied on for data collection and analysis purposes, they must be validated.

Realistic air-to-air, closed-loop electronic countermeasures. This capability is essential for electronic warfare effectiveness testing. Current open-loop capabilities cannot provide the realism of actual threats because they do not react to countermeasures as an actual system would. Closed-loop capabilities would permit the threat to alter sensor modes, conduct maneuvers, or enact electronic counter-countermeasures as an actual threat system would.

Computational Fluid Dynamics. Continuing developments are needed in this area to improve predictive capabilities for use during weapon development and design stages. The goal is to analyze more complex physics and nonlinear interactions among systems and system components with higher fidelity. Such modeling and simulation complements ground and flight testing to make them more effective and serves to reduce development costs and schedules across the flight spectrum.

Extended-range testing. To address the lack of sufficient land and air areas to test long-range over-land weapon systems, investments are needed to provide a surrogate capability to simulate the conditions of extended instrumented ranges.

Water-launched, long-range, submunition land-impact missile test ranges. Ranges are required to support littoral warfare programs involving submunitions dispensed against land targets fired from missiles launched at sea from distances of 60 km. Programs such as the Extended Range Guided Missile within the Standard Missile Program require scoring of dispersion of multiple munitions at two points-over-water release and ground impact. No current ranges can meet the full operational requirements because of size, safety, and environmental constraints.

Portable shallow water acoustic range. A 100-square-mile shallow underwater range would help test shallow water undersea warfare applications. Portability would allow the range to be relocated to various sea locations as needed for varying acoustic signature and underwater terrain conditions.

Threat-representative targets with multispectral signatures for realistic test conditions. Inadequate funds are available for developing the variety and procuring the quantities of targets necessary for sufficient test and evaluation. Targets of adequate signature characteristics are critical to validating that weapon sensors can differentiate targets from friendly combatants and perform as required for the circumstances. Land warfare target deficiencies include those for Stinger missile quality assurance, for air defense gunners against unmanned air vehicles, and for air defense of short- and medium-range tactical ballistic missiles. In naval warfare, targets are needed that will replicate diesel submarines, and capabilities are needed that will emulate conditions for testing the Versatile Exercise Mine System. In air warfare, affordable expendable targets are needed that have high-visual-signature fidelity as well as challenge the dual-mode seekers of antiradiation missiles.

Information warfare technologies testing. Capability is needed to test information warfare technologies for spoofing, anti- and counter-spoofing, resistance to viruses, accessibility and detection of hacking and eavesdropping, procedures for operations security, and other vulnerabilities.

Vehicle-in-the-loop automotive test simulator for roadway conditions. This simulator would provide information about the abilities of combat vehicles to operate within and beyond the established safety envelope while engaging moving targets. Using such a simulator would extend knowledge of how such vehicles can be used without endangering human operators. Investments are needed in the capabilities discussed above to meet future requirements.
 
 

CONCLUSION

The next few years will be a critical time for the Department's T&E infrastructure. The resources committed to modernization of T&E capabilities will determine the costs and, to some extent, risk levels of our future weapon systems. We are proud of what we have been able to accomplish to date. We have made our investments in test resources count by improving productivity while requiring less manpower. We have reengineered our processes to the degree feasible given the lack of new investment. We have contributed more than our share to the Department's downsizing in military, civilian, and contractor manpower, and in facility closures and mothballing. Our T&E infrastructure has continued to provide quality T&E support to weapon system acquisition programs.

Future opportunities will be limited by the decisions we make about T&E resources and modernization over the next few years. We need to recognize that for all practical purposes, reductions in T&E have reached their limit, and that future budgets must be based on customer needs, not on arbitrary targets for more cuts. We need to focus¾ not on "excess capacity"¾ but on the needs of our customers¾ the acquisition programs¾ and how to best support them. We need to recognize that cuts in T&E are driving up overall acquisition costs and cycle times. Relatively small investments in T&E can result in cost savings for users of T&E Services.

It is time to look toward the future. If we do not invest adequate funds to modernize the T&E infrastructure, we will continue to see a degradation in capability that will show up as increased weapon system costs and risks and increased delays in getting weapons that work into the hands of the warfighters. We need to provide quality T&E support for weapon systems that are becoming technologically far more advanced than our capability to ensure that they are operationally effective and suitable. And we need new investment in the technology, equipment, simulators, and instrumentation that will allow us to operate¾ and interoperate¾ in and across test and training ranges in the future.

We must increase investment to modernize T&E facilities if we are to provide quality T&E support to acquisition programs. And we must provide adequate funding to keep the doors of unique and critical facilities open.