Military Personnel: Actions Needed to Achieve Greater Results From Air Force Family Need Assessments (Letter Report, 03/08/2001, GAO/GAO-01-80)
To meet the needs of military families so that they can adapt and
function effectively in military life, the Department of Defense (DOD)
funds a wide variety of personnel and family support programs. These
programs use a combination of direct assistance and information and
referral to help with crises, deployments, moves, child care, personal
financial management, parenting, and transition out of the service. So
that funds are efficiently distributed to those with the greatest need,
DOD requires each service to assess the needs of military personnel and
their families and their use of family support programs. GAO examined
(1) how the Air Force determines the needs of its military families, (2)
what type of information about military family needs the Air Force
obtains, and (3) how information from needs assessments affects the
allocation of funding for Air Force family support programs. GAO found
that the Air Force conducts assessments of its personnel's needs every
two years and bases its assessments on a random sample survey. However,
the Air Force process does not meet other DOD criteria generally
recognized as important for needs assessments, including the use of
benchmarks to determine if needs have changed or emerged. Additionally,
the Air Force assessments cover needs related to deployments and
mobility, family advocacy, family life education, child care, parenting
education, family members' employment, transitions and relocations,
financial management, and dependents with special needs. Finally, GAO
found that the Air Force has inconsistently used assessment data to
budget for family programs.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GAO-01-80
TITLE: Military Personnel: Actions Needed to Achieve Greater
Results From Air Force Family Need Assessments
DATE: 03/08/2001
SUBJECT: Air Force personnel
Quality of life
Military dependents
Surveys
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Testimony. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-01-80
Report to the Honorable George R. Nethercutt, Jr., House of Representatives
March 2001 MILITARY PERSONNEL
Actions Needed to Achieve Greater Results From Air Force Family Need
Assessments
GAO- 01- 80
Let ter
March 8, 2001 The Honorable George R. Nethercutt, Jr. House of
Representatives
Dear Mr. Nethercutt: The proportion of military personnel with spouses and
children has risen dramatically since the military eliminated the draft and
established an all- volunteer force. For example, 60 percent of Gulf War
veterans versus 16 percent of Vietnam veterans were married with children
during their service in these conflicts. The demands and stress of military
life include
frequent time away from family, recurrent moves that can disrupt family and
social support networks, and the threat of family members' death or injury
due to military service.
To meet the needs of military families so that they can adapt and function
effectively in these circumstances, the Department of Defense (DOD) funds a
wide variety of personnel and family support programs. These programs use a
combination of direct assistance and information and referral to help with
crises, deployments, moves, child care, personal financial management,
parenting, and transition out of the service. In fiscal year 2000, $739
million was appropriated for such family support programs.
So that funds are efficiently distributed to those with the greatest need,
DOD requires each service to assess the needs of military personnel and
their families and their use of family support programs. In response to your
request for information on the needs of military families, and as agreed
with your office, this report focuses on the Air Force's family needs
assessment process. Specifically, we determined (1) how the Air Force
determines the needs of its military families, (2) what type of information
about military family needs the Air Force obtains, and (3) how information
from needs assessments affects the allocation of funding for Air Force
family support programs. In evaluating the Air Force's needs assessment
process, we used criteria
identified by research literature and experts as important in conducting
need assessments that are used for program planning and resource decisions.
These include the use of (1) a random sample survey to assess needs from the
perspectives of servicemembers and their families;
(2) benchmarks to define when needs have increased, decreased, or emerged;
(3) a framework to interpret the meaning of needs assessment results; (4) a
strategy to determine which needs will receive priority for funding; and (5)
information on the type and capacity of off base community and family
support services. Results in Brief The Air Force has conducted assessments
of its personnel's needs for family programs, as required by DOD
regulations, every 2 years since 1993.
It bases its assessments on a random sample survey distributed to active
duty personnel and civilian spouses and has established a means of i nt
erpreti ng the result s of t he survey. However, t his Ai r Force process
does not meet other criteria generally recognized as important for needs
assessments, including the use of benchmarks to determine whether needs have
changed or emerged, a strategy to determine which needs will receive
priority for funding, and information on the type and capacity of off base,
or civilian, community and family support services. Neither the DOD nor Air
Force family need assessment regulations include these criteria. The value
and accuracy of the Air Force's needs assessments for designing, planning,
continuing, expanding, or terminating family support programs, as required
by DOD regulation, may consequently be diminished. The Air Force's
assessments cover needs related to deployments and mobility, family advocacy
(i. e., domestic violence treatment and prevention), family life education,
child care, parenting education, family
members' employment, transitions and relocations, personal financial
management, and dependents with special needs. Although DOD's guidance
indicates that military families generally have needs in these areas, its
regulations do not require that any particular issues or aspects of military
family life be measured in needs assessments, and no single source
conclusively demonstrates or defines the needs of military families. Data
from the Air Force- wide 2000 needs assessment indicate that needs for
support related to deployments and separations are far more commonly
reported than needs associated with other family support. For example, more
than 50 percent of both active duty members and spouses surveyed said that
they needed to be able to make “morale” telephone calls during
deployments and separations. 1 1 “Morale” telephone calls are
placed to the deployed spouse or parent, without charge, with the help of a
military operator.
Although the Air Force has used needs assessment data to change family
program policies and to conduct program planning and some training, it has
inconsistently used the results to budget for family programs. DOD
regulation states that such results should serve as the basis for the
design, planning, continuation, expansion, or termination of family support
services. On one hand, an Air Force child care program used data from the
needs assessment to determine which of its bases had priority for a portion
of $1 million for a new initiative to provide extended day care services. On
the other hand, Air Force representatives for the family advocacy and family
support center programs said that needs assessment data is not typically
connected to family program funding decisions. Rather, family advocacy
budgets are based on the number of people in the area that they
expect to serve and an indicator of how long it takes to review domestic
violence cases. In light of DOD policy, we are recommending that the
Secretary of Defense direct the Secretary of the Air Force to use criteria
we have cited in this report in conducting family needs assessments and to
use the results of the
needs assessment process to budget for family support programs. To ensure
that all the services conduct assessments that are useful for designing,
planning, expanding, continuing, or terminating family support programs, we
are also recommending that the Secretary of Defense include the criteria we
have cited in its military family need assessment regulations and guidance.
Background Twelve years after the end of the military draft, Congress passed
the
Military Family Act of 1985 (Title VIII of the Department of Defense
Authorization Act for Fiscal Year 1986, Public Law 99- 145). The law
established an Office of Family Policy in the Office of the Secretary of
Defense to coordinate programs and activities of the military departments
relating to military families. The law also gives DOD authority to make
recommendations to the secretaries of the military departments with respect
to programs and policies regarding military families. 2 Since the passage of
the act, increasing numbers of federal programs, contracts, laws, and
regulations have been directed at providing and 2 The Military Family Act of
1985 was revised in 1996 (section 568 of the National Defense
Authorization Act for Fiscal Year 1996, Public Law 104- 106, as amended).
The revision did not affect the described provisions pertaining to the
Office of Family Policy.
enforcing support for military family members. In fiscal year 2000, Congress
appropriated $739 million for DOD's family programs then under the
jurisdiction of DOD's Office of Family Policy. These funds provide extensive
information and referral services, counseling services, financial management
education, spouse employment assistance, domestic violence treatment and
prevention, and child care, among other services. These funds also support
260 family support centers that serve as focal points for the delivery of
military family services in the United States and overseas. 3 Nearly 10
years ago, in conjunction with the buildup of DOD's family
support programs, DOD instructed each service to conduct servicewide needs
assessments and to use the results as the basis for designing, planning,
continuing, expanding, or terminating programs. In a 1999 update to this
instruction, DOD reemphasized the need to validate military families' needs
in relation to programs and services to support their
well- being. In the last few years alone, several new personnel and family
support programs and related initiatives have been implemented or are under
consideration by DOD or specific services, including the New Parent Support
Program, the Healthy Parenting Initiative, the Combat Stress Control
program, and increases in counselors in DOD schools (see app. III). In
addition, active duty military and family quality of life issues have
recently been topics of deliberation and study among DOD decisionmakers. For
example:
? In May and June 2000, DOD sponsored two separate forums where,
respectively, selected military personnel and family members and a group of
senior enlisted military personnel were invited to the Pentagon to express
their views on a range of military quality of life issues, including
housing, health care, military readiness, youth services and programs, and
general family support. ? In August 2000, DOD sponsored a family readiness
conference at which the goal was to teach strategies for creating self-
sustaining communities that provide effective family support in day- to- day
living and in times of demanding rates of military activity and crises. 3
Due to reorganization, DOD's Office of Family Policy has jurisdiction over
different programs than it did in fiscal year 2000. Currently, the office
has jurisdiction over family support centers, spouse employment programs,
relocation assistance programs, personal financial management programs,
casualty and mortuary services, volunteer programs, adoption reimbursement,
elder care, and family care plans.
? In its 2000 report to the President and Congress, DOD reports that its
Quality of Life Executive Committee plans to expressly develop a needs
assessment instrument to determine the needs of the top 20 high- tempo units
and military specialties. 4 ? In October 2000, the Secretary of Defense
launched a Military Quality of Life Summit to bring together corporate
executives and senior defense officials to improve servicemembers' quality
of life. A top priority of this partnership is to create job opportunities
for military spouses and servicemembers leaving military service.
Providing the right type and amount of support for military family members
is not a simple undertaking. Military family members outnumber total
uniformed active duty personnel. Of the total number of active duty
personnel and their family members, spouses, children, and adult
“dependents” are about 60 percent (about 2 million), while 40
percent (about 1. 3 million) are married and single active duty uniformed
personnel. The active duty force is largely comprised of young people who
are
managing major personal responsibilities (i. e., marriage and children) and
the demands of military service, which include frequent family separations;
recurrent moves to new locations; and life in a culture that sets high
expectations for personal responsibility, family responsibility, discipline,
and performance. In 1999, 80 percent of the total force was younger than 35,
55 percent of the force was married, 48 percent of military spouses were 30
or younger, and 47 percent of all military members had children. The Air
Force DOD regulations require that the services conduct a family needs
Assessment Process assessment survey and use the results for designing,
planning, continuing, expanding, or terminating family support programs. The
regulations do not Incorporates Some Key
require the services to use specific criteria or to use a specific process,
Elements but Not method, timeline, or reporting format for needs
assessments, and each
Others service implements the regulations differently. The Air Force has
conducted the required needs assessments by obtaining views from a
random sample of active duty servicemembers and civilian spouses. Most
recently, the Air Force offices with administrative responsibility for child
care and youth programs, family advocacy programs, and family support center
programs have contributed to the development of the 4 In 1995, the Secretary
of Defense established the Quality of Life Executive Committee, comprising
senior DOD officials and military leaders, to serve as his principal policy
adviser on quality of life issues.
Air Force- wide needs assessment. The Air Force has established six measures
that it applies to the results of its need assessment surveys to help
determine and assess family needs. These measures represent short- and long-
term goals that the Air Force has established for its family and community
support programs, and they provide the framework for interpreting and
assessing the results of the needs assessment survey. However, the
assessment process does not include other elements recognized in research
literature and by experts as important in conducting assessments that are
used for program planning and resource decisions. The Air Force's Needs The
Air Force's establishment of short- and long- term measures for its
Assessment Process Is Set family support programs is consistent with
practices we have identified as in a Results Management useful in developing
performance plans required in the Government Performance and Results Act. 5
The three short- term measures are: Framework
? Informal Community Connections: what kind of informal networks and
resources (e. g. friends, spouses, coworkers, and neighbors) servicemembers
and their family members use, if any, for personal or family problems and
how supportive these connections are.
? Leadership Support: how supportive various leaders (e. g., unit and wing
leaders and base commanders) are in the lives of the servicemembers and
their families.
? Interagency Collaboration: whether officials from the various base family
support programs work collaboratively to support families and collaborate
with off base community service providers.
The three long- term measures are:
? Personnel Preparedness: whether servicemembers plan to stay in the
military, what their concerns and problems are, how servicemembers and their
families are functioning, and how successful servicemembers are at managing
work and family responsibilities.
5 As we reported in Agency Performance Plans: Examples of Practices Than Can
Improve Usefulness to Decisionmakers (GAO/ GGD/ AIMD- 99- 69, Feb. 1999), an
agency may not see results of its programs for years; thus, intermediate
goals and measures can provide information on interim results and can be
used to show progress or a contribution to intended results.
? Family Adaptation: how well families have adapted to the Air Force way of
life, including how well they work together to solve problems; what the
degree of family conflict is; what parents' concerns about their children
are; and whether and where families seek help when they have
problems.
? Sense of Community: whether servicemembers and their families are
satisfied with the Air Force as a way of life, whether they perceive a
positive sense of community on the base, and whether they use support
services.
During on- site training led by contractors and Air Force family support
officials, family support program staff at each Air Force base
collaboratively review and interpret need assessment results using the six
measures to help determine whether program goals are being met and to
develop an appropriate action plan. This all- day training allows the
professional staff from the base family support program offices and key
military personnel to interact and discuss their views on the meaning and
significance of the need assessment results. In addition, base program staff
acquire skills and knowledge on how to use the needs assessment data to
develop community action plans. Family support professional staff who have
participated in this training believe that the needs assessment data provide
them with a concrete tool to validate adjustments and improvements in base
services. They stated that base leaders who provide
essential support for changes in programs and services attach a great deal
of weight to the needs assessment data.
Assessment Does Not The Air Force's needs assessment process and DOD's
regulations, lack Include All Elements to three elements that would support
program planning and resource Support Program Planning allocation. These
elements are the use of benchmarks to determine and Funding Allocation
whether needs have changed or emerged, a strategy to determine which needs
will receive priority for funding, and information on the type and capacity
of off base, or civilian, community and family support services. Using
Benchmarks to Determine
A primary purpose of need assessments is to identify services a community
Needed Community Services
lacks relative to some generally accepted standard. Some private sector
communities use such standards to assess community needs. For example, they
compare local and statewide indicators of health (e. g., birth rate, teen
pregnancy rate, and percent of uninsured adults); economic well- being (e.
g., unemployment rate, per capita income, and poverty rate); and social
welfare (e. g., percent of residents receiving Medicaid, food stamps, or
other public assistance; rate of child abuse and neglect; and incidence of
child delinquency). By establishing measures of comparison, program
administrators can more accurately determine how well their programs are
doing and the needs of their clients. Without regular assessments of the
needs of servicemembers and their families and the success of the programs
in meeting these needs, the Air Force cannot maintain quality programs. In a
1990 inspection of DOD family centers, the DOD Inspector General criticized
the Department for failing to compare yearly results of needs assessments to
determine whether programs were still needed and how
successful the programs had been. And during our review, Air Force family
support program staff said that it was difficult to evaluate how negative or
positive the results of their needs assessments were because they did not
have a basis for comparison. In addition, they said that base leaders want
to know how their base is doing relative to some standard (e. g., other
similar
Air Force bases or the previous year's results). This information is key to
base leaders when they review and evaluate budget requests for family
support programs. In reporting results to determine needs, the Air Force
considers only the percent of the population that responded a certain way to
an item in the needs assessment survey. 6 For example, members' taking time
off from work to solve family- related problems is an indicator of personnel
readiness that comes from the needs assessment. The random sample survey
done in 2000 indicates that 8 percent of the total active duty
servicemembers had to take either scheduled or unscheduled time off from
work to take care of youth behavioral problems. Without a measure of
comparison, however, it is not possible to determine whether a need has
surfaced or changed. Eight percent may be perceived as a small number that
does not indicate a need and that action is therefore not required. However,
a comparison of the 2000 survey to the prior year's survey (1998)
shows that the percent of active duty personnel who said they took
unscheduled or scheduled time off from work to take care of youth behavioral
problems increased from 3 to 8 percent, or more than 6 In the two prior need
assessments (1995 and 1998), the Air Force reported changes over time on
some need assessment results. For example, some 1995 results were compared
to 1993 results and 1998 results were compared to 1995 results. The 2000
servicewide need assessment reported no such comparisons.
160 percent. 7 Viewing the results from the standpoint of prior year
results- one measure of comparison- may signal cause for attention.
Prioritizing Needs Based on Because funds for some family support programs
are limited and can be
Assessments redirected to other base activities, at the commander's
discretion, it is important for the Air Force to prioritize the results of
the family needs assessment. The Air Force's final report on its needs
assessment includes needs associated with at least 65 family and community
support programs or services and 14 child care- specific services, among
other issues. The majority of these programs and services do not appear to
be required by law and therefore would not have dedicated funding. Rather,
non- mandated programs like these are funded through the base operations
account, which also funds other operations on the base. Depending on the
circumstances, the base commander may take funds that are budgeted for a
family support program and use them elsewhere. 8
A determination of priority may be based on the programs that servicemembers
identify on the survey as most needed. It also may be based on legal
mandates (i. e., programs that are required by law) or overall organization
or strategic goals. In a military environment, if the Air Force
were to prioritize programs based on organizational goals, it would look for
the needs assessment results that directly link to military readiness. For
example, direct implications for readiness are personal or family problems
that require time off from work, problems in meeting family
responsibilities, the need for assistance during deployments and
unaccompanied tours, and risks that servicemembers may be placed in
nondeployable status, because of family matters. 9 7 The 1998 and 2000
servicewide need assessments had the same margin of error. 8 As we reported
in Defense Budget: Real Property Maintenance and Base Operations Fund
Movements (GAO/ T- NSIAD- 00- 101, Mar. 1, 2000), the services have
considerable flexibility in using operations and maintenance funds (which
includes the base operations account), and they move funds in and out of
base operations accounts for different priorities. Base
operations funding is used for services such as utilities, base
communications, snow removal, security, and recreation. 9 Annually, DOD is
required to report information concerning permanent and temporary
limitations on the deployability of servicemembers. DOD has defined a
category of family matters that can result in a servicemember being
classified as temporarily nondeployable, including adoption, humanitarian
deferment issues, physical/ mental handicap of family member, problems with
a family care plan, domestic violence issues, financial stability, and other
unspecified family issues.
Determining Off Base Resources During its need assessment process, the Air
Force does not factor in what for Family Support civilian community support
resources are available, but its survey does ask respondents whether they
received support on or off base. DOD
regulations do not explicitly require the services to assess and factor
civilian resources in, but they do require the services to provide family
support programs only to the extent that local area services are not
available, accessible, affordable, or appropriate to meet the needs of
military families. During our review, DOD family support program officials
said that it is important to link military families with community resources
to meet their needs, if possible. Moreover, private sector family support
organizations advocate the use of surveys to determine the capacity of the
agencies and social service providers in local communities. Unless the Air
Force determines what support services are available in the civilian
community, how many people they can accommodate, and how they are used, its
base family support staff cannot know whether they are
duplicating services, underusing or overwhelming their own programs, or
concentrating funds on the right programs. Nonprofit organizations such as
the United Way and the Young Men's
Christian Association (YMCA) provide assistance to families in civilian
communities. About 1,400 United Way organizations finance youth development
and recreation, parent support and education, and early childhood
development services, among other things. The Armed Services YMCA, with 24
branches servicing the 50 states, provides a range of services to improve
the quality of life for military personnel and their spouses and children.
For example, the Armed Services YMCA in Norfolk, Virginia, provides services
specifically targeted to junior enlisted personnel for social, recreational,
personal, and child development; crisis prevention; and deployment
separation.
Because at least more than 50 percent of Air Force personnel live in the
civilian communities outside of Air Force bases, knowledge of available
family support services would be valuable. Data from the 2000 Air Force
needs assessment indicates that military family members use off base support
services. For example, in 26 of 28 areas of concern related to
children (e. g., abuse by a family member, adjusting to new schools, and
divorce of parents), more civilian military spouses said they sought help
off base. Military families may seek support from civilian community
services because they are more conveniently located. However, according to
active duty personnel and civilian spouses, a leading reason for not using
base
services is the perception that they provide limited confidentiality. For
example, according to the 1998 Air Force needs assessment, more active duty
personnel (20 percent) and civilian spouses (22 percent) said that limited
confidentiality was a barrier to using Air Force family support services
than any other reason, including “too busy,” “stigma of
using
services,” “lack of child care,” “staff not
knowledgeable,” “not interested in services,” and
“inconvenient location,” among others. 10 In addition, we
previously reported that a military dependent seeking counseling on domestic
violence issues cannot be assured that the information provided will not be
disclosed. 11 In focus groups we conducted during this review, military
spouses said that there is a general lack of confidence and trust in the
military family support system. The perception among some military spouses
is that those who seek help from military providers will be labeled as
“problem wives” or “complainers,” although other
spouses indicated that assistance such as provided by the Airman's Attic (i.
e., loaner program for clothing, toys, and household appliances) is a
positive experience and benefit.
Need Assessments The Air Force needs assessment survey includes questions
designed to Provide Information measure needs for crisis assistance,
deployment and mobility support, family advocacy, f ami l y l i f e educat i
on, parent i ng educat i on, i nf ormat i on About the Demands referral and
assistance, family member employment, transition and and Challenges
relocation assistance, personal financial management education and
Associated With information, and help with elder care responsibilities and
family members with physical and/ or mental handicaps. DOD guidance
indicates that Military Life
military family needs are generally met by programs providing assistance in
10 The Air Force did not include a question about possible barriers to the
use of family support programs in the 2000 needs assessment; therefore, the
needs assessment for 1998 provides the most recent data. 11 Military
Dependents: Services Provide Limited Confidentiality in Family Abuse Cases
(GAO/ NSIAD- 00- 127, Apr. 5, 2000).
these areas. 12 However, despite the emphasis in DOD family policy
regulations (i. e., DOD Directive 1342.17, issued Dec. 30, 1988) that
military family research and program evaluation be directed at understanding
the needs of DOD personnel and their families, there is no authoritative
description of military community and family needs across the diverse
military population. Table 1 shows the results from the Air Force's 2000
survey of active duty personnel and civilian spouses regarding the major
needs DOD has identified. 12 DOD regulations addressing these military
family needs concern programs and services
offered primarily through military family support centers. DOD has other
programs designed to support family needs, including, child care and various
morale, welfare, and recreation activities. Although the latter issues may
be touched on in need assessments, these programs are not under the
jurisdiction of the Office of Family Policy, and therefore,
under the regulation we refer to here (DOD Instruction 1342. 22, issued Dec.
30, 1992), they are not required to be included in the needs assessment.
Table 1: Results From the Air Force 2000 Community Needs Assessment Active
duty personnel
Civilian spouses that that reported a need
reported a need Support service or program
(percent) (percent) a
Deployment support
Morale calls 56 54 E- mail communications 51 49 Support during separations
30 37 Calls from unit to family 29 37 Car maintenance help 28 38
Predeployment support 23 17 Newsletters 14 23 Post- deployment assistance 12
7 Help with child care costs 11 13 Family support groups 10 17 Financial
planning 10 7 Emergency financial assistance 9 8 Spouse employment support 8
13 Family care plan help 6 5 Youth supervision 6 9 Family reunion training 4
5 Workshops about family
2 3 life/ parenting Respite care 2 4
Family advocacy b
Family maltreatment services 2 2 Family violence prevention
2 2 information Family life education 4 4
Parenting classes 4 6
Information and referral
Housing referral 23 21 Directory of community 15 23 services/ programs
Family member employment 12 16
assistance Transition assistance c
Retirement employment 5 5 information
(Continued From Previous Page)
Active duty personnel Civilian spouses that
that reported a need reported a need
Support service or program (percent)
(percent) a
Retirement/ separation workshops 7 6
Relocation assistance d
Moving services/ assistance 20 22 Relocation assistance 15 14 Relocation
information 11 14 Financial help with relocation
6 6 expenses Sponsorship assistance 14 10
Personal financial management
Budget counseling 9 7 Financial education/ information 12 10 Emergency
financial assistance 9 7
Special needs
Elder care services 2 2 Exceptional family member e 5 6
Note: The percentages do not add to 100 because respondents could select
more than one item. Based on Air Force calculations, the margin of error for
active duty members' responses is less than 1 percentage point, and the
margin of error for spouse responses is plus or minus 1 percentage point.
The 2000 needs assessment was used at all Air Force bases. A total of 58,
732 surveys were completed- 35, 732 among active duty members (55 percent
response rate) and 22, 194 among
spouses (25 percent response rate). The random sample of active duty members
was drawn by the Air Force Personnel Center, oversampling members in ranks
E1 to E4 to compensate for typically low response rates from this group.
Surveys of active duty members were distributed and collected at their duty
stations. The random spouse sample, also oversampled to account for
typically low response rates from civilian spouses, was drawn from Defense
Enrollment Eligibility Reporting System data provided by the Defense
Manpower Data Center. Spouse surveys were mailed directly to spouses, who
then returned their completed surveys by mail. a Differences between active
duty members' and spouses' responses cannot be attributed to differences
between married couples because the members and spouses were randomly
selected. In addition, not all the active duty respondents were married.
b The Family Advocacy Program is a mandatory program. c The Transition
Assistance Program is a mandatory program. d The Relocation Assistance
Program is a mandatory program. e The Air Force's Exceptional Family Member
Program identifies family members who have exceptional medical and/ or
educational needs so that personnel can be assigned to locations with
adequate medical and educational resources to meet the family member's need.
Considering only the number of responses, table 1 shows that Air Force-
wide, needs for support related to deployments and separations were far more
commonly reported than needs associated with other key
family support areas. 13 More than 50 percent of both active duty members
and spouses said that they needed morale telephone calls during deployments
and separations. In a focus group we conducted at one Air Force base,
spouses of enlisted Air Force members specifically highlighted a need for
morale calls and an increase in the use of videophones and phone cards for
these purposes. 14 The top five most commonly reported needs for deployment-
related support- needs for morale calls, email communication, support during
separations, calls from units to families, and car maintenance help- are
fairly consistent between the servicemembers and the spouses. The exception
is car maintenance help, which ranks third among spouses and fifth among
active duty members. Considering all the different types of community and
family support programs and services where needs were measured, Air Force-
wide, the top five most reported needed by active duty members were morale
calls during deployments/ separations (56 percent), email communication
during deployments/ separations (51 percent), adult fitness activities (49
percent), full- day child care on base (40 percent), and adult continuing
education (37 percent). 15 The top five most reported needed programs and
services by military spouses, Air Force- wide, were morale calls during
deployments/ separations (54 percent), email communication during
deployments/ separations (49 percent), adult fitness activities (42
percent),
13 This is consistent with Air Force data on the pace of military air
operations and deployments. In recent congressional testimony, the Air Force
Chief of Staff reported that Air Force personnel are deploying over three
times more often with a force 60 percent smaller than its former size. 14
While morale calls are highly valued by Air Force spouses, they said that
they have concerns about the privacy of their conversations, since these
calls must be placed by operators and are monitored, to some degree, for
time. Videophones, which involve a telephone and video connection, provide
another option for making morale calls. Air Force spouses told us that more
resources are needed for videophones as well as phone cards, which would
presumably allow them to make calls directly, without the use of an
operator.
15 The Air Force needs assessment also includes background questions to
develop a statistical/ demographic profile of the respondents and includes
other questions about what causes stress, ways of relieving stress, top
concerns Air Force- wide and at the base (e. g.,
housing, safety, pace of military operations, health care, and spouse
employment), supportiveness of the Air Force community, satisfaction with
multiple characteristics of the Air Force way of life, and problems
encountered in the family in the past 12 months.
car maintenance help during deployments/ separations (38 percent), and
support during separations (37 percent). 16 Needs Assessment The Air Force
has used need assessment results to budget for some family Results and
Budgets support programs but not for others, even though DOD regulations
state
that designing, planning, continuing, expanding, or terminating programs for
Family Support should be based on the results. For example, it has used the
assessments as Programs Are Not a basis for budgeting for some child care
programs but has not generally Consistently Linked
used them for family advocacy programs (e. g., programs to prevent and treat
domestic violence) and for programs under the family support centers (e. g.,
deployment support, spouse employment assistance, general family
readiness services, and financial management education and information,
among others). An Air Force family support center official who develops
budgets for one command's base programs said that the Air Force has no
requirement to use needs assessment data to help in budgeting for the
programs. According to an Air Force child care program official, in the past
needs assessment data was used to allocate funds for a new child care
initiative. Specifically, the results were used to determine which bases
should receive
priority for $1 million provided for an extended day care program. Though
child care programs provide 10 hours of child care a day, according to the
Air Force official, the Air Force downsizing had resulted in members' having
to work longer hours and needing extended care for their children. According
to the Air Force, the needs assessment results were used to determine which
bases would have priority in receiving additional funds for extended day
care services. In addition, the Air Force uses assessment data to project
the need for child care. In the early 1990s, DOD established a formula for
estimating the need of its military families for child care
services that was based on the number of children up to age 12 in military
families whose parents worked outside the home and needed some type of child
care. On the other hand, Air Force officials from the family advocacy and
family
support center programs said that needs assessment data had no direct impact
and is not typically connected to family program budgets or resource
allocations. An Air Force family advocacy official indicated that 16 The
same percent of spouses said that they needed support during separations and
calls
from the unit to the family.
the need assessment results are primarily used for research and evaluation
purposes. For example, the results have been used to understand community
functioning, to establish prevention- related initiatives in the family
support area, and to support Air Force research on techniques to estimate
the prevalence of domestic violence. According to an Air Force official,
family advocacy program budgets are determined on the basis of a formula
that accounts for the number of people in the area they expect to serve and
an indicator of how long it takes to review a domestic violence
case. An Air Force family support center official said that there is not
normally a direct connection between the results of the need assessments and
family program budgets, but cited one example in which need assessment
results had been used to create new family support services. Several years
ago, a
personnel readiness position was created to respond to emerging family needs
associated with an increase in Air Force deployments. However, the Air Force
was unable to obtain money to fund an additional civilian staff member for
the position and assigned duties associated with the position to an existing
active duty member.
The Air Force has also used needs assessment data for policy changes,
knowledge building, planning, and some training. For example:
? The Air Force changed its policy on the minimum age for using base fitness
centers because data showed a desire for teens to have access to the
centers.
? Recommendations on the Air Force's “home alone” policy were
based on data concerning the age that parents believe it is safe to leave
children at home alone, unsupervised, for about 2 hours.
? Family support staff at one base use the data to provide a “big
picture” view of the needs of military personnel and family members
connected to the base community.
? Family support professionals use the data to inform Air Force leaders and
at one base used data showing that servicemembers did not believe the base
leaders support family programs to encourage the leaders to participate in
an annual family- oriented base- wide event.
? Air Force Education and Training Command family support staff include
needs assessment results in squadron commander training briefings.
? Needs assessment results have been used to help develop installation plans
for family support services.
According to private sector organizations like the United Way, the failure
to use needs assessments in resource allocation decisions casts serious
doubt on the value of the assessments, from the perspective of both program
clients and managers. The extent to which the assessments are used for
resource allocation plays a role in clients' commitment to the process and
therefore the quality of the results. For example, family support staff at
one base believed that a reason for the civilian spouses' low response rate
to the assessment survey (i. e., 27 percent) was that there was no
indication
that the sponsor of the survey directly influenced base activities; there
was no cover letter from the base commander indicating sponsorship of the
survey. Conclusions Because the Air Force need assessment process does not
meet criteria recognized as important in conducting needs assessments that
are used for program planning and resource allocation, its assessment
process may not be useful for designing, continuing, expanding, or
terminating family
support services, as required by DOD regulation. Using these criteria in
designing and implementing its needs assessment process would give the Air
Force more and better information on which to determine what programs it
should reduce, expand, or eliminate based on information that needs have
increased, decreased, or emerged; the type of support available
in the civilian community; and the programs that have funding priority. In
addition, the Air Force's inconsistent use of need assessment results for
family program budgeting purposes does not fully support the DOD requirement
that needs assessments be used to design, plan, continue, expand, or
terminate family support programs. DOD's regulation is broad and lacks
specific requirements for conducting military family need assessments. With
specific guidance on conducting need assessments, including the criteria we
have cited in this report, the Department could ensure that the Air Force
and other services' need
assessments processes provide useful information on which to determine what
programs should be reduced, expanded or eliminated, as generally required by
DOD.
Recommendations for In light of DOD policy, we recommend that the Secretary
of Defense direct Executive Action the Secretary of the Air Force to require
the following in conducting its
needs assessments: (1) the use of benchmarks to define when needs have
increased or decreased, (2) the use of a plan to determine how need
assessment results will be prioritized in supporting resource allocation
decisions, (3) the use and integration of information on the type and
capacity of off base family support resources, and (4) a consistent and
defined role for the results of the need assessment process in supporting
family programs' budget development and resource allocations. We also
recommend that the Secretary of Defense direct the Assistant Secretary of
Defense for Force Management Policy to include the same four requirements in
current DOD regulations and guidance pertaining to the services' family need
assessments. Agency Comments and
In written comments on a draft of this report, DOD agreed with our Our
Evaluation recommendations and stated specific actions they will take to
implement each one. Specifically, DOD indicated it would revise family
support
program instructions to require the use of benchmarks in determining needs
and program effectiveness; review policy guidance and, as appropriate,
require the Air Force and other services to prioritize the results of their
need assessments and use data on the capacity of off base resources as a
variable in determining the priority for family support programs; and
develop a process to ensure the use of quantitative and qualitative data,
including need assessment results, in resource allocation decisions.
Although the Department concurred with the recommendations, it expressed
concern about some general implications and conclusions drawn in our report.
DOD's principal concerns and our response are summarized below. The full
text of DOD's comments is in appendix V.
DOD acknowledged that there is value in using benchmarks as a means of
measuring need in family support programs and that benchmarks can aid
installation commanders in making resource allocation decisions. However,
the Department said that our report implied that a needs assessment survey
is the only way that this can be accomplished. Although our report focuses
on the principal needs assessment instrument identified by the Air Force,
this is not intended as an exclusive endorsement of that method. One of our
criteria in evaluating the Air Force's needs assessment process was the use
of a random sample survey. However, we state in our report, “These
criteria are not exhaustive, but represent core elements of need assessment
processes. They do not exclude the use of supplementary need assessment
practices or tools including such things as focus groups or forums, where
appropriate ”.
DOD also stated that the assessment process is complex and is only one
factor in the budget development and resource allocation process. We
agree that the budget development and resource allocation processes may be
responsive to factors other than need assessments. However, need assessment
results must be among these factors, given that in the Air Force's case, it
is the principal tool for obtaining feedback from program users about the
need for family support services and programs. The
number of surveys completed in the Air Force's 2000 needs assessment was
58,000, and as we report, the survey itself covers needs associated with at
least 65 family and community support programs and services and 14 child
care- specific services, among other issues. DOD stated that our study was
limited in scope, including only one service and two bases, and therefore
may not present a complete picture of the process involved in program
prioritization and resource allocation decisions. Nonetheless, the
Department agreed to our recommendations to revise family need assessment
procedures for all services, including incorporating provisions to use need
assessment results in budget development and resource allocation. With
respect to scope, our report states that the focus and scope of this study
was the Air Force's family need assessment process. We met with the primary
Air Force sponsors of the
needs assessment. These officials have key managerial responsibilities for
Air Force family support programs covered in the needs assessment. They
provided us with information on the extent to which the Air Force needs
assessment data is used to support budget and resource allocation
decisions. We also contacted additional Air Force personnel that these
officials said we should contact. We visited one Air Force base to observe
how the Air Force was training local staff on the use and interpretation of
Air Force needs assessment results and were told that this process was
consistent throughout the Air Force. Earlier, in conducting background
research for this report, we traveled to another Air Force base. In
addition, our background research for this work involved interviews with
family support officials from the Army, Navy, Marine Corps, and Reserves and
site visits or contacts with family support staff at three other military
installations. We conducted our review from January 2000 to January 2001 in
accordance with generally accepted government auditing standards.
Appendix I contains information on the Army, Navy, and Marine Corps needs
assessment processes; appendix II includes a list of literature we reviewed
on needs assessments; appendix III contains examples of recent family
support programs and initiatives; and appendix IV shows the results of needs
assessments Air Force- wide, from a U. S. base and from an
overseas base. Appendix V contains the full text of DOD's comments and our
detailed response to them.
Scope and In conducting our work, we met with family and community support
Methodology
officials from DOD's Office of Family Policy and the Air Force to obtain
information on the requirements to conduct family and community need
assessments, discuss how family needs are determined, obtain copies of the
tools and instruments used to assess needs and to obtain need assessment
results. We interviewed officials to determine how results of
needs assessments were used in family support program budget decisions. In
addition, we visited or contacted family and community support professional
staff at three Air Force bases and interviewed them on the
need assessment process and how need assessments impacted program budget
decisions. At the bases, we conducted focus groups with military family
members to gain additional insights into military family needs, issues and
concerns and we met with senior base leadership. We observed needs
assessment training of family support professional staff at an Air Force
base to gain a detailed understanding of the Air Force process. Finally, we
reviewed studies conducted by nonprofit and/ or private sector,
academic, and government organizations that describe central features and
practices of community needs assessments. List of offices contacted and
locations visited:
? Department of Defense, Office of Family Policy, Washington, D. C.
? Department of Defense Family Readiness Conference, Phoenix, Arizona
? Air Force Family Advocacy, Brooks Air Force Base, Texas
? Air Force Child Development, Youth and Community Center Programs, Crystal
City, Virginia
? Air Force Family Matters, Headquarters, Washington, D. C.
? Air Force Family Matters, Air Education and Training Command, Randolph Air
Force Base, Texas
? Charleston Air Force Base, South Carolina
? Hanscom Air Force Base, Massachusetts
? National Military Family Association, Alexandria, Virginia
? Caliber Associates, Fairfax, Virginia We focused on the broad issues
concerning the assessment of military family and community needs rather than
the techniques used to collect needs information (e. g., surveys). For
example, we did not assess the strengths and weaknesses of the specific
content, scope, or overall validity
of the Air Force's family and community needs assessment instrument. We also
did not review the use of potentially more objective indicators of need,
which some literature recommends should be incorporated into community needs
assessments, for example, statistical indicators of economic health or
family well- being such as the number of families receiving/ eligible for
public assistance, the numbers of households with various family
configurations, the median age of women giving birth, the educational level
of persons 18 and over, and housing problems, among others.
We are sending copies of this report to appropriate congressional
committees; the Honorable Donald H. Rumsfeld, Secretary of Defense; the
Honorable Gail H. McGinn, Acting Assistant Secretary of Defense for Force
Management Policy; and the Honorable Lawrence J. Delaney, Acting Secretary
of the Air Force. Copies will also be made available to other
interested parties upon request. If you or your staff have any questions
about this report, please contact me at (202) 512- 2700. A list of
additional contacts and staff acknowledgments is in appendix VI.
Sincerely yours, Nancy Kingsbury Managing Director Applied Research and
Methods
Appendi xes Army, Navy, and Marine Corps Military Family Appendix I
Need Assessment Processes The Army uses surveys to identify the servicewide
needs of military family members. They include, the Survey of Army Families
(SAF), in use since 1987; the Sample Survey of Military Personnel (SSMP), in
use since 1958; and the Department of Defense (DOD) survey of active duty
personnel and spouses. These surveys are distributed at different times, to
different audiences, and differ considerably in content. The SAF is
distributed about every 4 years to civilian spouses of active duty Army
personnel, and the
SSMP is distributed every fall and spring to active duty Army personnel. The
DOD- wide survey of active duty personnel and spouses is sent out every 7
years and is designed to broadly assess attitudes and perspectives of
military life across all the services. Separate survey instruments are used
for active duty personnel and spouses of military personnel.
The Army also relies on information collected during its annual Family
Action Plan process. This process was implemented in 1984 and involves
holding forums with military family members to identify issues of concern to
Army families or issues that detract from their quality of life. The forums
are held at Army bases, major commands, and at headquarters. Issues that
cannot be addressed at lower levels are passed on to officials at higher
levels. From forums, the Army produces its annual Family Action Plan, which
prioritizes top concerns and issues for Army families and lays out a
strategy and vehicle to address them. Open forums like these have recently
become more widely used in DOD and are also used by state governments to
determine needs for services. While useful for obtaining grassroots input,
this process does not involve the use of representative surveys or random or
representative selection of forum participants to ensure representative
input about Army family needs.
The Navy also uses surveys to identify family member needs. Since 1994 the
Navy has implemented three needs surveys, each of which has a different
focus and target audience. The 1994 and 1996 need assessment surveys were
distributed to active duty Navy personnel and spouses. The emphasis in the
1994 survey was on housing and morale, welfare, and recreation issues, while
the emphasis in the 1996 survey was on broad family support issues and
concerns, including the use of and satisfaction with family support
programs. The Navy's 2000 needs survey had a
different focus from that in prior years it was distributed to senior Navy
leaders to assess their views of military family needs. The Marine Corps
does not have a dedicated servicewide family needs assessment instrument or
process. The Marine Corps relies on the results
of its quality of life survey to provide family needs information. This
survey is distributed only to active duty members.
Appendix I I Needs Assessment Literature There is no authoritative study or
body of research concerning how best to conduct military family need
assessments. In developing criteria, we reviewed studies and research
conducted by nonprofit and/ or private, academic, and government
organizations. These criteria are not exhaustive but represent core elements
of need assessment processes. They do not exclude the use of supplementary
need assessment practices or tools, including such things as focus groups or
forums, where appropriate, and they do not involve specific issues of survey
methodology. To identify principal elements of need assessment processes, we
focused on research conducted by organizations experienced and familiar with
providing family support services, such as the United Way and Family Support
America, formerly known as the Family Resource Coalition of America. United
Ways help meet health and human- care needs through approximately 1,400
community- based United Way organizations across the country. Family Support
America, formed in 1981, builds networks, produces resources, advocates for
family supportive public policy, provides consulting services, gathers
knowledge on family support, and offers a national certification for
family support centers and programs. Literature Reviewed Brown, B. S. Drug
Abuse Prevention Needs Assessment Methodologies: A Review of the Literature.
Washington, D. C.: National Institute on Drug Abuse, Center for Health
Services Research, 1997.
Department of Defense, Inspector General. Inspection of the Military
Department Family Centers, report 90- INS- 11. Arlington, Va.: Department of
Defense, 1990. Family Resource Coalition of America. Know Your Community: A
Step- by- Step Guide to Community Needs and Resource Assessment. Chicago:
Family Resource Coalition of America, 1995.
General Accounting Office. Health Service Program Needs Assessments Found
Inadequate, HRD- 81- 63. Washington, D. C.: GAO, 1981. Kaufman, R. “A
Needs Assessment Audit.” Performance Improvement. Feb. 1994, pp. 14-
16. Kaufman, R. “The New Realities, Strategic Planning, Needs
Assessment, and Organizational Contribution.” Tallahassee, Fl.:
Florida State University;
1998. http:// onap. fsu. edu/ onap/ download/ hsna/ plan. pdf (downloaded
Aug. 2000).
Kaufman, R., A. M. Rojas, and H. Mayer. Needs Assessment: A User's Guide.
Englewood Cliffs, N. J.: Educational Technology Publications, 1993.
Orthner, D. K., and S. Smith. “Measuring Program Needs: A Strategic
Design.” Evaluation and Program Planning. 9: 3 (1986), pp. 199- 207.
Parsons, R. J., and others. “Community Needs Assessment: An Integrated
Approach.” National Civic Review. 73 (1990), pp. 426- 35. United Way
of America. Needs Assessment: The State of the Art- A Guide for Planners,
Managers, and Funders of Health and Human Services. Alexandria, Va.: United
Way, 1982. United Way of America. Needs Assessment and Evaluation Practices
of United Ways: Report on a Joint Study by United Way of America and The
Urban Institute. Alexandria, Va.: United Way, 1984. Watkins, R., and others.
“Needs Assessment: A Digest, Review, and Comparison of Needs
Assessment Literature.” Performance Improvement, September, 1998, pp.
40- 53.
Recent Military Family and Personnel Support
Appendi x I I I Programs and Initiatives New Parent Support In fiscal year
1998, new parent support programs became a line item in the Program DOD
budget. Each service has initiated a voluntary program to provide (1)
personal support for new parents, (2) reinforcement of the skills and
strengths needed for quality parenting, (3) facilitation of emotional
preparation for parenthood, (4) increased awareness of the available
community resources, (5) reduced isolation of new parents, (6)
identification of families with the potential for problem parenting, and (7)
early referral to appropriate support services. Each program offers basic
support services to all parents who request services, while focusing more
intensively on families where there is a higher risk for child abuse. Each
program offers a set of core services to every expecting family served by a
military medical treatment facility, including a prenatal hospital visit and
assessment, one prenatal home visit, postnatal visits in the hospital and at
home, and education and support groups. Healthy Parenting
The healthy parenting initiative was established to provide military-
specific Initiative
parenting resources that are readily accessible and to address parenting of
children from infancy through adolescence in connection with the military
lifestyle. DOD believed that the challenges of military life were not
adequately addressed in the existing civilian- based parenting curricula
used in DOD family support programs. For example, for military parents who
have little time to attend traditional parent education classes, parenting
materials must be flexible and readily accessible. DOD believes that as a
result of an increasing level of military operations, many families struggle
with intermittent single parenting and helping their children cope with loss
and separation. Moreover, they indicate that because most active duty
members' families, even when intact, are usually separated from extended
family members, family- based parenting guidance is limited.
DOD's Office of Family Policy, in partnership with the Department of
Agriculture's Cooperative State Research, Education, and Extension System,
chartered the initiative to develop military- specific parenting resources.
The Air Force is designated as the Executive Agent for the project. The
initiative will produce material that can be used in a classroom and also
made available to parents through a variety of other formats (e. g., Web-
based material, one- page flyers, and audiotapes). In September 1999, the
Air Force developed three specific proposals outlining the following
parenting resources: (1) educational materials to address parenting in the
military and strategies to ensure ease of access to the materials, (2)
materials and strategies to improve communications between teens and
parents, and (3) command educational materials on links between healthy
families and mission readiness.
Combat Stress Control DOD issued regulations on the Combat Stress Control
Program in February Program 1999. This program is designed to ensure
appropriate prevention and management of combat stress reaction casualties
to preserve mission effectiveness and warfighting and to minimize the short-
and long- term adverse effects of combat on the physical, psychological,
intellectual, and social health of servicemembers. The program is to include
curricula, training, and exercise requirements for joint and service-
specific
operations that focus on primary, secondary, and tertiary prevention of
combat stress reactions in settings from the barracks to the battlefield.
The program includes training in the principles of combat stress management
and training that addresses leadership, communication with troops, unit
morale and cohesion, and individual psychological stressors before, during,
and after deployment.
Suicide Prevention In June 1999, the President announced expansion of the
Air Force suicide prevention pilot program throughout the military by the
end of 1999. The prevention program considers nine risk factors that were
found to be frequently associated with suicide victims and four factors
believed to protect people from suicide attempts. These factors include a
history of
mental health problems, substance abuse, relationship problems, poor coping
skills, legal problems, previous suicide attempts, financial problems, poor
job performance, and social isolation. The protective factors include
establishing social support in units, teaching individual coping skills,
establishing the belief among military personnel that seeking mental health
care is acceptable, and ensuring privacy protection. The program involves
the following: (1) distributing public service messages by DOD senior
leaders that express concern about suicide and the importance
of encouraging and protecting members who seek help; (2) debriefing
individuals and units following traumatic events; (3) establishing annual
suicide awareness and prevention training for all military personnel; (4)
integrating the delivery of preventive services from six agencies- mental
health, family advocacy, health and wellness centers, family support
centers, child and youth centers, and chaplains; (5) including suicide
prevention training in all professional military education programs;
(6) developing a more robust database of those who committed and attempted
suicide so that risk factors and trends can be analyzed and
preventive measures taken; and (7) conducting unit risk assessment surveys
and unit intervention as required. Increase in DOD In September 1999, the
DOD Education Activity announced that it planned School Counselors
to nearly double the number of counselors and psychologists in its schools
by June 2000 to help students deal with increasing stresses in military
life. This amounts to an increase of 200 counselors and psychologists.
According to the Education Activity, this increase was in response to
parents' concerns about their children, particularly in Europe, where the
high rate of deployments had increased family disruptions. In addition, the
counselors were to increase their contact with students to help them plan
for the future, including college or vocational educational programs.
Increase in In 1996, DOD's Office of Family Policy stated a commitment to
the Technology- Based development of technology strategies to improve access
to and use of family program information as part of its Family Program
Internet Family Support Initiatives. Examples of the Web sites that have
been developed follow: Delivery Mechanisms
MAPsite (http:// dticaw. dtic. mil/ mapsite). This is a customer- oriented
public site that provides military members, civilians, and their families
self- assessment tools, advice, and links to family program areas.
Military Teens on the Move (http:// dticaw. dtic. mil/ mtom). This site was
designed to help military teens meet the challenges of frequent moves and
the demands of their parents' mobile lifestyle.
DOD Job Search (http:// dod. jobsearch. org). This site features want ads,
resume writing, and referral systems geared to transitioning military
personnel and their spouses, DOD federal civilian employees, and the spouses
of relocating active duty members. In addition, the Army, Navy and Air Force
have public access Web sites designed to provide information on a range of
quality of life and family support- related services and information. The
Army's Web site is called the Virtual Army Community Service Center (http://
trol. redstone. army. mil/ acs/ virtual/ target. html ). The Navy's site is
known as LIFELines (http:// www. lifelines4qol. org ). The Air Force's
family support- related Web site is known as Crossroads (http:// www.
afcrossroads. com/ home. cfm).
Results of Some Air Force Needs Assessments
Appendix V I (2000) Table 2: Results of the Air Force 2000 Needs Assessment
Air Force- Wide and for Air Force Bases in Aviano, Italy and Bedford,
Massachusetts. Percent of active duty personnel that Percent of civilian
spouses that reported a need a reported a need b
All Air Aviano Air
Hanscom Air All Air
Aviano Air Hanscom Air
Support service or program Force base base Force base base Deployment
Morale calls 56 63 47 54 64 42 Email communications 51 56 50 49 51 43
Support during separations 30 28 28 37 43 28 Calls from unit to family 29 28
25 37 43 27 Car maintenance help 28 32 22 38 57 32 Predeployment support 23
25 22 17 16 14 Newsletters 14 14 9 23 25 14 Post- deployment assistance 12
14 13 7 5 6 Help with child care costs 11 14 8 13 8 12 Family support groups
10 10 9 17 17 19 Financial planning 10 11 8 7 7 5 Emergency financial
assistance 9 10 6 8 9 4 Spouse employment support 8 8 7 13 22 12 Family care
plan help 6 6 4 5 4 5 Youth supervision 6 6 7 9 10 10 Family reunion
training 4 3 5 5 4 7 Workshops about family life/ parenting 2 2 2 3 2 3
Respite care 2 3 3 4 3 5
Family advocacy
Family maltreatment services 2 3 2 2 0 2 Family violence prevention
information 2 2 2 2 0 2
Family life education 4 55456
Parenting classes 4 55685
Information and referral
Housing referral 23 33 21 21 27 17 Directory of community 15 18 21 23 31 27
services/ programs Family member employment 12 13 12 16 19 15
assistance Transition assistance
Retirement employment information 5 4 5 5 4 7
(Continued From Previous Page)
Percent of active duty personnel that Percent of civilian spouses that
reported a need a reported a need b All Air
Aviano Air Hanscom Air
All Air Aviano Air
Hanscom Air Support service or program Force base base Force base base
Relocation assistance
Retirement/ separation workshops 7 5 8 6 4 7 Moving services/ assistance 20
21 24 22 20 22 Relocation assistance 15 15 18 14 10 15 Relocation
information 11 12 15 14 13 16 Financial help with relocation expenses 6 8 5
6 6 3 Sponsorship assistance 14 16 17 10 15 11
Personal financial management
Budget counseling 9 8 7 7 4 4 Financial education/ information 12 14 11 10
10 8 Emergency financial assistance 9 9 6 7 8 2
Special needs
Elder care services 2 2 2 2 0 0 Exceptional family member 5 9 6 6 6 7
Note: The percents do not add to 100 because respondents could select more
than one item. a The margin of error for active duty responses from Aviano
is plus or minus 5 percentage points and
from Hanscom is 4 percentage points. The margin of error for spouse
responses at Aviano and Hanscom is plus or minus 6 percentage points. b
Differences between active duty members and civilian spouses cannot be
attributed to differences between married couples. Survey participants were
randomly selected. In addition, not all active duty respondents were
married.
Considering only the number of responses, the needs most commonly reported
by active duty members at Aviano and Hanscom were for morale calls and email
communication during deployments, separations, and unaccompanied tours.
Nearly two- thirds of active duty respondents at Aviano said that they
needed morale calls, and more than 50 percent said that they needed email
communication. At Hanscom Air Force Base,
50 percent of active duty respondents said that they needed email
communication during deployments, separations, or unaccompanied tours, and
47 percent said they needed morale calls. The same pattern of results was
found for spouses, with the exception that morale calls and car maintenance
help were the needs most commonly reported by spouses at Aviano Air Force
Base. Specifically, nearly two- thirds of spouses at Aviano said they needed
morale calls during deployments, separations, and unaccompanied tours, and
nearly 60 percent said they needed car maintenance help during these
periods. Consistent with other findings,
more spouses at Hanscom Air Force Base said they had needs for morale calls
(42 percent) and email communication (43 percent) compared to other support
needs. The Air Force- wide results show that when comparing among the
different categories of need, more active duty personnel and spouses
indicated a need for some services or support concerning deployments and
separations, relocation, and information and referral, compared to other
areas. These results are also reflected in the data for Hanscom and Aviano.
The greatest numbers of both active duty members and spouses indicated the
need for some type of deployment/ separation- related support (i. e.,
maximum percent of active duty is 63 percent and maximum percent of spouses
is 64 percent). Comparatively, the next greatest numbers of active duty
members and spouses indicated needs for housing referrals (i. e., maximum
percent of active duty is 33 percent and maximum percent of spouses is 27
percent) or a directory of community services (i. e., maximum percent of
active duty is 21 percent and maximum percent of spouses is 31 percent) and
moving services (i. e., maximum percent of active duty is 24 percent and
maximum percent of spouses is 22 percent) . Similar to the Air Force- wide
need assessment results, needs expressed by active duty personnel and
spouses at Hanscom and Aviano in some cases
exceeded expressed needs for the conventional family support programs and
services listed in table 2. In considering all the different types of
community and family support programs and services where needs were
specifically measured, the five most commonly reported for active duty
members at Hanscom were full- day child care on base (51 percent), email
communication during deployments/ separations (50 percent), adult fitness
activities (50 percent), morale calls during deployments/ separations (47
percent), and adult continuing education (35 percent). The five most
commonly reported needs for active duty members at Aviano were the same as
the results for Hanscom- only the order was different: morale calls during
deployments/ separation (63 percent), email communication during deployment/
separation (56 percent), adult fitness activities (54 percent), full- day
child care on base (50 percent), and adult continuing
education (41 percent). Spouses at Hanscom and Aviano had three top five
needs in common with each other and with active duty members morale calls
and email communication during deployments/ separations and adult fitness
activities. The top five most commonly reported needs by spouses at Hanscom
were adult fitness activities (45 percent), email communication
during deployments/ separation (43 percent), morale calls during
deployments/ separations (42 percent), chapel services (39 percent), and
couples communication (36 percent). The top five most commonly reported
needs by spouses at Aviano were morale calls during deployments/ separations
(64 percent), car maintenance help during deployments/ separations (57
percent), email communication during
deployments/ separations (51 percent), adult fitness activities (44
percent), and children/ youth/ programs and services (36 percent). The need
assessment reports for Aviano and Hanscom were selected on the basis of
convenience, with no attempt to match or contrast the bases on select
criteria. However, the two bases are distinctly different in mission. An Air
Force official indicated that the single most important factor that
accounts for differences in base need assessments is mission, as differences
in mission translate to differences in individual/ unit behaviors, work
styles, staffing, and resource distribution. Hanscom Air Force Base, located
approximately 20 miles from Boston, is part of Air Force Materiel Command.
Its primary mission is to manage the development and acquisition of
electronic command and control systems, which gather and analyze information
on potentially hostile forces. In short, Hanscom is primarily concerned with
the business functions involved in developing and procuring electronics. On
the other hand, Aviano, located in Italy, is home to the 16th Air Force
which is part of Air Force European Command. Its mission is to execute air
operations in support of the European Command and NATO. The 16th Air Force
supports enforcement of peace in the Balkans and enforcement of the no- fly
zone in northern Iraq. In spite of these distinctly different missions,
needs assessment results appear remarkably similar.
Appendi x V Comments From the Department of Defense
See comment 1. See comment 2.
See comment 3 See comment 4.
See comment 5. See comment 6.
See comment 7. See comment 8.
See comment 9. See comment 10. See comment 11.
The following are GAO's comments on DOD's letter dated February 1, 2001. GAO
Comments 1. DOD comments that we seemed to use the terms “needs
assessment process” and “needs assessment survey”
interchangeably in our report. It noted that there is a distinct difference
between the survey that two of the services use and the complex process that
all the services use.
The focus of our report is the Air Force family need assessment process. We
did not conduct a detailed assessment of other services' processes, although
we obtained some information on these during our background research (see
app. I). In addition, we reviewed DOD need assessment requirements that
pertain to all the services. During our review, the Air Force presented its
family and community needs assessment survey as its primary method of
determining the needs of its families. In technical comments on our draft
report, the Air Force
identified additional tools and methods, including the Air Force Quality of
Life Survey, Leisure Needs Survey, DOD Worldwide Survey, and administrative
data, among others. While these additional sources may
be useful, the Air Force did not indicate how they are integrated, assessed,
or prioritized to arrive at a representative and reliable indication of Air
Force family and community needs. As we report here, there is such a
process, although with limitations, associated with the Air Force's family
and community needs assessment survey.
2. DOD states that we have not fully captured Air Force practices in
assessing needs, benchmarking, and linking the results of the needs
assessment process to program and resource allocation decisions. However,
the comment identifies no specific Air Force benchmarking practices or needs
assessment results that are linked to program and resource allocation
decisions. 3. DOD notes in its comments that its family support regulations
apply only to family centers and do not apply to child and youth, education,
or family advocacy programs. We note in our report, however, that DOD
family center regulations do not apply to all DOD family support programs,
including such programs as child care and various morale, welfare, and
recreation programs. They also do not apply to family advocacy programs.
However, the Air Force offices that oversee child and youth and family
advocacy programs contribute to the development of the Air Force needs
assessment survey and submit questions related to needs for these programs.
Thus, while these programs are not required to conduct a needs assessment
under the
regulation, they are engaged in the process, and it is not unreasonable to
inquire how they use the information derived from it. Therefore, we included
these offices in our review.
4. DOD notes that its policy does not direct the services to use a needs
assessment survey to assess needs, but rather, broadly requires the services
to assess the needs of its members. It also notes that policy does not
require resource allocations to be based on needs assessments. However, DOD
Instruction 1342.22, issued December 30, 1992, states on page 4 that the
secretaries of the military departments will develop a comprehensive
evaluation system, including a needs assessment survey, to measure the
effectiveness of family centers. Further, the regulation states that the
survey is to be designed, constructed, and conducted to provide
scientifically valid and reliable
information about the needs and use patterns of individuals and families.
The survey results are also to serve as the “basis for the design and
planning of future services and the continuation, expansion, or
termination of others.” While DOD released a “directive- type
memorandum” on March 23, 1999, that modified the requirements under
this instruction to establish baseline services at military installations,
this change did not alter any other requirements under the instruction,
including those related to family needs assessments. Therefore, on balance,
it appears that DOD did, and continues to intend for the needs assessment to
serve as the basis for resource allocation.
5. DOD notes that our report generally does not reflect the complete process
the Air Force uses when making budget and resource allocation decisions.
They also comment that we state in our report that the Air Force does not
tie the results of the needs assessment process to budget and resource
allocation. As requested, we determined how
information from needs assessments affects the allocation of funding for Air
Force family support programs. We note that the Air Force has used needs
assessment results to budget for some family support programs but not for
others, even though the DOD regulations state that designing, planning,
continuing, expanding, or terminating
programs should be based on the results. We state that the Air Force has
used the assessments as a basis for budgeting for some child care programs
but has not generally used them for family advocacy programs and for
programs under the family support centers. Air Force family advocacy
officials indicated that family advocacy program budgets are determined
based on the number of people they expect to serve and how long it takes to
review a domestic violence case. We also
note that, according to Air Force family support center officials, there is
not normally a direct connection between the results of the need assessments
and family program budgets. However, we report that need assessment results
recently impacted budget decisions, specifically regarding the creation of a
personnel readiness position in family centers.
6. DOD notes that the services use various methods to assess need and
compare data from these sources to benchmark progress. Its comments further
say that our report does not indicate whether required benchmarking refers
to measuring only increases and decreases in needs solely based on the needs
assessment survey. DOD comments that only using the survey may omit other
important data that should be considered. Although our report focuses on the
principal needs assessment instrument identified by the Air Force, we are
not exclusively endorsing survey methods for development of benchmarks. One
of our criteria in evaluating the Air Force's needs assessment process was
the use of a random sample survey. However, as we state in our report, these
criteria represent core elements of need assessment processes, but do not
exclude the use of supplementary need assessment practices or tools. In our
observation of feedback and training based on the needs assessment,
scattered reference was made
to service- wide results, but base staff did not appear to have easy access
to benchmarks or comparative information and expressed a desire for such
information to better understand their own base's results. As indicated in
the Department's comments, such information would have been made available
on specific request.
7. DOD commented that base resource allocation decisions cannot be based on
the results of a single need assessment instrument nor based on a review of
a single service's process. We do not state or imply that family support
resource allocation decisions should be based solely on the results of any
one need assessment survey. As requested, we determined how information from
needs assessments affects the allocation of funding for Air Force family
support programs. As noted in our report, the Air Force has used needs
assessment results to budget for some family support programs but not for
others. We also note that, “according to private sector organizations
like the United Way, the failure to use needs assessments in resource
allocation decisions casts serious doubt on the value of the assessments,
from the perspective of both program clients and managers.”
8. DOD commented that, while it is difficult to determine the number of
programs required by law, many of the programs are required by law and that
leaves very little discretionary funding available for reallocation among
these programs. We agree that it is difficult to quantify the number of
family support programs that are required by law, and DOD did not provide us
with a specific number. Our reference
to the number of programs required by law was made in the context of the
list of programs and services that are included in the Air Force needs
assessment report and for which respondents to the needs assessment are
asked to indicate a need. We note that the Air Force's final report on its
2000 needs assessment includes needs associated with at least 65 family and
community support programs or services
and 14 child care- specific services, among others. Specifically, among the
65 programs and services we reference, a minority of them appear to be
linked to the three known mandated family support programs Family Advocacy,
Transition Assistance, and Relocation Assistance.
9. DOD notes that its policy requires communities to assess the capacity of
off base resources and to use that data in determining the allocation of
resources for programs. The policies we reviewed required that family
centers coordinate with local services to avoid duplication and
that family needs be partially based on the services available in the local
community. Thus, they did not explicitly require an assessment of local
capacities but implied one for the type of services typically provided by
family centers. Air Force personnel and their family members must reside in
areas that range from major metropolitan areas to fairly isolated ones. This
underscores the need for systematic consideration of the capacity of
community services in supporting family needs. In communities of substantial
size, such information should be readily available. In addition, existing
published resources such as, The National Directory of Children, Youth and
Families Services, may be useful, comprehensive resources to help identify
and learn basic information about family support providers in locations
across the United States. 10. According to DOD, we imply that the Air Force
does not assess the capacity of off base resources and use the results of
the assessments. DOD states that in determining the allocation of resources
for programs the Air Force has specific guidance to ensure that such
information is obtained and available. It is not clear what Air Force
guidance is being referred to. During our review, the Air Force provided us
with policy guidance (Air Force Instruction 90- 500, Feb. 1, 2000),
related to the authority and criteria for establishing groups to promote
cross- organizational collaboration in addressing individual, family, and
communi ty concerns. However, we found no references to integrating
information on the type and capacity of off base resources into the
community needs assessment. During our observation of needs assessment
training at one Air Force base and our review of the training documents, we
did not find that family support staff were required to integrate
information on the capacity of off base resources in reaching conclusions
about what actions were to be taken based on the need assessment results.
11. DOD noted that rigidly quantifying the capacity of off base resources to
deal with the needs of military members and families would be difficult if
not impossible, but that where possible, all the services partner with the
civilian communities. We neither state nor imply that the Air Force should
make a rigid quantification. Rather, we state that if the Air Force does not
determine what support services are available in the civilian community, how
many people they can accommodate, and how they are used, its base family
support staff cannot know whether they are
duplicating services, underusing or overwhelming their own programs, or
concentrating funds on the right programs. Existing publications such as,
The National Directory of Children, Youth and Families Services, may help
identify professional family support providers across the United States and
provide basic information to support these efforts.
Appendix VI
GAO Contacts and Staff Acknowledgments GAO Contacts Carolyn Copper, Ph. D.
(202) 512- 3762 Betty Ward- Zukerman, Ph. D. (202) 512- 2732 Acknowledgments
In addition to those named above, Kwai- Cheung Chan, John Oppenheim, Ph. D.,
Nancy Ragsdale, and Samantha Goodman made important contributions to this
report.
(713066) Let er t
GAO United States General Accounting Office
Page 1 GAO- 01- 80 Military Personnel
Contents Letter 3 Appendixes Appendix I: Army, Navy, and Marine Corps
Military Family Need Assessment Processes 26
Appendix II: Needs Assessment Literature 28 Appendix III: Recent Military
Family and Personnel Support Programs and Initiatives 30
Appendix IV: Results of Some Air Force Needs Assessments (2000) 33
Appendix V: Comments From the Department of Defense 37 Appendix VI: GAO
Contacts and Staff Acknowledgments 47
Page 2 GAO- 01- 80 Military Personnel
Page 3 GAO- 01- 80 Military Personnel United States General Accounting
Office
Washington, D. C. 20548 Page 3 GAO- 01- 80 Military Personnel
Page 4 GAO- 01- 80 Military Personnel
Page 5 GAO- 01- 80 Military Personnel
Page 6 GAO- 01- 80 Military Personnel
Page 7 GAO- 01- 80 Military Personnel
Page 8 GAO- 01- 80 Military Personnel
Page 9 GAO- 01- 80 Military Personnel
Page 10 GAO- 01- 80 Military Personnel
Page 11 GAO- 01- 80 Military Personnel
Page 12 GAO- 01- 80 Military Personnel
Page 13 GAO- 01- 80 Military Personnel
Page 14 GAO- 01- 80 Military Personnel
Page 15 GAO- 01- 80 Military Personnel
Page 16 GAO- 01- 80 Military Personnel
Page 17 GAO- 01- 80 Military Personnel
Page 18 GAO- 01- 80 Military Personnel
Page 19 GAO- 01- 80 Military Personnel
Page 20 GAO- 01- 80 Military Personnel
Page 21 GAO- 01- 80 Military Personnel
Page 22 GAO- 01- 80 Military Personnel
Page 23 GAO- 01- 80 Military Personnel
Page 24 GAO- 01- 80 Military Personnel
Page 25 GAO- 01- 80 Military Personnel
Page 26 GAO- 01- 80 Military Personnel
Appendix I
Appendix I Army, Navy, and Marine Corps Military Family Need Assessment
Processes
Page 27 GAO- 01- 80 Military Personnel
Page 28 GAO- 01- 80 Militay Personnel
Appendix II
Appendix II Needs Assessment Literature
Page 29 GAO- 01- 80 Militay Personnel
Page 30 GAO- 01- 80 Military Personnel
Appendix III
Appendix III Recent Military Family and Personnel Support Programs and
Initiatives
Page 31 GAO- 01- 80 Military Personnel
Appendix III Recent Military Family and Personnel Support Programs and
Initiatives
Page 32 GAO- 01- 80 Military Personnel
Page 33 GAO- 01- 80 Military Personnel
Appendix IV
Appendix IV Results of Some Air Force Needs Assessments (2000)
Page 34 GAO- 01- 80 Military Personnel
Appendix IV Results of Some Air Force Needs Assessments (2000)
Page 35 GAO- 01- 80 Military Personnel
Appendix IV Results of Some Air Force Needs Assessments (2000)
Page 36 GAO- 01- 80 Military Personnel
Page 37 GAO- 01- 80 Military Personnel
Appendix V
Appendix V Comments From the Department of Defense
Page 38 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 39 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 40 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 41 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 42 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 43 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 44 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 45 GAO- 01- 80 Military Personnel
Appendix V Comments From the Department of Defense
Page 46 GAO- 01- 80 Military Personnel
Page 47 GAO- 01- 80 Military Personnel
Appendix VI
Ordering Information The first copy of each GAO report is free. Additional
copies of reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit cards are
accepted, also. Orders for 100 or more copies to be mailed to a single
address are discounted 25 percent. Orders by mail: U. S. General Accounting
Office P. O. Box 37050 Washington, DC 20013
Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW)
U. S. General Accounting Office Washington, DC
Orders by phone: (202) 512- 6000 fax: (202) 512- 6061 TDD (202) 512- 2537
Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists. Orders by Internet:
For information on how to access GAO reports on the Internet, send an e-
mail message with “info” in the body to:
info@ www. gao. gov or visit GAO's World Wide Web home page at: http:// www.
gao. gov
To Report Fraud, Waste, or Abuse in Federal Programs
Contact one:
? Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
? e- mail: fraudnet@ gao. gov
? 1- 800- 424- 5454 (automated answering system)
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Presorted Standard
Postage & Fees Paid GAO Permit No. GI00
*** End of document. ***
|
NEWSLETTER
|
|
Join the GlobalSecurity.org mailing list
|
|
|
