FNS RESEARCH CORNER
SUMMARY OF RESEARCH
John R. Endahl
The FNS Research Corner provides a continuing series of summaries of recently completed and current research conducted by the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) in the area of child nutrition. For further information, contact the Office of Research and Analysis (ORA) at (703) 305-2117. Links to published studies and reports as well as descriptions of ongoing studies conducted by ORA are available from the FNS web site at http://www.fns.usda.gov/ora/.
RECENTLY COMPLETED RESEARCH
Evaluation of the Fresh Fruit and Vegetable Program: Interim Report
The Fresh Fruit and Vegetable Program (FFVP) aims to increase fruit and vegetable consumption among students in the Nation’s poorest elementary schools by providing free fresh fruits and vegetables to students outside of regular school meals. FFVP began as a pilot program in 2002 and was converted into a nationwide program in the Food, Conservation, and Energy Act of 2008, also known as the Farm Bill (PL110-234). The 2008 Farm Bill required an evaluation of the program to determine “whether children experienced, as a result of participating in the program (A) increased consumption of fruits and vegetables; (B) other dietary changes, such as decreased consumption of less nutritious foods; and (C) such other outcomes as are considered appropriate by the Secretary.”
The results presented in this interim report, for the 2010-2011 school year, focus on the total quantity of fruits and vegetables consumed and total energy intake (also referred to as total caloric intake), allowing the assessment of whether any additional fruit and vegetable consumption was in addition to or in place of other foods consumed.
This evaluation estimates the impact of FFVP using a regression discontinuity design by comparing schools immediately above and below the funding cutoffs in each of a nationally representative sample of States. The final preferred analytic sample included 4,696 students in 214 schools within 2.5 percentage points of the funding cutoff in each State; 2,225 students in 99 FFVP schools just above the funding cutoff, and 2,471 students in 115 non- FFVP schools just below the funding cutoff. The study collected information on student food intake using diary-assisted 24-hour recall interviews, which have been successfully used with elementary school aged children. In FFVP schools, the diary was completed on a day on which FFVP fruits and/or vegetables were offered to students, allowing us to estimate the impact of FFVP on intake on FFVP days.
The results indicate that FFVP increased average fruit and vegetable consumption among students in participating schools on FFVP days by approximately one-quarter of a cup per day (p 0.001). This represents an increase of 15 percent over fruit and vegetable consumption levels in the absence of FFVP. No increase in total energy intake was found. If an increase in total energy (caloric) intake had been found, we might have been concerned that FFVP could contribute to weight gain. Instead, increased fruit and vegetable consumption appears to have replaced consumption of other foods.
An increase in fruit and vegetable consumption of one-quarter of a cup per day is within the range observed in various other interventions to increase fruit and vegetable consumption in school children. Although there is no standard consensus as to what constitutes a meaningful change in fruit and vegetable intake, it is generally accepted that children with the lowest intakes are at greatest risk of poor health outcomes, and that the greatest benefit would be conferred by increasing intakes of fruits and vegetables among this group. In this context, the fact that FFVP targets poorer schools is potentially significant. Because children in low socio-economic status households are more likely to have the lowest intakes of fruits and vegetables, increasing fruit and vegetable intakes in this population even by small amounts is likely to confer a health benefit.
The final evaluation report will expand on this interim report by including impacts for additional outcomes and by including detailed information on how FFVP was implemented in a nationally representative sample of schools.
School Food Purchase Study III
This is the third study that provides national estimates of the type, quantity, dollar value and unit price of food acquisitions by public school districts participating in the National School Lunch Program (NSLP) and School Breakfast Program (SBP). The study examines the overall changes in the composition of the entire school food market basket including foods purchases for a la carte sales and the relative importance of donated USDA Foods. It also provides insight into the relationship between district characteristics, purchasing practices, and food costs.
A nationally representative sample of 420 public school districts provided source documents (vendor summaries, copies of invoices, etc.) of all food acquisitions for 3 months of the 2009/10 school year (SY). School districts were randomly assigned evenly to all four quarters of the school year. The sampled districts also completed a procurement practices survey to collect basic descriptive information on the district, its food service operation, and procurement procedures.
Main findings of interest were as follows:
- In SY 2009/10, unified public school districts acquired food valued at more than $8.5 billion. Of the total value of school food acquisitions, 81 percent was purchased commercially, 11 percent was USDA Food donations, and 8 percent was processed product containing donated USDA Foods.
- Milk and other dairy products represented the largest food group by value, accounting for 22 percent of the total. Bakery product represented 14 percent, and fruits and fruit juices 13 percent. Poultry, prepared foods, and vegetables all fell in the 9-11 percent range.
- School districts acquired a wide variety of food products as evidenced by the 865 distinct products acquired. However, the top 100 items accounted for 73 percent of the total value of foods acquired.
- USDA Foods continued to be an important source for certain foods. Donated USDA Foods accounted for 40 percent or more of the value of cheese, turkey, beef, and chicken acquired by school districts, and almost a third of the fruit.
- Ongoing USDA efforts to improve the nutritional profile of school meals can be seen in the changes in food acquisitions between SY 1996/97 and SY 2009/10. Poultry, bakery products, and fruits and fruit juice acquisitions increased by over 50 percent. The relative importance of fresh produce increased from 8 percent of total pounds to 13 percent. Acquisitions of fruit juices and water replaced fruit drinks and carbonated beverages. Acquisitions of vegetable oils and shortenings, margarine, butter, and sugar have noticeably decreased by at least one-third.
- There is clear movement toward the increased acquisition of foods that offer more convenience in terms of preparation and service. Purchases of prepared foods (pizza, prepared sandwiches, etc.) and soups increased by 39 percent and 63 percent, respectively.
- Methods of food procurement varied among school districts and by food type. Most school districts use formal bidding procedures (fixed price, fixed price with an escalator) to purchase most product lines. However, for fresh produce, where market conditions and prices can change rapidly, informal procurement methods (bids, quotes) are used frequently.
- Almost half of all school districts participated in some form of cooperative buying with other districts, particularly for canned/staple foods and frozen foods.
- About one in five school districts purchased locally grown produce. Of those buying locally, the majority (68 percent) were school districts with 1,000 to 4,999 students. Apples were the most common locally purchased item.
- Approximately 13 percent of public school districts serving 15 percent of the student enrollment used food service management companies (FSMCs). FSMCs have concentrated their operations in mid-sized school districts but are found in districts of all sizes.
- No one procurement method produced the lowest food prices for all foods. Formal bidding methods and formal contract price terms were most successful.
A separate report is forthcoming that links the food purchase data to USDA’s nutrient databases to examine the food energy, nutrient, and MyPyramid equivalent availability of the mix of foods acquired by school districts.
Nutrient and MyPyramid Analysis of USDA Foods in Five of its Food and Nutrition Programs
This study examines the nutrient content and food group analyses of the USDA Foods distributed through five USDA food and nutrition programs in Fiscal Year 2009. The five food and nutrition programs included in this study were the National School Lunch Program (NSLP), the Child and Adult Care Food Program (CACFP), the Commodity Supplemental Food Program (CSFP), the Food Distribution Program on Indian Reservations (FDPIR), and The Emergency Food Assistance Program (TEFAP).
The study constructed representative USDA Food profiles offered and delivered to administering agencies for NSLP, CACFP, CSFP, FDPIR, and TEFAP using the lists of foods available for each program, records of foods distributed, and data contained in the following nutrient and food group databases: the USDA Food and Nutrient Database for Dietary Studies, version 4.1, the USDA National Nutrient Database for Standard Reference, release 23, and the USDA MyPyramid Equivalents Database. The computed “as offered” and “as delivered” nutrient and food group values per participant for each nutrition assistance program were compared to four dietary standards: Dietary Reference Intake, USDA’s Thrifty Food Plan, Dietary Guidelines for Americans 2010, and the Healthy Eating Index 2005.
Direct Certification in the National School Lunch Program: State Progress in Implementation
This report responds to a legislative requirement of Public Law 110-246 to assess the effectiveness of State and local efforts to conduct direct certification of children for free school meals. Under direct certification, children are determined eligible for free meals without the need for household applications by using data from other means-tested programs. The 2004 Child Nutrition and WIC Reauthorization Act required local educational agencies (LEAs) to establish a system of direct certification of children from households that receive Supplemental Nutrition Assistance Program (SNAP) benefits. This report presents information on the outcomes of direct certification for SY 2010/11.
In SY 2010/11, 85 percent of LEAs directly certified some children from SNAP-participating households. These LEAs enroll 97 percent of all students in schools participating in the National School Lunch Program. States and LEAs directly certified more than 9.9 million students during SY 2009/10. This was an increase of 23 percent (1.9 million) over the number of directly certified students a year earlier. The percentage of SNAP participating children certified for free school meals without application increased from 72 percent in SY 2009/10 to 78 percent in SY 2010/11.
States and LEAs continue to find success with both centralized and district-level matching systems. Among the most successful States there is great variation in the complexity of matching algorithms. Some States use relatively simple systems based on a small number of identifiers; others use more complex systems involving probabilistic matching. States and LEAs are making investments that have contributed to recent gains in their direct certification rates. One State improved its performance by simplifying its direct certification user interface, improving the flexibility of data upload features, and providing detailed documentation and training manuals for users at the school district level. Other State and LEA investments promise continued improved performance in coming years. These include preparing for required direct certification matching against SNAP datasets three times per year, and enhancing computer matching procedures.
Using American Community Survey Data to Expand Access to the School Meals Programs
FNS commissioned the Committee on National Statistics (CNSTAT) of the National Research Council of the National Academies to convene an expert panel to study the technical and operational issues that arise in using data from the American Community Survey (ACS) – a new continuous survey replacing the long-form survey of the decennial census—to obtain estimates of students who are eligible for free or reduced-price meals for schools and school districts. Such estimates would be used to develop “claiming percentages” that, if sufficiently accurate, would determine federal reimbursements to districts for the schools that provide free meals to all students under a new special provision that eliminates the base-year requirements of current provisions.
This report presents and describes in detail the panel’s analyses and findings; the ACS Eligibility Option (AEO) developed by the panel; the panel’s recommendations for facilitating implementation of the AEO, improving the accuracy of ACS estimates of eligibility for the school meals programs, and enhancing the quality and availability of survey and administrative data for a broad range of uses.
The panel sought to identify a universally-applicable method for estimating ACS-based claiming percentages and, if sufficient data on school district costs and increased participation under a universal free meals program could be obtained, to specify when it would be cost-beneficial for a school district to adopt the AEO for some or all of its schools. Unfortunately, ACS estimates are not sufficiently accurate for use in a one-size-fits-all version of the AEO. ACS direct estimates, when compared with administrative estimates for all such school districts and for all schools in the panel’s five case study districts, generally understate the percentage of students eligible for free meals and overstate the percentages eligible for reduced-price and full-price meals. This is particularly true in schools and districts with high percentages of students eligible for free and reduced-price meals, precisely those districts most likely to be interested in the AEO if accurate claiming percentages could be developed.
The panel suggests a more tailored approach to using ACS estimates in a new ACS Eligibility Option where an interested district would examine its annual ACS eligibility percentages to determine whether they were sufficiently close to the district’s own certification percentages. If the ACS eligibility percentages are close, the district could apply for and USDA could approve the district’s adoption of the AEO.
Note: This is a report of the National Academies' National Research Council, Committee on National Statistics (CNSTAT), which was commissioned by the USDA Food and Nutrition Service. It is available on the FNS website by permission. It may also be obtained through the National Research Council’s website.
Regional Office Review of Applications (RORA) for School Meals 2010
This is the sixth in a series of annual reports that examines administrative error incurred during the local educational agencies’ (LEAs) approval process of applications for free and reduced-price school meals. It does not examine the accuracy of household reporting of information on the applications or errors made in counting and claiming.
School districts were stratified into 28 strata defined by seven FNS regions and four size categories within each region. Almost 2,800 applications from SY 2009/10 were selected for review. The LEA’s determination of household size, total gross income, and certification status (free, reduced-price, paid) for the selected students was recorded. FNS Headquarter staff reviewed each application and made an independent assessment of household size, total gross monthly income, and certification status, based on the information on the applications. FNS’ independent assessments were compared to the LEA’s determinations.
About 98 percent of students submitting applications for meal benefits in School Year (SY) 2009/10 were certified for the correct level of meal benefits, based on information in the application files. This was comparable to school year 2008/09. Over 60 percent of those students incorrectly certified were certified for more benefits than they were entitled. More errors continue to be made processing income-based applications, with many of these errors associated with the determination of a household’s gross income.
The Food and Nutrition Service (FNS) has continued to be proactive in efforts to improve program integrity without compromising access to low-income families. Technical assistance and training materials have been provided to State and local partners to reduce administrative errors and improve program integrity. FNS will continue to conduct annual reviews of a statistical sample of LEA application eligibility determinations to measure changes in administrative error rates.
Modeling of High-Risk Indicators of Certification Error in the National School Lunch Program
This study facilitates further efforts to combat certification error in the National School Lunch Program (NSLP). The project included two distinct efforts. First, we developed statistical models to identify indicators of Local Education Agencies (LEAs) with high-risk of certification error. Second, we designed a web-based monitoring tool that will apply the model parameters to Verification Summary Report (FNS-742) data annually, and publish results for access by State Child Nutrition agencies (SAs).
Statistical models were estimated using certification error measures from two other studies: (1) the NSLP and School Breakfast Program (SBP) Access, Eligibility, and Certification (APEC) study; and (2) the Regional Office Review of Applications (RORA). Explanatory variables are from Verification Summary Report, Common Core of Data (CCD), Census Small Area Income and Poverty Estimates (SAIPE), and Bureau of Labor Statistics Local Area Unemployment Survey (LAUS). In developing these models, a variety of alternative model specifications were tested and final models were selected based on predictive power and good fit to the data. The resulting models performed well in validation analysis, indicating that they are good predictors of certification error. Model results suggest that districts with higher overall expected certification error risk include: (1) larger districts, (2) districts not using random verification samples and with higher levels of benefit reduction through verification, (3) districts not using random verification samples and with higher levels of verification nonresponse, and (4) districts with lower levels of students certified through categorical eligibility.
The statistical models produce overall certification error estimates that can be used by the monitoring tool to categorize district certification risk in a way that is simple, targeted, and responsive to time-varying LEA characteristics. In 2009, the 16 percent of districts identified as high- risk by the tool were estimated to account for 68 percent of national certification error. In the six years of data examined in the study, most districts were never high-risk and only 7 percent of districts were high-risk in every year.
The monitoring tool can be used by SAs to select LEAs for additional administrative reviews (AARs). The tool synthesizes the model-based certification error estimates and other LEA information to provide a platform for consistent application of AAR selection criteria. The tool also provides historical information from VSR since SY 2008-2009, and the tool stores SA selections for AARs for review and consideration in future years.
Evaluation of the Impact of Incentives Demonstrations on Participation in the Summer Food Service Program (SFSP): FY 2010 Arkansas and Mississippi
In the 2010 Agriculture Appropriations Act (P.L. 111-80), Congress authorized demonstration projects to develop and test methods of providing access to food for low-income children in urban and rural areas during the summer months when schools are not in regular session, as well as rigorous independent evaluations of each projects’ effectiveness. FNS launched the first of these projects, collectively known as the Summer Food for Children Demonstrations in the summer of 2012 (See Research in Progress below).
This report presents findings from the 2010 enhanced Summer Food Service Program (eSFSP) demonstrations conducted in Arkansas and Mississippi. The Arkansas demonstration offered per-lunch incentives to encourage SFSP providers (sponsors) to operate for a greater portion of the summer. The Mississippi demonstration offered new recreational or educational activities at SFSP feeding sites to induce higher levels of low income child participation.
While each State showed improvement over a number of outcomes, it is important to note that there are many factors external to the program changes demonstrated that could influence the estimates shown in this report, such as other sources of funding, delays in demonstration setup, State outreach efforts, local economic factors, and other issues. It is difficult to disaggregate the effects of the demonstrations from confounding factors that may have impacted demand for the SFSP. The results of this initial year of these demonstrations (2010) need to be carefully viewed in that context. Nonetheless, the changes observed are consistent with a generally positive impact of measures of SFSP service levels.
RESEARCH IN PROGRESS
The following section provides a brief description of some on-going FNS research and the current status of these studies:
School Nutrition Dietary Assessment IV (SNDA-IV)
This study provides up-to-date information on the school meal programs, the school environment that affects the programs, and the nutrient content of meals. FNS conducts a similar study about every five years to measure the progress schools are making toward meeting school meal nutrient standards. The last School Nutrition Dietary Assessment Study (SNDA-III) was conducted in School Year 2004/05. Data for this study, collected in SY 2009/10, will enable FNS to measure the nutritional content and quality of meals offered to students and selected by students in the school meals programs. Unlike SNDA-III, this study does not include the collection of student dietary intake data that would allow for the examination of the contribution of school meals to students’ overall diets. A final report is anticipated in 2012.
Special Nutrition Program Operations Study
This study will collect information needed to address current policy issues related to the Special Nutrition Programs. The study is designed to collect data from a nationally representative sample of about 1,500 school food authorities (SFAs) and all Child Nutrition State Agencies. Data collection for the base year will occur in School Year 2011-12 with optional data collection from the same sample of SFAs to occur the following two school years. This ongoing survey capability is intended to reduce FNS’ information collection costs and reduce the length of time necessary to obtain required data and thus provide information in a timelier manner. The surveys will provide a cross-sectional “snapshot” of program characteristics, as well as longitudinal estimates of year-to-year changes in operations. The study will provide general descriptive information on the characteristics of the school-based Child Nutrition Programs necessary for the preparation of program budgets, data on various aspects of the program administration to inform program policy and regulations, as well as data to identify areas in need of technical assistance and training. Results from the first year data collection are expected in 2012.
Summer Food for Children Demonstrations
Food insecurity continues to be a problem in summer months when school is out and free and reduced-priced school meals are not available to many low-income children. USDA has created the Summer Food for Children demonstrations which will explore and test a number of alternatives to the existing Summer Food Service Program (SFSP) over the next several summers. Rigorous evaluations are planned for each demonstration alternative.
Strengthened SFSP Demonstrations will test changes to the existing structure and delivery mechanism of SFSP to determine if they lead to increased participation.
- Two initial demonstrations – a project in Arkansas that provides incentives to extend the duration of SFSP operations, and a project in Mississippi that enhances the program with funding for enrichment activities, began in Summer 2010.
- Two additional demonstrations, testing meal delivery in rural areas, and “backpack” food packages for consumption over weekends, will begin in Summer 2011.
USDA is also exploring Household-Based Alternative Demonstrations which will provide summer food benefits using Supplemental Nutrition Assistance Program (SNAP) and WIC electronic benefit transfer (EBT) technology as the delivery mechanism, to give low-income families with children more resources to use at food stores during the summer.
- Small-scale “proof-of-concept” demonstrations of EBT-based approaches will begin Summer 2011.
- Expanded operations of successful first-year models, including additional test sites and variation in operational parameters such as the value of benefits provided, will begin Summer 2012.
- USDA will also examine the feasibility and potential benefits of using cards similar to store “gift cards” to provide more purchasing power to these families, to determine if a proof-of-concept demonstration of that approach would be worthwhile.
Healthy, Hunger-Free Kids Act of 2010 Studies
The recently passed Child Nutrition Reauthorization legislation includes a number of studies and evaluations. Some of these studies are funded, while others are subject to appropriations or have no funds provided. There are three studies that were funded that have been funded to date:
- Study of Direct Certification for Children Receiving Medicaid Benefits: Section 103 requires demonstrations of direct certification for households receiving medical assistance under the Medicaid program beginning in School Year 2012/13 in selected LEAs nationwide. The evaluation will look at the effectiveness of direct certification with Medicaid to identity children eligible for free meals that are not currently being certified for free meals through direct certification through SNAP or through the current application process. Demonstrations will be conducted in a select number of LEAs in Alaska, Florida, Illinois, and New York and state-wide in Kentucky and Pennsylvania. An interim report is due to Congress by October 1, 2014.
- Universal Meal Service in High Poverty Areas: Section 104(a) provides an alternative to household applications for free and reduced-price meals in high poverty LEAs and schools, referred to as the Community Eligibility option. For those eligible LEAs and schools, reimbursements are based on claiming percentages derived from the percent of students directly certified. The evaluation will examine both eligible LEAs and schools that elect the special assistance payments and those that do not. The evaluation will examine the impact of electing to receive special assistance payments on program integrity, availability and type of breakfast program, nutritional quality of school meals, and program participation. The Community Eligibility Option was implemented in Illinois, Kentucky, and Michigan in School Year 2011/12 and will also be implemented in the District of Columbia, New York, Ohio, and West Virginia in School Year 2012/13. A report is due to Congress by December 31, 2013.
- NSLP Indirect Cost Study: Section 307 requires a study to assess the extent to which school food authorities (SFAs) participating in the National School Lunch Program (NSLP) and School Breakfast Program (SBP) pay indirect costs, including assessments of the methodologies used to establish indirect costs, the types and amounts of indirect costs that are charged and not charged to the school food service account, and the types and amounts of indirect costs recovered by school districts. As school district budgets have tightened, some school nutrition programs have experienced escalating indirect charges which may affect the effectiveness of the federal child nutrition programs. This study is intended to determine whether the indirect costs charged and/or recovered are consistent with requirements for the allocation of costs to the school food service operation. Web surveys will be conducted with nearly 2,000 SFA Food Service Directors and business managers as well as surveys of a census of State Child Nutrition Directors and SEA Finance Officers. A report is due to Congress by October 1, 2013.
Endahl is Senior Program Analyst, Office of Research and Analysis, Food and Nutrition Service, U.S. Department of Agriculture.