FNS Research Corner Summary of Research

The FNS Research Corner provides a continuing series to summarize recently completed and current research conducted by the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) in the area of child nutrition.  For further information, contact the Office of Policy Support (OPS) at (703) 305-2017.  Links to published studies and reports as well as descriptions of ongoing studies conducted by OPS are available from the FNS Internet web site at http://www.fns.usda.gov/ops/research-and-analysis.

Research in the field

The following section provides a brief description of the research studies that are currently in the field:

 The Access, Participation, Eligibility and Certification (APEC) series

The Improper Payments Elimination and Recovery Act (IPERA) requires Federal agencies to report annually on the extent of erroneous payments in their programs, including overpayments (payments that were made improperly or exceeded the proper level) and underpayments (payments that were denied improperly or were lower than the proper level). This series is conducted at approximately five-year intervals to provide a baseline of calculated error in the School Meals programs.

The objective of the third study in this series is to calculate updated, nationally-representative estimates of annual erroneous payments in the school meals programs by replicating the “APEC methodology,” (which was implemented in APEC-I and APEC-II) using data collected in SY 2017-18. The estimates will measure error in the certification, meal claiming and aggregation processes. In addition, the study includes: (1) robust, statistically reliable national estimates of the annual amount of erroneous payments in NSLP and SBP among sub-groups, such as student, school, and SFA type; (2) a sub-study on the differences in error rates among SFAs using different program integrity implementation strategies; (3) qualitative analyses examining the reasons for erroneous payments; and (4) model-based State-level point estimates for the 48 continental States and the District of Columbia. Data collection for SY 2017-2018 is currently underway. The report on APEC-III findings will be published in Fall 2020.

Child Nutrition Program Operations Study-III

 This multi-year panel study collects and analyzes data on select operational aspects of the Child Nutrition programs. This includes a range of descriptive information from the State and school district levels, as well as data on special topics of current interest, such as the level of State and local subsidies beyond Federal reimbursements, nutrition education and promotion in schools, policies regarding unpaid meal balances, professional standards and training requirements, and the use of behavioral economics in the school food environment. The panel data covers the 4 school years starting with SY 2015-2016. Data collection for SY 2017-2018 is currently underway. Results are expected over the next three years and will be used to inform Child Nutrition program management and policy development.

Evaluation of the Direct Certification with Medicaid (DC-M) Demonstrations

Under the Healthy, Hunger-Free Kids Act of 2010, Congress mandated that FNS conduct a demonstration that directly certifies students for free school meals based on income eligibility identified through Medicaid data. Under the administrative pilot authority in Section 18(c) of the Richard B. Russell National School Lunch Act (NSLA), beginning in school year (SY) 2016-2017, FNS began to conduct new demonstrations to evaluate direct certification with Medicaid in the National School Lunch Program (NSLP) and School Breakfast Program (SBP). FNS awarded demonstrations to a second cohort of eight states for SY 2017-2018. The evaluation study for this cohort is currently underway and will examine the impact of the demonstration on participation in NSLP and SBP as well as the costs and savings to the federal government, states, and local agencies in meal claiming and reimbursements, labor time, and administrative costs. The final results from the first cohort of states will be published in early 2019, while the results from the second cohort of states will be published in late 2019.

 Evaluation of CN Special Grants: Team Nutrition Training Grants

 This study examines the Team Nutrition Training Grant (TNTG) program in order to describe the activities of grantees, their implementation strategies, and the characteristics of sites and participants. The Team Nutrition Training Grants provide states with support for nutrition education, training, and technical assistance activities to support implementation of USDA nutrition standards for meals and snacks offered through the Child Nutrition Programs (NSLP, SBP, and CACFP). Technical assistance is currently being provided to the FY 2016 cohort of TNTG grantees to support grantee evaluation activities. The study results are expected in Spring 2020 and will inform guidance for strengthening evaluation practices among future grantees.

Child Nutrition Reducing Burden Study

This project will generate the report to Congress required by the passage of the Final FY 2017 Omnibus appropriations provided funding for FNS to contract for an independent study to identify the best means of efficiently consolidating Child Nutrition Program reporting requirements for school food authorities and state agencies (House Report 114-531). This study will examine the policies and regulations, processes for reporting, and the resulting burden felt by State agencies and school food authorities. Data collection is currently underway.

Evaluation of the School Meal Data Collection Process

In FY 2014, FNS initiated a study to provide a comprehensive view of the data collected at the Federal, State, and local levels to determine what data are available that could inform program policy without increasing the data collection burden on program partners. This study will continue and extend this effort to determine the accuracy of the administrative data submitted to the Federal level on key measures, such as meal counts and program participation. Specifically, this study will examine the following FNS forms: FNS-10 – Report of School Operations; the FNS-742- SFA Verification Collection Report; and the FNS-834-State Agency Direct Certification Rate Data Element Report. The study will identify potential points of error related to data collection and aggregation and identify best practices in ensuring data accuracy. Data collection is currently underway.

Summer Meal Program Participant Characteristics and Meal Analysis of Quality (Summer Meals Study)

The Summer Meals Study will provide a comprehensive, up-to-date understanding of sponsor, site, and child-level participation in the Summer Food Service Program (SFSP) and Seamless Summer Option (SSO). This is the first national study to simultaneously examine program characteristics in relation to participation and satisfaction with the programs among households.  Specifically, the study will examine children’s and their caregivers’ satisfaction with the program, ways in which FNS could better accommodate their needs, and why families living in eligible areas are not participating in the Summer Meal Programs. The study will also describe sponsors’ and sites’ meal service characteristics, the nutritional quality of the meals offered and the activities the sites provide in an effort to determine how these factors influence children’s participation and satisfaction with the program. In addition, the study will investigate the barriers and facilitators to preparing and serving summer meals, and identify reasons former sponsors are no longer participating in the Summer Meal Programs. Data collection is expected to begin in the Spring and Summer of 2018.

Assessment of the Administrative Review Process

FNS has recently launched a new Administrative Review process for the school meals programs. This study will assess the extent to which these reviews effectively identify risk areas and noncompliance with program requirements. Findings will help to ensure that State agencies are able to provide meaningful technical assistance and require appropriate corrective action for noncompliance. The study will also examine the management of review and oversight resources. Where weaknesses in process or outcomes are found, the project will seek to identify more effective methods already in use in Government and/or industry for application in school meals. Data collection is expected to begin in the Summer of 2018.

Assessment of Alternatives to the State Administrative Expense (SAE) Formula

The Child Nutrition programs have changed substantially since the SAE formula was last revised in the 1990s. This project will assess the effectiveness of the current formula used for State administrative expense fund allocations and develop and test a range of possible alternative algorithms to improve the formula. Data collection is expected to begin in the Summer of 2018.

RECENTLY COMPLETED RESEARCH

Evaluation of Pilot Project for Procurement of Unprocessed Fruits and Vegetables

Authorized by Section 4202 of the Agricultural Act of 2014 (P.L. 113-79, the 2014 Farm Bill), the USDA Pilot Project for Procurement of Unprocessed Fruits and Vegetables was designed to provide States with additional flexibility in the procurement of unprocessed fruits and vegetables. Participating States and school food authorities (SFAs) can purchase approved items with existing USDA Foods National School Lunch Program entitlement funds from any USDA Pilot-authorized vendor in support of the school meal standards. Using administrative purchase data from three school years (SY 2013-2014, 2014-2015, 2015-2016) and qualitative interview data with State Distributing Agency officials, the evaluation examines the quantity and cost of each type of fruit and vegetable in the Pilot, characteristics of participating school food authorities, States’ perceptions of the challenges and benefits to participation, and comparisons to USDA Foods and USDA Department of Defense Fresh Fruit and Vegetable Program. The final report, along with a series of infographics, was published in March 2018.

Key results include:

• In the first year, Pilot purchases of unprocessed fruits and vegetables totaled 692,741 pounds—and nearly 5.8 million pounds in the second year. By the end of the second year, twice as many vendors were approved to participate in the Pilot as compared to the first year.
• Overall, participating States spent almost $600,000 in the first year and almost $4.7 million in the second year. For the five initial States, Pilot purchases as a proportion of all unprocessed fruit and vegetable purchases increased in the second year (Table 2).
• The Pilot provided new flexibility for SFAs to spend entitlement funds on unprocessed fruits and vegetables to meet school meal requirements. Participating SFAs obtained 70 different types of unprocessed fruits and vegetables through the Pilot, including 42 varieties of produce they did not previously receive through the USDA Foods or USDA DoD Fresh Programs, even though some items were available through those programs

CACFP Tiering Series

The Improper Payments Elimination and Recovery Act (IPERA) of 2010 (Public Law 111-204) requires all federal agencies to calculate the amount of erroneous payments in Federal programs and to periodically conduct detailed assessments of vulnerable program components. This report provides the 2015 national estimate of incorrect payments for the meal reimbursements.  The assessment calculates both overpayments and underpayments for FDCHs that earn either Tier 1 (higher) or Tier II (lower) reimbursements depending on the location and circumstances of the child care provider or the participating children. Tiering errors result in improper payments because misclassified family daycare homes do not receive the appropriate level of reimbursement for the meals and snacks provided to the children.  The assessment also estimates the dollar amount of improper payments attributable to these tiering errors.

Key results include:

• Approximately 99 percent of family daycare homes nationally were correctly classified. The study estimates the average 2015 tiering error rate at 1.3 percent, based on incorrect tiering for 0.9 percent of Tier I and 3.7 percent of Tier II FDCHs, respectively. 

• An estimated 1.0 percent of all FDCH meal claims were reimbursed at an improper rate, which represents 5.4 million meals.  Meal claims reimbursed at improper rates ranged from 0.9 percent for Tier I to 1.7 percent for Tier II family daycare homes. 

• The cost of improper payments due to tiering errors, or the improper payment rate, was 0.54 percent (0.47 percent for Tier I FDCHs and 1.83 percent for Tier II FDCHs) of the estimated $771.1 million in family daycare home reimbursements for 2015. The estimated national cost of misclassification errors was $3.37 million for Tier I FDCHs and $1.09 million for Tier II FDCHs, resulting in a gross cost (overpayments and underpayments) of $4.46 million for all FDCHs (Figure 3) and net cost (overpayments minus underpayments) of $2.28 million.

• The improper payment rate of 0.54 percent for 2015 is similar to the estimate of 0.84 percent for 2014 and not statistically different from estimates from prior years.  The small variations in estimates of misclassification errors since assessments began are consistent with what would be expected in the presence of sampling error.

Evaluation of Demonstration Projects to End Childhood Hunger: Interim Evaluation Report

This study—authorized by the 2010 Child Nutrition Act—tests innovative strategies to end childhood hunger and food insecurity. The interim evaluation report describes (1) the demonstration projects, (2) planning and early implementation activities, and (3) findings from the baseline data collection for four projects located within Chickasaw Nation, Kentucky, Nevada, and Virginia. A fifth demonstration project was implemented in Navajo Nation but not evaluated due to changes in program design. The demonstrations, which occurred during 2015-2017 and operated for 12 to 24 months, include:

• Chickasaw Nation

Provided monthly home-delivered food boxes containing shelf-stable, nutritious foods and a $15 voucher for fresh fruits and vegetables to children eligible for free school meals or attending a school where all children receive free school meals. School districts located in 12 rural counties in the Chickasaw Nation service area in Oklahoma participated.

•  Kentucky

Provided a $45 to $55 average increase in monthly household Supplemental Nutrition Assistance Program (SNAP) benefits, calculated as a fixed-income deduction based on the county’s average distance to the grocery store plus an additional 10 percent earned-income deduction to SNAP households residing at least 4-10 miles from grocery stores, with children under age 18 and positive net income. The demonstration operated in 17 rural counties in eastern Kentucky.

• Nevada

Provided an additional $40 per month in SNAP benefits per eligible child (treatment group 1) or same additional SNAP benefits plus case management and nutrition education (treatment group 2) to households participating in SNAP with incomes below 75 percent of the Federal poverty level with at least one child under age 5.

• Virginia

Provided three meals during the school day and food packages for weekends and school breaks, $60 monthly Electronic Benefit Transfer (EBT) benefits per eligible child during summer months, and nutrition education for parents and guardians. In the participating schools in the rural southwest part of the State and urban districts in Richmond, all children were offered school meals and food packages; those eligible for free or reduced-price school meals are also offered summer EBT benefits.

The Interim report was published in February 2018.

RESEARCH IN PROGRESS

The following section provides a brief description of ongoing FNS research and the current status of these studies:

School Meal Programs Research

School Nutrition and Meal Cost Study (SNMCS)

 The School Nutrition and Meal Cost Study (SNMCS) will explore both the nutrition and cost domains of school meals in an integrated study design.  It will collect a broad range of data from a nationally representative sample of about 500 School Food Authorities (SFAs), 1,200 schools, 2,400 students and parents, and a large sample of school meals (over 5,000 lunches and 3,000 breakfasts) during School Year 2014/15. The data collection includes the administration of several different types of instruments and modes, including self-administered web-based SFA director and school principal surveys, a foodservice manager survey, an electronic menu survey, competitive foods checklists, cafeteria environment observation, plate waste observation, Automated Multiple Pass Method 24-hour dietary recalls, measurement of student’s height and weight, student/parent surveys, meal cost interviews, and collection of administrative cost data.

These data will provide needed information about how federally-sponsored school meal programs are operating after implementation of the new nutrition standards and other changes in regulations. Comparisons of results of this study with previous School Nutrition and Dietary Assessment (SNDA) and School Lunch and Breakfast Cost (SLBC) studies will provide information that can be used to assess the effects of the new nutrition standards on foodservice operations, the nutrient content of school meals as offered and served, meal costs and revenues, and student participation and dietary intake. The integrative structure of the SNMCS will support analysis of the relationships among substantive areas such as nutritional quality of meals, meal costs, student participation, and plate waste. The study will produce five separate reports in Summer 2018 summarizing study findings (SFA/school characteristics and food service operations; nutrition quality of meals; meal costs; student participation and satisfaction, plate waste, and dietary intakes of participants and nonparticipants; and a study methodology report) and a stand-alone summary of findings.

School Nutrition and Meal Cost Study-II (SNMCS-II)

 Given FNS’s commitment to periodically review the school meal programs and provide critical information to policymakers and other stakeholders, SNMCS-II will build on the results of SNMCS-I.  It will provide a comprehensive picture of the school meal programs in School Year 2019-2010 and will provide critical information about the nutritional quality, cost, and acceptability of school meals.  The integrated structure of SNMCS-II will further the precedent set by SNMCS-I and support analysis of the relationships among key domains including the relationship between the nutritional quality and cost of school meals.  This study will also assess the need for meal reimbursement adjustments for five outlying areas (Alaska, Guam, Hawaii, Puerto Rico, and the U.S. Virgin Islands).  Estimates of the costs to produce school lunches and breakfasts in these outlying areas will be compared to those costs in the contiguous 48 States.

Community Eligibility Provision Characteristics Study

 With the expansion of the Community Eligibility Provision (CEP) nationwide in School Year 2014-2015, FNS is in need of updated information on the characteristics of participating and eligible non-participating CEP schools and school districts.  This study will examine operational issues and perceived incentives and barriers for adopting CEP as well as the impacts on lunch and breakfast participation and per meal revenues.  Using data collected in School Year 2016-2017, the study will update the information obtained in the implementation component of the earlier Community Eligibility Provision Evaluation of the early implementers of CEP (https://www.fns.usda.gov/community-eligibility-provision-evaluation).

Successful Approaches to Reduce Sodium in School Meals

FNS is currently conducting a study on the market availability of foods served in school meals that meet the current and future sodium requirements. The study will also identify best practices in schools that are successfully meeting sodium targets that could be used to provide technical assistance to School Food Authorities developing lower sodium menus. The final report is expected in late 2018.

Study of Non-response to School Meals Income Verification

The objectives of the study are to evaluate the accuracy of the school meals benefit determination process nationwide, note the results and how they compare to the results from a similar FNS study from 2004, and discuss policy implications of the findings. As part of the evaluation, the study will also examine the current methodologies and processes used by local education agencies (LEAs) to conduct the school meals verification process. Data will be collected in SY 18-19. The final report is expected in late 2019.

Evaluation of CN Special Grant: Administrative Reviews and Training (ART) Grants

 This study will provide FNS with formative research for the Administrative Review and Training (ART) Grant–funded on the implementation of interventions and the process that followed to meet intended grant outcomes. This study will help FNS understand any effects of the ART grant interventions on administrative processes, examine long-term sustainability of grant-funded activities and describe challenges to ART grant implementation and sustainability. Data collection is planned for Spring of 2019.

Study of School Food Authority (SFA) Procurement Practices

This study is a nationally representative descriptive study of current procurement practices used by SFAs, including the scope and nature of food service management company contracts, cooperative buying arrangements, recordkeeping used to track rebates, discounts, credits, local purchasing preferences, and food purchasing specifications. The final report is expected in 2020.

Farm to School Evaluations/Census (F2S)

Farm to School programs aim to strengthen knowledge about agriculture, food, nutrition, and the environment; increase children’s consumption of fruits and vegetables; increase opportunities for farmers and other food producers; and support economic development. This census study will provide information on the magnitude of local sourcing procurement and the prevalence of other farm to school activities during School Year (SY) 2017/2018 and SY 2018/2019. This study will update information from the farm to school census study of SY 2013/2014 and SY 2014/2015. The final report is expected in 2020.

Evaluation of the Independent Review Process

The Healthy Hunger-Free Kids Act (HHFKA) included several provisions to help increase effectiveness and integrity in Child Nutrition programs. As part of an effort to decrease certification error during the eligibility determination process, Section 304 of the HHFKA requires certain local educational authorities to conduct a second, independent review of applications for free and reduced price meal benefits before eligibility determinations are made. FNS published a final rule establishing the requirements related to the provision, which became effective in SY 2014-2015. This evaluation will assess the process and effectiveness of this requirement, and help to determine if changes in the process are necessary. The final report is expected in 2020.

Direct Certification in the National School Lunch Program: Report to Congress

This project will generate the required annual report to Congress on the effectiveness of State and local efforts to certify SNAP participant children for free school meals without the need for household applications. This report will respond to a legislative requirement of the Food, Conservation, and Energy Act of 2008 (FCEA) (Public Law [P.L.] 110-234, also known as the 2008 Farm Bill) to assess the effectiveness of State3 and local efforts to directly certify children for free school meals under the National School Lunch Program (NSLP). The 2008 Farm Bill requires annual Reports to Congress. This will be the ninth report in the series, The Food and Nutrition Service (FNS) will use results from this report in identifying those States that must develop and implement direct certification continuous improvement plans (CIPs), as required by Section 101 of the Healthy, Hunger-Free Kids Act (HHFKA) of 2010 (P.L. 111-296). It will calculate state-specific performance metrics from data reported by State agencies and school food authorities. The project will also highlight direct certification operations in a handful of States for insight into successful strategies and promising practices. The final report is expected in Summer 2018.

Review of Child Nutrition Data and Analysis for Program Management

The current routine data collection requirements for the child nutrition programs have their roots in the paper and early computer eras and reflect concerns with paperwork and reporting burden. In many cases, data collected at the service delivery point (e.g., school or child care provider) are aggregated at one or more level before reporting to FNS as State data, resulting in a significant loss of potentially valuable information along the way. Some States/school districts have developed more sophisticated systems; however, there is no comprehensive approach to utilizing program data timely and effectively for program oversight and management improvement at the Federal, State and local levels. This project will provide a comprehensive assessment of the FNS child nutrition management information needs and make recommendations for information system improvements, as well as identify the reporting and recordkeeping requirements and costs needed to support such improvements. The final report is expected in Spring 2019.

Child and Adult Care Food Program Research

Child and Adult Care Food Program (CACFP) Provider Characteristics Study

 This study will provide a description of the Child and Adult Care Food Program (CACFP) and provide valuable information on how the program has changed in recent years. The study will also focus on a subgroup of sponsors and centers—CACFP At-Risk Afterschool Program. Data was collected in 2015 and will provide information on the characteristics of 1) sponsors; 2) homes and centers; and 3) children served. The final report is expected in Summer of 2018.

CACFP Meal Claims Feasibility

 The aim of this IPERA compliance reporting feasibility study is to develop a reliable method of assessment of erroneous meal claims in CACFP-Family Day Care Homes, test the method on a sample of FDCHs for the purpose of estimating the rate of improper payments and provide annual estimates of erroneous payments. Specifically, the study focuses on accurately estimating meals that are claimed but not served.  Previous efforts to develop a reliable method for validating meal reimbursement vs. meals served to children at FDCHs produced unreliable results. However, a study in 2010 held more promise, when parents’ report of child attendance was matched with observed meals served at the center. Consequently, an approach that included attendance –time-derived meals (ATDMs) with parent report of child drop off and pick up times combined with data on meals served, presented the most meaningful option for identifying a method for estimating meals claimed errors in these settings. The study objectives include: 1) design a method to accurately estimate the rate of erroneous payments; 2) test the method on a sample of family daycare homes (FDCHs); and 3) provide feasibility analyses and a report of the reliability of the method. If the method is deemed unreliable, an alternative approach will be proposed. Data collection was recently completed and the final report is expected in late 2018.

CACFP Tiering Aging Study

The Improper Payments Information Act of 2002 (IPIA) (Public Law 107-300) requires all federal agencies to calculate the amount of erroneous payments in Federal programs and to periodically conduct detailed assessments of vulnerable program components. This assessment of the family daycare homes (FDCHs) component of CACFP provides a national estimate of the share of the roughly 125,000 participating FDCHs that are approved for an incorrect level of per meal reimbursement, or reimbursement “tier” for their circumstances. FDCHs can earn Tier I (higher) or Tier II (lower) reimbursements depending on the location and circumstances of the child care provider or the participating children.  Tiering errors result in improper payments because misclassified family daycare homes do not receive the appropriate level of reimbursement for the meals and snacks provided to the children. The assessment also estimates the dollar amount of improper payments attributable to these tiering errors. Release of the report is expected in late 2018.

Erroneous Payments in Childcare Centers (EPICCS)

Building upon the methods developed for schools in the Access, Participation, Eligibility, and Certification (APEC) studies, this study has three important goals: (1) provide FNS with a

reliable measure to estimate erroneous payments in the child care center component of the Chilc and Adult Care Food Program (CACFP), (2) develop reliable estimation models that FNS can use to update erroneous payment estimates annually, and (3) prepare a white paper on methodologies to compute state-level estimates of erroneous payments. Data collection was recently completed and the final report will be published in Summer 2019.

Study of Nutrition and Wellness Quality in Child Care Settings (SNAQCS)

The Study on Nutrition and Wellness Quality in Child Care Settings  (SNAQCS) is a congressionally mandated study that will 1) assess nutrition and wellness policies/practices and meal quality for infants and children in CACFP child care settings; 2) describe food and nutrient intakes of infants and children in CACFP child care centers and outside of child care; 3) determine the meal costs and revenues in CACFP child care centers; and, 4) describe and assess plate waste in CACFP child care centers. Data collection was recently completed and the final report is expected in 2020.

Research Conducted through Grant Activities

Rural Child Poverty Nutrition Center

The purpose of this grant is to establish the Rural Child Poverty Nutrition Center (RCPNC) at the University of Kentucky. The objective of the Center is to reduce child food insecurity by improving program coordination among USDA-FNS Child Nutrition Programs and other nutrition assistance programs. RCPNC has formed strategic partnerships to ensure successful program implementation, relying on guidance in terms of content area expertise, technology, training, and outreach. The University of Kentucky will manage sub-grantees who have proposed creative strategies to increase participation in these programs among families with children in persistently poor rural counties in 15 states. Sub-grantees’ programs will be evaluated using qualitative and quantitative methodologies, and evaluation findings and implementation strategies will be widely disseminated. The final report of project activities will be submitted to FNS in June 2019.