FNS Research Corner Summary of Research

RECENTLY COMPLETED RESEARCH

 Evaluation of the Elementary Schools Pilot Project for Canned, Frozen, or Dried Fruits and Vegetables in the Fresh Fruits and Vegetable Program

This report shares results from the evaluation of a pilot project called for in Section 4214(d) of the Agricultural Act of 2014 to offer canned, frozen, or dried (CFD) fruits and vegetables as part of the Fresh Fruit and Vegetable Program (FFVP). The study evaluated four key objectives: (1) impact on students’ fruit and vegetable consumption, (2) impact on school participation in the FFVP and operation of the program, (3) implementation strategies used by schools participating in the pilot, and (4) acceptance of the pilot project by key stakeholders. Four states (Alaska, Delaware, Kansas, and Maine) applied for and were accepted into the pilot.  The pilot study included 58 schools from 31 districts across the four states.

Key findings include:

• Access to fresh fruits and vegetables was a key factor for states deciding to apply for the pilot. All four pilot states cited difficulty in accessing fresh fruits and vegetables as a major reason for applying, while 59% of non-pilot states indicated that they did not apply due to their schools having adequate access to fresh fruits and vegetables.

• On an average school week during the pilot, 50% of schools offered both fresh and CFD fruits and vegetables, 41% offered only fresh fruits and vegetables, 6% offered only CFD fruits and vegetables, and 3% did not offer any fruits or vegetables as part of FFVP.

• Students’ daily in-school consumption of fruits and vegetables decreased by approximately one-fourth of a cup after the FFVP-CFD was implemented. A decrease in consumption of fresh fruit was the largest cause of the overall change with fresh fruit consumption declining by more than one-fifth of a cup after the pilot was implemented.

• Most foodservice managers indicated that the FFVP-CFD pilot improved snack quality, quantity, and variety, and school officials in the pilot areas wanted it to continue.  However, two-thirds of parents preferred their child to receive fresh fruits and vegetables instead of canned, frozen or dried options.

Direct Certification in the National School Lunch Program – State Progress in Implementation, School Year 2014/15

This report responds to a legislative requirement of Public Law 110-296 to assess the effectiveness of state and local efforts to conduct direct certification of children for free school meals.  Under direct certification, children are determined eligible for free meals without the need for household applications by using data from other means-tested programs.  The 2004 Child Nutrition and WIC Reauthorization Act required local educational agencies (LEAs) to establish a system of direct certification of children from households that receive Supplemental Nutrition Assistance Program (SNAP) benefits.  For SY 2014/15 and beyond, states that fail to achieve a direct certification rate of at least 95% are required to develop and implement continuous improvement plans (CIPs). This report presents information on the outcomes of direct certification for SY 2014/15.

Key findings from the 2014-2015 study include:

• States and LEAs directly certified 11.1 million children – 9.8 million children based on participation in SNAP and 1.3 million children based on participation in other programs. This total represents a decrease of 11% from the previous year.

• SY 2014-2015 was the first year the Community Eligibility Provision (CEP) was available nationwide for eligible LEAs. When this count of SNAP-participant students in CEP and special provision schools in non-base years is added to the total, 15.1 million children were directly certified in SY 2014–2015 compared to 14.5 million children in SY 2013-2014, an increase of 4%.

• In SY 2014-15, an estimated 91% of school-age SNAP participants were directly certified for free meals. This represents a four percentage-point improvement from SY 2013-2014.

• Twenty-three States and the District of Columbia achieved direct certification rates at or above HHFKA’s 95% performance target.

Regional Office Review of Applications (RORA) for School Meals 2015

This is the eleventh in a series of annual reports that examines administrative error incurred during the local educational agencies’ (LEAs) approval process of applications for free and reduced-price school meals. It does not examine the accuracy of household reporting of information on the applications or errors made in counting and claiming.

Key results include:

• In SY 2014/15, about 98.1% of students submitting applications for meal benefits were certified for the correct level of meal benefits.

• Certification error (1.9%) was lower than previous years (2004-2014) among all applications approved or denied. Of the 1.9% incorrectly determined eligible for their meal status, 0.14% were certified for fewer benefits than eligible and 1.78% were certified for more benefits than eligible.

• More errors continue to be made processing income-based applications. On income-based applications, more errors are associated with the determination of a household’s gross income than household size.

Special Nutrition Program Operations Study, School Year 2013-2014

The Special Nutrition Program Operations Study is a multi-year study designed to provide FNS with key information about the operational policies and practices of School Food Authorities (SFAs) and State Child Nutrition Agencies that can be used to address policy needs, develop informed regulations and guidance, and provide needed technical assistance.  Survey data is collected from a nationally representative sample of over 1,400 SFA directors in public school districts operating the National School Lunch Program and a census of State Child Nutrition Directors. Final reports sharing the findings from SY 2012/13 and SY 2013/14 were released in October 2016.

Information from the SY 2013/14 study provides a baseline for observing the improvements resulting from the implementation of provisions found in the Healthy, Hunger-Free Kids Act of 2010. Key results include:

• Ninety-seven percent of SFAs were certified to receive the additional 6-cent reimbursement for meeting the new meal pattern requirements in SY 2013-14. Of those not certified, most did not submit certification materials due to inadequate training to complete the application process, limited staff resources, or paperwork burden.

• SFAs continued to report that maintaining the budget with increased food costs (59.0%), student acceptance (58.8%), and maintaining student participation (56.3%) were very challenging in implementing the new meal patterns. SFAs were more challenged in SY 2013-14 in finding products that meet standards in part due to implementing the new breakfast requirements and the whole grain-rich requirement.

• In SY 2013-14, 71% of SFAs reported changing their USDA Foods use in support of the new meal requirements. SFAs are making better use of the wide variety of fruits, vegetables, whole grain-rich foods, and low-sodium, low-fat products available as USDA Foods.

Examining the Potential to Expand Data Matching in the School Meal Program Eligibility and Verification Processes

This analysis was conducted to determine whether any additional means-tested programs might be feasible for use in the direct certification of school-age children participating in school meals or for verification of household income on meal applications. Study objectives included (1) summarizing the experiences, benefits, challenges, and barriers of current data matching practices in State Agencies (SAs) and local education agencies (LEAs) and (2) identifying additional data sources with potential for improving matching processes in the school meals programs.

Key findings include:

• The Income Eligibility Verification Systems (IEVS) database and six means-tested programs were identified as potential candidates for data matching in the NSLP. The means-tested programs include: Special Supplemental Nutrition Program for Women, Infants, and Children; Affordable Care Act Health Insurance Marketplace; Public Housing and Housing Choice Voucher Programs; Low Income Home Energy Assistance Program (LIHEAP); Weatherization Assistance Program; and Earned Income Tax Credit.

• Computer matching to wage and benefit information for verification of school meal applications is not feasible. Matching to wage data or other income reporting systems such as the IEVS systems requires individual social security numbers (SSN) for each adult household member; only the last four digits are currently required to apply for meal benefits.
• Only the Public Housing Programs and LIHEAP were deemed potentially feasible for NSLP data matching, and each has limitations. Income eligibility limits are established by the U.S. Department of Housing and Urban Development (HUD) for both programs and vary based on the median income for the county or metropolitan area where participants live.
• To evaluate data matching with other means-tested programs, States need data on the number of students that could be certified that are not already directly certified through existing programs. If there is substantial overlap between households participating in these new means-tested programs and programs currently used in NSLP data matching, it may not be cost effective to add these programs.
• States perceived a number of challenges to adding means-tested programs to current data matching processes. These challenges include establishing data sharing agreements with State Agencies, addressing the variations in privacy and confidentiality policies, and training staff to effectively perform the data matches. Start-up costs and time requirements for integrating additional program data files into existing data matching processes can also pose a challenge.

RESEARCH IN PROGRESS

The following section provides a brief description of some ongoing FNS research and the current status of these studies:

Child Nutrition Operations Study II (CN-OPS II)

The objective of the Child Nutrition Program Operations Study II (CN-OPS II) is to collect timely data on policy, administrative, and operational issues within the CN programs. The study will collect data via online survey in SY 2015-16 and SY 2016-17 with an optional two years of data collection. FNS will identify the most relevant policy needs for each of the four years of data collection.  The contractor will develop survey instruments to address these policy needs. Each year, the survey instruments will include some repeated questions as well as modules of survey items for specific policy needs relevant to that year.  The study will obtain data for each school year individually and will examine trends over the four years. The final reports for Year 1 and Year 2 are scheduled for release over the next several years.

 School Nutrition and Meal Cost Study (SNMCS)

The School Nutrition and Meal Cost Study (SNMCS) will explore both the nutrition and cost domains of school meals in an integrated study design.  It will collect a broad range of data from a nationally representative sample of about 500 School Food Authorities (SFAs), 1,200 schools, 2,400 students and parents, and a large sample of school meals (over 5,000 lunches and 3,000 breakfasts) during SY 2014/15. The data collection includes the administration of several different types of instruments and modes, including self-administered web-based SFA director and school principal surveys, a foodservice manager survey, an electronic menu survey, competitive foods checklists, cafeteria environment observation, plate waste observation, Automated Multiple Pass Method 24-hour dietary recalls, measurement of student’s height and weight, student/parent surveys, meal cost interviews, and collection of administrative cost data.

These data will provide needed information about how federally-sponsored school meal programs are operating after implementation of the new nutrition standards and other changes in regulations.  Comparisons of results of this study with previous School Nutrition and Dietary Assessment (SNDA) and School Lunch and Breakfast Cost (SLBC) studies will provide information that can be used to assess the effects of the new nutrition standards on foodservice operations, the nutrient content of school meals as offered and served, meal costs and revenues, and student participation and dietary intake.  The integrative structure of the SNMCS will support analysis of the relationships among substantive areas such as nutritional quality of meals, meal costs, student participation, and plate waste. The study will produce five separate reports in 2017 summarizing study findings (SFA/school characteristics and food service operations; nutrition quality of meals; meal costs; student participation and satisfaction, plate waste, and dietary intakes of participants and nonparticipants; and a study methodology report) and a stand-alone summary of findings.

The Access, Participation, Eligibility and Certification (APEC) series

The Improper Payments Elimination and Recovery Act (IPERA) requires Federal agencies to report annually on the extent of erroneous payments in their programs, including overpayments (payments that were made improperly or exceeded the proper level) and underpayments (payments that were denied improperly or were lower than the proper level). This series is conducted at approximately five-year intervals to provide a baseline of calculated error in the School Meals programs.

The objective of the third study in this series is to calculate updated, nationally-representative estimates of annual erroneous payments in the school meals programs by replicating the “APEC methodology,” (which was implemented in APEC-I and APEC-II) using data collected in SY 2017-18. The estimates will measure error in the certification, meal claiming and aggregation processes. In addition, the study includes: (1) robust, statistically reliable national estimates of the annual amount of erroneous payments in NSLP and SBP among sub-groups, such as student, school, and SFA type; (2) a sub-study on the differences in error rates among SFAs using different program integrity implementation strategies; (3) qualitative analyses examining the reasons for erroneous payments; and (4) model-based State-level point estimates for the 48 continental States and the District of Columbia. The report on APEC-III findings will be published in Fall 2020.

Evaluation of the Direct Certification with Medicaid for Free and Reduced-Price Meals Demonstrations

Under the Healthy, Hunger-Free Kids Act of 2010, Congress mandated that FNS conduct a demonstration that directly certifies students for free school meals based on income eligibility identified through Medicaid data. Under the administrative pilot authority in Section 18(c) of the Richard B. Russell National School Lunch Act (NSLA), beginning in school year (SY) 2016-2017, FNS began to conduct new demonstrations to evaluate direct certification with Medicaid in the National School Lunch Program (NSLP) and School Breakfast Program (SBP). Because direct certification has the potential to improve student access to school meals, reduce administrative burden for schools and LEAs, and improve certification accuracy, FNS awarded demonstrations to a first cohort of seven states to evaluate the impact of using Medicaid data to directly certify students for both free and reduced price meal eligibility in SY 2016-2017. The study examines the impacts of the demonstrations on participation in NSLP and SBP, as well as costs and savings to the federal government, states, and local agencies in terms of meal claiming and reimbursements, labor time, and administrative costs. The report on the first year of demonstrations is expected in late 2018.

Direct Certification in the National School Lunch Program: Report to Congress

This project will generate the required annual report to Congress on the effectiveness of state and local efforts to certify SNAP participant children for free school meals without the need for household applications. It will calculate state-specific performance metrics from data reported by State Agencies and School Food Authorities. The project will also highlight direct certification operations in a handful of states for insight into successful strategies and promising practices.

Online Application Feasibility Study

FNS recently designed a prototype web-based school meal application to increase the accuracy of household reporting of information to determine eligibility for free and reduced price meals. This new study will test the utility of the new FNS online school meal application as a web- based survey to estimate meal certification error in the population in the years between the Access, Participation, Eligibility, and Certification (APEC) studies. The online school meal application will be tested in a limited number of areas to determine comparability with the results of the APEC household survey. The resulting white paper will include recommendations for any potential changes to the online school meal application in order to increase the accuracy of eligibility information.

Cognitive Testing of School Meal Applications

This study will conduct cognitive testing of the paper-based school meals application forms, as well as usability testing of the new electronic prototype school meals application. Through cognitive testing with real users, the study will examine the school year (SY) 2016-17 prototype paper application for free and reduced price school meals, which was published by FNS in April 2016. Usability testing of the electronic prototype school meals application will evaluate the respondent’s effectiveness and efficiency in application completion, and satisfaction with the experience of interacting with the application. The U.S. Census Bureau (USCB), Center for Survey Measurement is conducting this study through an inter-agency agreement with FNS. Data collection for the cognitive testing will occur January-April 2017, and data collection for usability testing will occur April-July 2017.

Study of Non-Response to the School Meals Application Verification Process

The objectives of the study are to evaluate the accuracy of the school meals benefit determination process nationwide, note the results and how they compare to the results from a similar FNS study from 2004, and discuss policy implications of the findings. As part of the evaluation, the study will also examine the current methodologies and processes used by local education agencies (LEAs) to conduct the school meals verification process.

 Successful Approaches to Reduce Sodium in School Meals

FNS is currently conducting a study on the market availability of foods served in school meals that meet the current and future sodium requirements. The study will also identify best practices in schools that are successfully meeting sodium targets that could be used to provide technical assistance to School Food Authorities developing lower sodium menus. The final report is expected in 2018.

Evaluation of Team Nutrition Training Grants

This study examines the Team Nutrition Training Grant (TNTG) program in order to describe the activities of grantees, their implementation strategies, and the characteristics of sites and participants. The Team Nutrition Training Grants provide states with support for nutrition education, training, and technical assistance activities to support implementation of USDA nutrition standards for meals and snacks offered through the Child Nutrition Programs (NSLP, SBP, and CACFP). Technical assistance is being provided to the FY 2016 cohort of TNTG grantees to support grantee evaluation activities. The study results will inform guidance for strengthening evaluation practices among future grantees.

CACFP Tiering Series

The Improper Payments Information Act of 2002 (IPIA) (Public Law 107-300) requires all federal agencies to calculate the amount of erroneous payments in Federal programs and to periodically conduct detailed assessments of vulnerable program components. This assessment of the family daycare homes (FDCHs) component of CACFP provides a national estimate of the share of the roughly 125,000 participating FDCHs that are approved for an incorrect level of per meal reimbursement, or reimbursement “tier” for their circumstances. FDCHs can earn Tier I (higher) or Tier II (lower) reimbursements depending on the location and circumstances of the child care provider or the participating children.  Tiering errors result in improper payments because misclassified family daycare homes do not receive the appropriate level of reimbursement for the meals and snacks provided to the children. The assessment also estimates the dollar amount of improper payments attributable to these tiering errors. Release of the 2015 report is expected soon.

CACFP Erroneous Payments in Child Care Centers (EPICCS)

Building upon the methods developed for schools in the Access, Participation, Eligibility, and Certification (APEC) studies, this study has three important goals: (1) provide FNS with a reliable measure to estimate erroneous payments in the child care center component of the CACFP, (2) develop reliable estimation models that FNS can use to update erroneous payment estimates annually, and (3) prepare a white paper on methodologies to compute state-level estimates of erroneous payments. Data collection for this study is currently underway. The study includes a final report and white paper that will be released in Summer 2019.

Study on Nutrition and Wellness Quality of Child Care Settings (SNAQCS)

The Healthy, Hunger-Free Kids Act of 2010 directs USDA to conduct a study of child care providers to assess the nutritional quality of foods provided in child care settings, opportunities for physical activity, and facilitators and barriers to providing healthy foods and physical activity and participation in CACFP. This study will also examine the costs of preparing meals for CACFP providers.  The final report is expected in 2020.

Characteristics of Summer Food Service Participants and Programs

The Summer Food Service Program (SFSP) serves only a small percentage of the children who receive free and reduced price meals through the National School Lunch Program during the school year. This national study aims to provide detailed information on SFSP operations and administration at the state agency, sponsor, and site levels, as well as characteristics and experiences of the providers and a small sample of participants and eligible nonparticipants. The data were collected in Summer 2015. The findings and analyses of this study are intended to replicate and expand upon the last national study of the SFSP published in 2003 by the Economic Research Service. Since the 2003 study, significant policy changes have been implemented that could affect participation in the program by children, sponsors, and sites, so this study will also provide a picture of changes in participation since the implementation of recent policy developments affecting the SFSP and Seamless Summer Option (SSO). The final report is expected in 2017.

Summer Food Service Program Participant Characteristics and Meal Analysis of Quality

The study will provide up-to-date and nationally representative data on participant characteristics and quality of summer meals.  Specifically, the study will collect information on 1) reasons why children and their caregivers participate in SFSP and their satisfaction with SFSP; 2) characteristics of participating families and eligible, non-participating families; 3) reasons why eligible children and their families do not participate in SFSP; 4) foodservice characteristics of SFSP sites; 5) the content of SFSP meals and snacks; and, 6) facilitators and barriers to preparing and serving SFSP meals. Data collection is expected in Summer 2018.

Assessment of Alternatives to the State Administrative Expense (SAE) Formula

Child Nutrition Programs have changed substantially since the SAE formula was last revised in the 1990s. This project will assess the effectiveness of the current formula used for State administrative expense fund allocations and develop and test a range of possible alternative algorithms to improve the formula.

Assessment of the Administrative Review Process

FNS has recently launched a new Administrative Review Process for the school meals programs. This study will assess the extent to which these reviews effectively identify risk areas and noncompliance with program requirements. Findings will help to ensure that State Agencies are able to provide meaningful technical assistance and require appropriate corrective action for noncompliance. The study will also examine the management of review and oversight resources. Where weaknesses in process or outcomes are found, the project will seek to identify more effective methods already in use in government and/or industry for application in school meals.

Evaluation of the Independent Review Process

The Healthy, Hunger-Free Kids Act (HHFKA) included several provisions to help increase effectiveness and integrity in Child Nutrition Programs. As part of an effort to decrease certification error during the eligibility determination process, Section 304 of the HHFKA requires certain local educational authorities to conduct a second, independent review of applications for free and reduced price meal benefits before eligibility determinations are made. FNS published a final rule establishing the requirements related to the provision, which became effective in SY 2014-2015. This evaluation will assess the process and effectiveness of this requirement, and help to determine if changes in the process are necessary.

Rural Child Poverty Nutrition Center

The purpose of this grant is to establish the Rural Child Poverty Nutrition Center (RCPNC) at the University of Kentucky. The objective of the Center is to reduce child food insecurity by improving program coordination among USDA-FNS Child Nutrition Programs and other nutrition assistance programs. RCPNC has formed strategic partnerships to ensure successful program implementation, relying on guidance in terms of content area expertise, technology, training, and outreach. The University of Kentucky will manage sub-grantees who have proposed creative strategies to increase participation in these programs among families with children in persistently poor rural counties in 15 states. Sub-grantees’ programs will be evaluated using qualitative and quantitative methodologies, and evaluation findings and implementation strategies will be widely disseminated. The final report of project activities will be submitted to FNS in June 2019.