Volume 39, Issue 1, Spring 2015, Spring 2015
FNS Research Corner
By John Endahl, PhD; Melissa Abelev, PhD
The FNS Research Corner provides a continuing series of summaries of recently completed and current research conducted by the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) in the area of child nutrition. For further information, contact the Office of Policy Support (OPS), formerly the Office of Research and Analysis (ORA) at (703) 305-2117. Links to published studies and reports as well as descriptions of ongoing studies conducted by OPS are available from the FNS Internet web site at http://www.fns.usda.gov/research-and-analysis.
Recently Completed Research
NSLP Direct Certification Improvement Study
The purpose of this study was to expand the understanding of direct certification best practices and the relationship between State practices and direct certification performance. This study collected data in School Year (SY) 2012/13 through a national survey and case studies. The national survey of direct certification practices was administered to all 50 States, the District of Columbia, Guam, and LEAs in local matching States. The survey design tailored questions based on a State’s data-matching method (central- or local-level) and the level of respondent (State or LEA). In-depth case studies were conducted in seven States and selected LEAs. Program and technical staff involved in direct certification at the State and LEA levels were interviewed via site visits. This study provides a descriptive analysis of States’ Direct Certification practices and includes a Best Practices report that States can use to improve their Direct Certification efforts.
Key findings include:
- Central matching at the State level has become the dominant means of implementing direct certification. In SY 2012/13, 38 States used central matching systems and 14 States used local matching systems.
- States and districts face common challenges in conducting direct certification. Both States and local agencies noted that incomplete or delayed enrollment data were barriers to direct Half of local matching States reported not having staff to perform State-level computer matching.
- Nearly all States had plans to improve direct certification systems in the coming States are planning a wide range of large and small changes to their systems, including adding more data elements, increasing the frequency of matching, and adding processes to resolve unmatched records.
- States using probabilistic matching software had higher average direct certification performance rates compared to States not doing Probabilistic matching software calculates the likelihood of a match this is called probabilistic matching, points are awarded for near matches such as spelling variations. This matching software is most useful in matching students with long, less common names. States that used this available software had an average direct certification performance rate of 89.6% compared to 86.8% for States not using the software.
- Nearly 90 percent of States used other program data sources in addition to SNAP to directly certify students. The top three data sources in addition to SNAP data were TANF, Foster Care, and FDPIR data.
- Many States only match direct certification once per month if they matched more often they could improve performance rates. Central matching States that performed direct certification matching more frequently than monthly had a higher average performance rate (92%) than States matching monthly (88%) and less frequently than monthly (81%).
- Linking direct certification systems to point of service systems and adoption of web- based direct certification systems are areas of opportunity to increase performance rates. Based on case study results these two strategies hold promise of improving administrative efficiency and utilizing resources more effectively as States strive to improve direct certification systems and meet required direct certification performance
Evaluation of Direct Certification with Medicaid (DC-M)
The Healthy, Hunger-Free Kids Act of 2010 (HHFKA; P.L. 111-296) required the USDA Food and Nutrition Service to conduct a demonstration that adds Medicaid to the list of programs used to directly certify students for free school meals. Although students receiving Medicaid are not categorically eligible for free meals, the DC-M demonstration authorizes selected States and districts to use income information from Medicaid files to directly certify those students found to be eligible for free meals.
The purpose of the Access Evaluation is to assess the potential impacts of DC-M on students’ access to free school meals by conducting retrospective simulations of DC-M in SY 2011/12, the year before the demonstration began. The Year 1 Report presents the preliminary impacts of DC- M on participation and cost for the first year of the demonstrations, SY 2012/13. A report of findings from Year 2 of the DC-M demonstration is forthcoming and will provide a comprehensive picture of implementation.
Key findings of the reports include:
- DC-M could increase the direct certification rate by almost 12 percentage points in Access Evaluation districts.
- For the Access Evaluation, DC-M could increase the number of districts eligible to participate in the Community Eligibility Provision established by HHFKA.
- New York City was the only place to fully implement DC-M at the beginning of SY 2012/13 and to demonstrate significant impacts on certification for school meal benefits. DC-M increased the percentage of students directly certified to receive free meals by 7 percentage points in New York City.
Direct Certification in the National School Lunch Program — State Progress in Implementation, School Year 2012/13
This report responds to a legislative requirement of Public Law 110-246 to assess the effectiveness of State and local efforts to conduct direct certification of children for free school meals. Under direct certification, children are determined eligible for free meals without the need for household applications by using data from other means-tested programs. The 2004 Child Nutrition and WIC Reauthorization Act required local educational agencies (LEAs) to establish a system of direct certification of children from households that receive Supplemental Nutrition Assistance Program (SNAP) benefits. For SY 2012/13, States that fail to achieve a direct certification rate of at least 90 percent are required to develop and implement continuous improvement plans (CIPs). The performance target increases to 95 percent for SY 2013/14 and beyond. This report presents information on the outcomes of direct certification for SY 2012/13.
In SY 2012/13, 91 percent of LEAs that participate in the NSLP directly certified some SNAP participants. These LEAs enroll 99 percent of all students in schools that participate in the NSLP. This is an increase from SY 2004/05, when 56 percent of LEAs, enrolling 77 percent of all students in NSLP schools, directly certified some SNAP-participant students. The number of school-age SNAP participants directly certified for free school meals was 12.3 million for SY 2012 2013, an increase of 6 percent from SY 2011/12. Analysis in this report estimates that 89 percent of children in SNAP households were directly certified for free school meals, which is 3 percentage points higher than last year’s rate of 86 percent. Twenty-four States achieved direct certification rates at or above the performance target of 90 percent established in the Healthy, Hunger-Free Kids Act (HHFKA) of 2010, and 16 States achieved direct certification rates of at least 95 percent. Only one State had a direct certification rate lower than 60 percent.
Motivated in part by the performance benchmarks set forth in HHFKA, interviewed States have recently increased match frequency, enhanced matching algorithms to include probabilistic matching, and begun performing matches with additional program data sources such as foster care data.
Regional Office Review of Applications (RORA) for School Meals 2013
This is the ninth in a series of annual reports that examines administrative error incurred during the local educational agencies’ (LEAs) approval process of applications for free and reduced- price school meals. It does not examine the accuracy of household reporting of information on the applications or errors made in counting and claiming. Key results include:
- For School Year 2012/13, LEA correctly certified 96.4 percent of students submitting applications for meal benefits, comparable to certification error rates found in previous school years (2004-2012).
- Of those students incorrectly certified, slightly less than three-quarters (72 percent) were certified for more benefits than were justified based on the documentation available. Income-based applications are more prone to administrative errors than categorically eligible applications, with many of these errors associated with the determination of a household’s gross income.
FNS has continued to be proactive in efforts to improve program integrity without compromising access to low-income families. Technical assistance and training materials have been provided to State and local partners to reduce administrative errors and improve program integrity.
Evaluation of the Summer Electronic Benefit Transfer for Children Demonstration – Summer 2013 Results
As part of its efforts to end child hunger, FNS is studying alternative approaches to providing food assistance to children in the summer months. The 2010 Agriculture Appropriations Act (P.L. 111-80) authorized and provided funding for USDA to implement and rigorously evaluate the Summer Food for Children Demonstrations, one component of which is the Summer Electronic Benefits Transfer for Children (SEBTC). The SEBTC benefit is provided to households with children from pre-kindergarten through 12th grade that are certified for Free or Reduced Price school meals in the demonstration school food authorities (SFAs). The amount of the initial benefit—an approximately $60 value per month per child in the household—is comparable to the cost of free lunches plus breakfasts under the NSLP and SBP. Benefits— provided monthly on an EBT card —are administered by grantees in the summer for the period when schools are not in session. The benefit is administered using either the State’s existing EBT system for WIC or the EBT system for SNAP.
FNS started SEBTC with a proof-of-concept test in 2011, serving about 12,500 children in five sites. The demonstration was then expanded to a total of 14 sites in 10 States in 2012, reaching over 66,000 children and providing an average of approximately $150 per child and $250 per household over the course of the summer. The prior evaluations of SEBTC in 2011 and 2012 found that a benefit of $60 per month per child in eligible households on a SNAP or WIC EBT card reduced very low food security among children (VLFS-C) by 20% and 33%, respectively, compared to a $0 benefit.
In 2013, households in the 6 evaluation sites were randomly assigned into one of two groups that received either a $60 or $30 benefit per child per month. The demonstration reached about 100,000 in 2013, with 51,000 receiving same benefits as 2012 and 49,000 “new” children in the evaluation. Results from the evaluation indicate that:
- The $30 benefit was as effective in reducing the most severe category of food insecurity among children during the summer as the $60 benefit. The prevalence of VLFS-C in summer 2013 was similar for both the $30 and $60 benefit However, the $60 benefit reduced less severe food insecurity among children by about 10% compared to the $30 benefit. Results were similar across SNAP and WIC sites.
- Households in the $30 and $60 groups redeemed their benefits at similar rates. Between 91-93% of households who received the $30 and the $60 benefits used it at least once during summer. Of those households that used their benefits at least once, the redemption rates (percentage of benefits redeemed) were approximately 71 and 74% for the $30 and $60 groups, respectively. The average benefit redeemed per eligible child was $134 for the $60 group and $64 for the $30 group during summer.
- SEBTC can reach a significant proportion of children eligible for free and reduced- price school Active consent rates for the demonstration ranged from 23% to 42% of eligible children, which exceeds the 16 percent of eligible children who participate in traditional summer feeding programs.
- SEBTC children in $30 group ate more healthfully. They ate about 7% more fruits and vegetables, 10% more whole grains, and 4% more dairy than $60 group.
Evaluation of the Enhanced Summer Food Service Program Demonstration – Summer 2012 Results
The Summer Food for Children Demonstrations also included the Enhanced Summer Food Service Program (eSFSP) demonstrations, which test the impact of a number of enhancements to the existing Summer Food Service Program (SFSP).
Two of these projects in 2012 included:
- The Meal Delivery demonstration in Delaware, Massachusetts, and New York offered meal delivery to homes or drop-off sites near homes of eligible children in rural areas;
- The Backpack demonstration in Arizona, Kansas, and Ohio provided weekend and holiday bags or packs with meals to SFSP children for consumption when SFSP sites were not
Results from two evaluations of the 2012 demonstrations indicate:
- Meal Delivery and Backpack demonstrations differed in their meal preparation, delivery methods, outreach and recruitment, and staffing.
- Both demonstrations were limited in their reach, serving about 260,000 meals to almost 6,000 children in six States.
- Stakeholders had positive views of the Both participating families and site operators felt the demonstrations were an important resource to address summer hunger.
- Impacts on meals served and participation are mixed. Backpack sites experienced both decreases and increases in participation while meal deliveries likely increased the number of children served.
- Demonstration sites, like regular SFSP sites, experienced a lot of change year to year, which makes it difficult to ascribe different outcomes to the demonstration itself.
Research In Progress
The following section provides a brief description of some on-going FNS research and the current status of these studies:
Special Nutrition Program Operations Study
The Special Nutrition Program Operations Study is a multi-year study designed to provide FNS with key information about the operational policies and practices of School Food Authorities (SFAs) and State Child Nutrition Agencies that can be used to address policy needs, develop informed regulations and guidance, and provide needed technical assistance. Survey data is collected from a nationally representative sample of over 1,400 SFA directors in public school districts operating the National School Lunch Program and a census of State Child Nutrition Directors.
Results of the first year of data collection, SY 2011/12, were reported in last year’s journal and are available on the FNS Internet web site. Results of the second year of the study, based on data collected during SY 2012/13, are forthcoming. Topics addressed include implementation of the new school meal standards, competitive food standards, professional staffing standards, school lunch and breakfast meal pricing and accounting, and standards for school wellness policies.
Information from this study will provide a baseline for observing the improvements resulting from the implementation of provisions found in the Healthy, Hunger-Free Kids Act of 2010.
School Nutrition and Meal Cost Study (SNMCS)
The School Nutrition and Meal Cost Study (SNMCS) will explore both the nutrition and cost domains of school meals in an integrated study design. It will collect a broad range of data from a nationally representative sample of about 500 school food authorities, 1,200 schools, 2,400 students and parents, and a large sample of school meals (over 5,000 lunches and 3,000 breakfasts) during School Year 2014/15. The data collection includes the administration of several different types of instruments and modes, including self-administered web-based SFA director and school principal surveys, a food service manager survey, an electronic menu survey, competitive foods checklists, cafeteria environment observation, plate waste observation, Automated Multiple Pass Method 24-hour dietary recalls, measurement of student’s height and weight, student/parent surveys, meal cost interviews, and collection of administrative cost data.
These data will provide needed information about how Federally-sponsored school meal programs are operating after implementation of the new nutrition standards and other changes in regulations. Comparisons of results of this study with pervious School Nutrition and Dietary Assessment (SNDA) and School Lunch and Breakfast Cost (SLBC) studies will provide information that can be used to assess the effects of the new nutrition standards on foodservice operations, the nutrient content of school meals as offered and served, meal costs and revenues, and student participation and dietary intake. The integrative structure of the SNMCS will support analysis of the relationships among substantive areas such as nutritional quality of meals, meal costs, student participation, and plate waste. The study will produce five separate reports in 2017 summarizing study findings (SFA/school characteristics and food service operations; nutrition quality of meals; meal costs; student participation, dietary intake, and other outcomes; and plate waste) and a stand-alone summary of findings.
Study on Nutrition and Wellness Quality of Child Care Settings (SNAQCS)
The Healthy Hunger-Free Kids Act of 2010 (Section 223) directs USDA to conduct a study of child care providers to assess the nutritional quality of foods provided in child care settings, opportunities for physical activity, and facilitators and barriers to providing healthy foods and physical activity and participation in CACFP. This study will also examine the costs of preparing meals for CACFP providers. Interim reports will be published in 2017. The final report is expected in 2019.
The Access, Participation, Eligibility and Certification (APEC) series
The Improper Payments Elimination and Recovery Act (IPERA) requires Federal agencies to report annually on the extent of erroneous payments in their programs, including overpayments (payments that were made improperly or exceeded the proper level) and underpayments (payments that were denied improperly or were lower than the proper level). This series is conducted at approximately five-year intervals to provide a baseline of calculated error in the School Meals programs. Data were collected in SY 2012/13 to examine certification and non- certification error in the National School Lunch Program and School Breakfast Program. A report on the findings will be published in early 2015.
Child and Adult Care Food Program (CACFP) Characteristics Study
FNS is currently conducting a national study on the Characteristics of CACFP Sponsors and Providers. The study, which is focused on the child care component of the program, will collect data in 2016 and publish a descriptive report in late 2016. The study will provide policy-makers, advocates, and the general public with up-to-date information since the last major study in 1997 about: sponsor types, training and technical assistance, monitoring, program operations and management, provider types, provider operating hours, types of meals and snacks served, participant characteristics, and funding sources.
CACFP Tiering Series
The Improper Payments Information Act of 2002 (IPIA) (Public Law 107-300) requires all Federal agencies to calculate the amount of erroneous payments in Federal programs and to periodically conduct detailed assessments of vulnerable program components. This assessment of the family daycare homes (FDCHs) component of CACFP provides a national estimate of the share of the roughly 125,000 participating FDCHs that are approved for an incorrect level of per meal reimbursement, or reimbursement “tier” for their circumstances. FDCHs can earn Tier I (higher) or Tier II (lower) reimbursements depending on the location and circumstances of the child care provider or the participating children. Tiering errors result in improper payments because misclassified family daycare homes do not receive the appropriate level of reimbursement for the meals and snacks provided to the children. The assessment also estimates the dollar amount of improper payments attributable to these tiering errors.
Characteristics of Summer Food Service Participants and Programs
The Summer Food Service Program (SFSP) serves only a small percentage of the children who receive free and reduced price meals through the National School Lunch Program during the school year. This national study aims to provide detailed information on SFSP operations and administration at the state agency, sponsor, and site levels, as well as characteristics and experiences of the providers, participants, and eligible nonparticipants. The findings and analyses of this study are intended to replicate and expand upon the last national study of the SFSP published in 2003 by the Economic Research Service/Mathematica Policy Research, Inc. (ERS/MPR). Since the 2003 study, significant policy changes have been implemented that could affect participation in the program by children, sponsors, and sites, so this study will also provide a picture of changes in participation since the implementation of recent policy developments affecting the SFSP and SSO.
Putting Behavioral Economics to Work in School Cafeterias
These funds support a multi-year, integrated research program, launched in Fiscal Year 2010 in collaboration with the Economic Research Service and the National Institute for Food and Agriculture, to develop, test, and promote applications of behavioral economic theory in the school nutrition environment. The goal is to identify, develop, and document evidence-based strategies, tools, and techniques that schools can use to shape their environments to support and encourage healthful food choices and behaviors. A key partner in this effort is the Cornell University Behavioral Economics in Nutrition (BEN) Center and their efforts to increase the use of “Smarter Lunchroom” techniques in the Nation’s schools.
Rural Child Poverty Nutrition Center
This grant will fund the evaluation and implementation of efforts to reduce child hunger by increasing participation in Child Nutrition programs. Local communities can apply for grants to facilitate greater coordination of resources in communities with persistent poverty. A main grant partner will manage a subgrant process to communities, as well as conduct an overall evaluation of the impact on participation of the grantees’ efforts.
Assessment of Changes in Local Wellness Policies and Competitive Foods Practices
This grant provides funding to the University of Illinois-Chicago and its partners to monitor and evaluate local school wellness policies (LWPs) of local educational agencies (LEAs) participating in the National School Lunch Program. The study will assess the impact of state laws and district policies on the school environment, school practices, and student outcomes.
Qualitative methods will examine HHFKA policy implementation and facilitators/barriers to compliance. Results will be disseminated in a variety of reports, briefs, presentations, and manuscripts in 2017 and 2018.
Both John Endahl and Melissa Abelev are associated with the Office of Policy Support, Food and Nutrition Service, U.S. Department of Agriculture. Endahl is Senior Programs Analyst, and Abelev serves as Special Nutrition Evaluation Branch Chief.