Impact of Educational Interventions on Hazard Analysis Critical Control Point (HACCP) Program Implementation in Iowa Schools

Please note that this study was published before the implementation of Healthy, Hunger-Free Kids Act of 2010, which went into effect during the 2012-13 school year, and its provision for Smart Snacks Nutrition Standards for Competitive Food in Schools, implemented during the 2014-15 school year. As such, certain research may not be relevant today.

The importance of food safety in schools is well established, and food safety issues have been documented in school foodservice (Brown et al., 1982; Connors et al., 1999; FDA, 2000; FDA, 2004; Giampaoli et al., 2002; Kim & Shanklin, 1999; Youn & Sneed, 2002). The General Accounting Office (GAO, 2000) reported 20 outbreaks of foodborne illness in schools in 1997, eight of which were associated with food prepared on school premises. A more recent GAO report (2003) indicated that the number of foodborne illness outbreaks is increasing in schools by about 10% per year since the early 1990s. One outbreak of Escherichia coli 0157:H7 in Finley, WA, has received much attention because of the health impact on the three children involved and the resulting litigation against the school district (Cary, 2001; Cary, 2003).

Congress responded to national concerns for food safety by including new food safety requirements in the Child Nutrition and WIC Reauthorization Act of 2004. That legislation requires that all schools implement a food safety plan based on HACCP principles and post- inspection scores, and have two health inspections annually (Child Nutrition and WIC Reauthorization Act of 2004). USDA (2005) developed guidance for school districts on how to develop those food safety plans. Research indicates that a limited number of school districts had implemented HACCP programs prior to the legislation (Giampaoli et al., 2002; Henroid & Sneed, 2004; Hwang et al., 2001; Youn & Sneed, 2003). For example, in the Hwang et al. study (2001), 13% of school corporations in Indiana had implemented HACCP. Giampaoli et al. (2002) found that nearly 30% of directors reported to have implemented HACCP. While there is limited research related to food safety in schools, research that has been done indicates food handling problems that need to be addressed. Time and temperature abuses and inadequate hand washing are two practices identified as concerns in schools (Brown et al., 1982; Giampaoli et al., 2002; Gilmore et al., 1998; Kim & Shanklin, 1999; Richards et al., 1993). Research in central food production systems also indicates that time and temperature abuses occur (Blakelee & Penner, 1999; Brown et al., 1982; Kim & Shanklin, 1999). Connors et al. (1999) found that some school foodservice managers did not know the recommended holding temperatures for milk, thus, knowledge of food safety is another problem.

Research is available on the implementation of HACCP prerequisite programs in schools (Youn & Sneed, 2002; Youn & Sneed, 2003), attitudes and challenges to implementing HACCP (Giampaoli et al., 2002; McSwane & Linton, 2000), and best practices and implementation strategies for successful HACCP programs (Sneed & Henroid, 2003). While food safety knowledge may be limited, there is no strong evidence that knowledge and practices are highly correlated (FDA, 2004; Speer & Kane, 1990). Little research has documented the readiness of schools for implementing HACCP or examined interventions that might support implementation of food safety programs. The purposes of this study were to determine the impact of educational interventions on employee food safety knowledge and attitudes, food handling practices, and implementation of HACCP programs in school foodservice, and to examine foodservice directors’ perceived usefulness of various interventions that might support food safety.

Methodology

Subjects

A convenience sample of 40 schools from 40 different school districts was selected from the state of Iowa for the study. Directors were informed of the scope of the three-year study, benefits of participation, and confidentiality of data. Participating school foodservice directors were asked to select one school with a production kitchen within the district to be included in the study. All employees in the schools participated. The Iowa State University Institutional Review Board approved the study.

Study Design and Interventions

A pre- and post-test quasi-experimental research design was used in the three-year study. Data were collected on site at participating schools (40 at pre-test, 38 at post-test) during Years One and Three. A formal report was prepared and mailed to the school foodservice director after each site visit. The report included a summary of observations, identified strengths, and recommendations for change. Between Years One and Three, extensive training interventions were provided, along with provision of mentoring and technical assistance from the project team and Cooperative Extension field staff. A summary of the project timeline is shown in Figure 1.

Formal training was provided on food safety (ServSafe®) and HACCP. Each facility was offered an opportunity to send five employees to local ServSafe® training sessions free of charge. One hundred and eight-five school employees attended. It should be noted that foodservice managers were not required to have a food safety certificate in the state of Iowa.

In addition, two HACCP training sessions were presented in six regions across the state of Iowa, one each during Years One and Two. Regional training sessions were held to minimize travel for participants and allow for regional Extension Nutrition & Health Field Specialists to attend. Two representatives from each school were invited to attend, with some school foodservice directors bringing the entire management team for the school. The first session focused on the need for HACCP implementation in schools, assessment of readiness for HACCP implementation, development of written standard operating procedures, understanding of the first three HACCP principles, and implementation plans for the first year. The second session focused on sharing progress made during the first year, understanding the last four HACCP principles, and developing individualized implementation plans for the last year of the project. There were 86 participants who attended regional HACCP training sessions the first year and 84 who attended the second year.

Based on an observed absence of written standard operating procedures (SOP) and documentation related to food safety, written SOP and monitoring forms were developed. A notebook of resources was distributed to participants from each school. The notebook included suggested content for program descriptions, checklists for assessing prerequisite programs and SOPs, written SOPs, monitoring forms to accompany SOPs as needed, food product flow charts, and a school case study showing application of HACCP principles. These resources also were provided on a CD. In addition, all resources were posted on the Iowa State University food safety website. [NOTE: LINK TO WEBSITE also www.schoolhaccp.org]

Based on assessed needs, short training sessions were developed around monthly themes: FAT TOM (the November lesson focusing of factors that contribute to microbial growth); All Wrapped Up! (the December lesson focusing on proper storing and labeling of foods); Keep it Cold! (the January lesson focusing on the need to keep cold food cold); and Write it Down! (the April lesson focusing on the need to document times and temperatures of various food, equipment, and processes). An eight-lesson “HACCP Journey” series was developed for school foodservice managers/directors to use to introduce employees to the seven HACCP principles. An employee food safety orientation brochure, SOPs, and orientation checklist also were developed. Contests were held to involve employees in the project. In one contest, participants were asked to submit names for the project. The winning name, Safe Food: Safe Kids, formed the basis for a logo used for project educational materials. In another contest, participants were asked to share training ideas that they had found successful. Winners were awarded infrared thermometers.

Survey Instruments

Baseline (pre-test) and follow-up (post-test) assessments conducted at each school consisted of three components: 1) an on-site visit to assess employees’ food handling practices and the HACCP program components used; 2) an interview with the school foodservice manager; and 3) a test of employee knowledge and attitudes toward food safety. A food safety assessment form was developed (Henroid & Sneed, 2004) based on the Iowa inspection form and previous research conducted by Giampaoli et al. (2002). All three research team members conducting assessments were trained on using the food safety assessment form.

Prior to pre-test visits, the research team visited two schools to pilot test the assessment form and process. Each researcher conducted independent assessments of the foodservice production facility and employee food handling practices. Results were compared and discussed after each assessment to establish inter-rater reliability. The assessment form was modified after each pilot test.

The written employee questionnaire was developed to determine knowledge of and attitudes about food safety. Twenty knowledge questions and six attitude questions were included. The questionnaire was reviewed by content specialists and pilot tested, and a Kuder- Richardson 20 reliability coefficient was determined for knowledge items (0.65) (Henroid & Sneed, 2004). A school HACCP project evaluation form was developed to obtain feedback from school foodservice directors and managers participating in the study on their use of project training materials (dichotomous Yes/No scale) and their perceived usefulness of the training sessions and trainers (usefulness scale: did not use, very useful, some usefulness, and not useful). Open-ended questions were asked to determine the impact of the project on health inspection scores, how project participation was useful to school district, what changes could have been made to make the project more useful; and what support/resources were needed to continue with HACCP implementation.

Data Collection

School foodservice assessments were conducted during Years One and Three. Each three-hour assessment included observations of food preparation and service. Informal interviews with school foodservice employees were conducted to determine food safety knowledge and food handling practices. In addition to observations, objective measurements such as food temperatures, temperatures of refrigerators and freezers, dish machine temperatures, and chemical sanitizer concentrations were taken. Food temperatures were checked with calibrated, tip-sensitive digital thermometers. Thermal strips were used to check high-temperature dish machines and test strips were used to check sanitizer concentrations in low-temperature dish machines, pot-and-pan sinks, and sanitizer buckets. The results were compared to standards in the State of Iowa Food Code, based on the 1997 FDA Food Code, to determine compliance.

At the time of the assessment, school foodservice managers were given employee tests and an addressed, postage-paid envelope. Each test was numbered with a three-digit code to identify the specific school. School foodservice managers distributed the employee tests and, upon completion, mailed tests to the research team. During Year Three, the school HACCP project evaluation form was mailed to foodservice directors in all 40 school districts that began the project, along with a postage-paid return envelope. A reminder letter and an additional questionnaire were mailed to non-respondents two weeks after the original mailing to encourage response.

Data Analysis

All statistical analyses were conducted using SPSS for Windows (version 11.0, 2001, Chicago, Ill). Statistical significance at p≤0.05 was used for all tests. Means, standard deviations, and frequencies were computed for all variables. A food safety practice score (FSPS) was calculated based on observations to determine an overall assessment of the operation for purposes of comparison. The score was determined by dividing the number of “yes” answers on the assessment by the total number of “yes” and “no” responses, and multiplying the score by 100. A “yes” response indicated that the practice was present or that practices were observed being done properly for a majority of the observation time. Areas that were not observed or not applicable were not included in the calculation. Paired samples t-tests were used to compare means for food safety knowledge, attitudes, and food handling practices between pre-test and post-test. One-way ANOVA was used to assess differences in school foodservice employees’ food safety knowledge based on years of experience, job title, and food safety certification. A Tukey’s HSD test was used to determine differences in mean scores when the models were significant. A Cronbach’s alpha reliability coefficient was calculated to determine the reliability of attitudinal items (Cronbach, 1951).

Results And Discussion

Schools districts participating in the study had approximately four schools and averaged 1,813 students (Henroid & Sneed, 2004). Twenty-five of the 40 schools visited were regional kitchens that delivered food to an average of 2.7 satellite kitchens. All schools used conventional foodservice production systems. The majority of managers in these school districts was female, had received a high school diploma or equivalent, and had 15 or more years of school foodservice management experience. Most of the people in charge of school foodservice had the title of foodservice director or operations manager and were between the ages of 40 and 60 years old. Twenty-nine of 40 managers had food handler certification, such as ServSafe®. Position title, years of school foodservice experience, and food safety certification of foodservice employees participating in the study are summarized in Table 1. The most significant difference between pre-test and post-test is the increase in the number of employees who were certified at post-test (189 or 64.5% at post-test compared to 110 or 38.5% at pre-test), which reflected the fact that this project paid for five employees from each school to attend a ServSafe® training session.

Food Safety Knowledge and Attitudes

The food safety knowledge of employees was high both at pre-test and post-test. There were 309 pre-tests and 301 post-tests returned. Post-test scores were higher than pre-test scores (16.7 + 2.7; 15.9 + 2.4; p = 0.0001). ANOVA examined the relationship between demographic variables (years of experience, job title, and certification) on knowledge scores. The model was significant for both pre-test (F=15.9, p = 0.0001) and post-test (F = 10.7, p = 0.0001). For both models, job title and certification were significant. For both models, managers and those who had a food safety certification had higher knowledge scores than individuals who were not managers and were not food safety certified. Overall employee attitudes toward food safety were favorable, both at pre-test and post-test. Mean attitude scores for the six items ranged from 4.2 to 4.8, with a mean of 27.5 (out of 30) at pre-test and 28 at post-test. While the difference was statistically significant, it was not of practical significance. A Cronbach’s alpha coefficient was calculated for the attitude items and found to be reliable (α = 0.71 at pre-test and 0.84 at post-test).

Assessment of Food Handling Practices

Observed food handling practices at Years One and Three are summarized in Table 2. The FSPS was higher for Year Three (87.0+9.7 compared to 67.5+14.4; p = 0.0001). Temperature documentation is the area that improved most, including documentation of food temperatures at time of service and temperatures of equipment (refrigerators, freezers, milk coolers, and dish machines). Improvements in employees’ knowledge of food temperatures and thermometer calibration also were noted. There were few instances in which cooling of foods was observed (ten at pre-test and seven at post-test); thus, inadequate cooling (one of the leading causes of foodborne illnesses) was observed in only a small number of school kitchens.

Participants’ Evaluation of Project

Project evaluations were returned by 34 of the 40 school districts. School foodservice managers reported changes as a result of participating in this project. Written SOPs were developed based on project samples in 23 school districts and 17 modified recipes to include critical control points. A large number (from 25 to 31, depending on what was recorded) of schools began documenting as a result of the project (Table 3). Only one school had a HACCP team in place prior to the project, while 12 did by Year Three. It is interesting to note that about half of the respondents (14 of 30) indicated that participation in the project impacted their health inspection scores. Reasons provided by managers included “inspectors are confident of our kitchen”; “food handling practices are improved”; “documentation is done, including use of dish machine strips”; and “cooks have more confidence in dealing with inspectors.”

Table 4 summarizes the usefulness of various resources provided through the project. School foodservice managers (24 of 30) rated the site visit report– which included observations, recommendations, and microbiological test results and written SOPs and documentation forms–as very useful. Training also was rated as very useful by most respondents. Project staff was perceived to be very useful mentors by 27 of 34 respondents.

 

There were many comments to the open-ended question of the value of participation in the project to the school district. Several foodservice managers wrote that the project had made the staff more aware of food safety and that the project reinforced good food safety practices that supervisors had tried to implement. Training was cited as a major benefit of the project. For a few districts, the project provided the impetus to make changes in facilities and purchase new equipment. Consistent documentation was another benefit mentioned. Many respondents reported that they had improved their food safety practices as a result of participating in the project.

When asked about suggested improvements to make the project more useful, many had no comments and several indicated that they were very pleased. One comment said, “You did a good job in presenting the project. It is a time factor that makes it difficult to implement more of the project into the school system.” Time and scheduling were two barriers that were noted by several respondents, which is consistent with other research (Blakeslee & Penner, 1999; Giampaoli et al., 2002; Hwang et al., 2001; Youn & Sneed, 2002).

Participants in this study were limited to districts in Iowa and most of these were very small, which is typical of the majority of districts nationwide. Although “generalizability” may be limited, results provide areas of focus for the development of HACCP programs and related prerequisite programs in school foodservice operations. Pre-test results indicate that employees in school foodservice had sufficient knowledge in food safety, but needed assistance with developing prerequisite programs in preparation for HACCP. Although food safety knowledge scores were high, food handling practices were not always consistent with accepted standards. Many food handling practices did improve after the educational intervention, thus, food safety education should be a priority for both managers and employees, because operations with certified individuals used more appropriate food safety practices. School foodservice managers need to facilitate employee involvement and empower them to make HACCP procedures an integral part of school foodservice operations. Informal observations of researchers showed that when a team of individuals from a school attended training, more progress was made in implementation of food safety and HACCP practices.

Conclusions And Applications

 Based on this research, there are several conclusions and applications for school foodservice directors.

• Educational interventions appear to be effective in improving food handling practices in schools. For example, at pre-test, few schools calibrated thermometers or documented temperatures. When the importance and techniques were explained to employees, the incidence of these practices increased. Thus, education and training programs related to various aspects of the food safety requirements and implementation procedures will support food safety program implementation.
• Food safety certification is related to food safety knowledge Certification also may give employees more confidence as they implement food safety programs.
• ServSafe® training was rated by managers as being very Thus, resources should be set aside to provide certification training to as many staff members as possible.
• About half of the school foodservice managers utilized food safety lessons that were provided to them in this project. Training resources will be used by some districts and should be provided. However, consideration should be given to the time required to do training, the scheduling of training, the expertise of trainers, and other factors that might impact the use of these materials.
• External observations and feedback about those observations was rated as very useful. School foodservice directors may collaborate with directors in other districts and review each other’s schools as a method for obtaining external feedback.
• Ready-to-use resources that can be customized for a school or school district, such as written SOP and documentation forms, are useful to school foodservice This may be especially useful for small districts with limited resources.
• External mentors, especially those who may be viewed as having specific expertise, are rated as very useful to school State agencies may consider developing the food safety expertise of a staff member who can serve as a mentor for districts in the state and be available to answer questions. State Extension staff also may be an excellent resource for training for state agencies and school districts.
• Resources should be provided on CD or in written form. It is difficult for foodservice directors and managers in many states to download resources from a website, partly due to a lack of availability of computers and the Internet in the foodservice areas of

Acknowledgments

This research project was funded by the U.S, Department of Agriculture (USDA) Cooperative States Research, Education, and Extension Service, Project No. 2001-51110-11371. The contents are solely the responsibility of the authors and do not necessarily represent the views of USDA.