Assessment of a HACCP Training Network

Please note that this study was published before the implementation of Healthy, Hunger-Free Kids Act of 2010, which went into effect during the 2012-13 school year, and its provision for Smart Snacks Nutrition Standards for Competitive Food in Schools, implemented during the 2014-15 school year. As such, certain research may not be relevant today.

The National Food Service Management Institute (NFSMI) has developed a national network of food safety trainers that provides training in Hazard Analysis and Critical Control Point (HACCP) systems. The HACCP system is a prevention-based food safety program currently being implemented in school foodservice operations across the United States. to assist school nutrition professionals in ensuring that children receive safe meals. Section 111 of the Child Nutrition and WIC Reauthorization Act of 2004 addresses food safety in school nutrition programs and required schools participating in the National School Lunch Program (NSLP) and the School Breakfast Program (SBP) to implement a food safety program based on HACCP principles as of July 1, 2005 (USDA Food and Nutrition Service, 2005a). Specifically, the Process Approach to HACCP was recommended by the U.S. Department of Agriculture (USDA) because it is an appropriate method for school foodservice (USDA Food and Nutrition Service, 2005b). The Process Approach to HACCP categorizes the flow of food preparation into three broad categories, or processes, that are determined by the number of times a menu item passes through the temperature danger zone. The USDA has information available to assist with HACCP implementation (USDA Food and Nutrition Service, 2005b). The NFSMI offers both a network of trainers knowledgeable in HACCP and instructor manuals covering all aspects of HACCP implementation and maintenance (NFSMI, 2005a).

More than 29 million children are served meals daily through the NSLP and SBP administered by the USDA’s Food and Nutrition Service. Although cases of foodborne illness occurs in relatively small numbers compared to the millions of school meals served daily, these are preventable (GAO, 2003). The HACCP system offers a prevention approach to food safety. In a position statement, the School Nutrition Association (SNA), formerly the American School Food Service Association (ASFSA) (2003), supported the development and implementation of a systematic approach to food safety including HACCP principles into school foodservice systems.

Henroid and Sneed (2004) concluded that training programs on implementing prerequisite HACCP programs were needed in school foodservice after they evaluated several factors, including employee food safety knowledge, attitudes, and food handling practices. Results indicated that proper practices were not being followed in many schools and inadequate hand- washing practices were observed in most foodservice operations. Hwang et al. (2001) reported that only 62% of Indiana school foodservice operations had a sanitation-training program for employees. Henroid and Sneed (2004) summarized that “initial results indicate that employees in school foodservice may have sufficient knowledge about food safety but need assistance in developing prerequisite programs in preparation for HACCP.”

Most school foodservice directors realize the benefits of HACCP in preventing food-borne illness. Sneed and Henroid (2003) questioned school foodservice directors who had or were in the process of implementing a HACCP program in their school districts, and responses confirmed the belief that HACCP not only could improve food safety but also be an insurance policy against liability. A survey by Giampaoli et al. (2002) revealed that foodservice directors had a positive attitude about food safety and the use of HACCP programs in their districts.

Eighty-two percent indicated that it was important for them to learn more about food safety issues. However, many were unsure of how to apply HACCP in their operations (Giampaoli et al., 2002). While it appears that school foodservice authorities believe that food safety is important, a full understanding of HACCP may be lacking by many school foodservice personnel.

There are many sources available for education and training on HACCP (Henroid, 2003). The NFSMI offered HACCP training sessions from 2000 to 2004 that included lectures, videos, and workshop activities to provide interactive learning experiences in HACCP and food safety, as applied to school foodservice. Four hundred persons who were recommended by state agencies were trained in two- to three-day workshops over this five-year period. At the conclusion of each session, participants were encouraged to provide training in their states modeled after the NFSMI training. This training was conducted prior to legislation mandating that school food authorities develop a school food safety program based on HACCP principles. This research study was conducted prior to the suggestion of the Process Approach to HACCP in school foodservice.

Currently, the NFSMI provides training in the Process Approach to HACCP for foodservice operations.

The purpose of this research is to investigate the effectiveness of the HACCP training provided to school foodservice directors, administrators, and managers by the NFSMI from 2000 to 2004. Information was collected from individuals who attended the NFSMI HACCP training and from those individuals who subsequently were trained by them. The HACCP training provided by the NFSMI and the HACCP training provided to foodservice employees by HACCP Trainers were evaluated based on a previous general assessment model used by the NFSMI (2005b) that was revised for this study. Findings from this study can be used by the NFSMI and others who provide HACCP training to assist in further development of HACCP training materials and determine how these training materials can best be presented to school foodservice directors and staff.

Methodology

For the purpose of this study, two self-administered questionnaires were developed: one for the participants of the NFSMI workshops (the Trainers) and one for the foodservice employees who received training from the NFSMI workshop participants. Questionnaires were developed by revising a general assessment model previously used by the NFSMI (2005b). Four hundred individuals who participated in the NFSMI HACCP training workshops over a five-year period, 2000-04, were mailed a survey. During this five-year period, the same two NFSMI trainers conducted all training sessions. As the individuals who attended the NFSMI training conducted further training sessions, they were encouraged to submit rosters of individuals they trained in their state or local district following the NFSMI model. Eleven hundred names were submitted, and these individuals who attended training sessions presented by the NFSMI-trained Trainers were mailed a survey.

The questionnaire mailed to the Trainers included questions with multiple responses to identify the most helpful and effective teaching and training strategies. Questions addressed the length of the training session, physical facilities, and general background information. Other items measured the perceived level of knowledge of HACCP principles before and after the NFSMI training. The perceived overall quality of the HACCP training was evaluated using a five-point rating scale of satisfaction.

To assess and evaluate HACCP training provided by the Trainers to foodservice employees, a questionnaire was developed that included identifying HACCP training strategies that were helpful and strategies that may have improved the training, items that measured the perceived level of knowledge of HACCP principles before and after HACCP training, and items for participants to indicate their overall evaluation of the quality of the HACCP training using a five- point rating scale. Demographic items also were included in the questionnaire.

The Statistical Package for Social Sciences for Windows software (version 12.0, SPSS, Inc., Chicago, IL) was used for analyzing data. Descriptive statistics, including means, standard deviations, percentages, and frequency distributions, were computed. Two-sample t-tests were used to compare assessments from the Trainers and the foodservice employees. Paired t-tests were used to examine pre- and post-differences of HACCP knowledge items. T-statistics with p<0.05 were considered significant. Significant t-statistic values would indicate that the reported knowledge was different between pre- and post-training for the same respondents. In comparing assessments between Trainers and foodservice employees, significant t-statistic values would indicate a difference in average responses between the Trainers and foodservice employees. The University of Mississippi’s Institutional Review Board approved questionnaires and all research procedures.

Results And Discussion

Four hundred questionnaires were mailed to individuals who attended the HACCP training at the NFSMI (Trainers), and 77 usable surveys were returned for a response rate of 19%, which was lower than desired by the researcher and considered a weakness of this study. Individuals could return their surveys using a toll-free fax number or mail. In a research study on school foodservice implementation of HACCP, a low response rate of 18% was reported from school foodservice directors, even after mailing two follow-up reminder postcards (NFSMI, 2005b).

The majority (n=44) of Trainers indicated that the training they received at the NFSMI was the only training they had received in HACCP. About half of the Trainers (n=39) were employed in school districts at the time of training. Others worked in state agencies that administer child nutrition programs (n=25) or at other government agencies (n=11). Participants were chosen by state agencies to attend the HACCP training and came to the NFSMI from 32 states representing all regions of the United States. Thirty participants had been in their current positions between five and ten years, while 16 Trainers had been in their current positions for more than 20 years at the time of the survey.

The Trainers were asked to identify which items would have improved the quality of the HACCP training provided by the NFSMI. Participants responded by choosing the following: 1) more demonstrations of teaching strategies (n=32); 2) having materials sent to participants before the session (n=22); 3) more group projects (n=14); 4) more group discussion (n=10); 5) more handout materials (n=9); 6) more individual activities (n=8); and 7) more lecture information (n=5).

These results correspond with results reported by Sneed and Henroid (2003) in which foodservice directors recommended that HACCP resource materials should focus on practical applications and include self-assessment forms, lesson plans, and activities. The length of the sessions was reported as reasonable and only five respondents indicated that they would have liked more time for the overall HACCP workshop while four indicated that they thought the HACCP workshop was too long. Some of the responses included write-in comments, such as the suggestion that more time should have been spent in identifying critical control points in a school foodservice establishment.

When asked the question, “How do you feel about your preparation to deliver the HACCP training,” 39 Trainers responded that they felt well prepared and confident about their teaching ability; 18 felt well prepared but were uneasy about teaching; 11 indicated that they did not mind teaching but were still unclear about the principles of HACCP; five indicated that they were not well prepared and were uneasy about teaching; and three indicated that they did not really want to teach HACCP. These results indicate that training should include more information about teaching strategies. Additionally, training in the process approach to HACCP that specifically applies HACCP to school foodservice may relieve these concerns in future training. This approach streamlines the HACCP process with the grouping of menu items into one of three categories, or processes (USDA Food and Nutrition Service, 2005b).

Overall, 59 of the Trainers reported that they had conducted one or more HACCP training sessions with employees. Twelve Trainers indicated that they had each conducted 12 workshops since their HACCP training at the NFSMI. The number of individuals attending these second-tier workshops ranged from five to 230 individuals (mean=28); the most common frequency was 20 attendees. A cumulative number of 332 HACCP workshops had been conducted by the Trainers who participated in the study.

The Trainers reported improvements in their perceived knowledge of HACCP principles (p<.001) when comparing pre-HACCP training and post-HACCP training at the NFSMI (Table 1). The largest improvements were reported in “overall knowledge of HACCP principles,” “corrective action,” “critical limits,” and “record keeping/documentation.” A relatively small improvement was reported in “employee hygiene,” suggesting that the Trainers may have felt they already had the basic knowledge associated with hygiene. These significant results in comparing pre-HACCP training and post-HACCP training may have been due to the majority (n=44) of Trainers indicating that the training they received at the NFSMI was the only training they had received in HACCP, and that they felt confident with the level of knowledge attainment. Also, the Trainers indicated a very high level of satisfaction with the NFSMI training.

The assessment of the HACCP training provided by the Trainers to foodservice employees involved mailing 1,100 questionnaires to those employees, and 163 surveys were returned for a response rate of 15%. As with the survey mailed to the Trainers, a toll-free fax number was available to return completed surveys, or respondents were invited to mail their surveys back to the researchers.

The individuals who received training from the Trainers were mostly female employees (97%). Fifty-one percent of the participants were 45 to 54 years old. Overall, 95% were 35 to 64 years old. When questioned about language, 98% indicated that English was their first language, 1% responded that Spanish was their first language, and 1% indicated other languages as their first

language. Many of the foodservice employees indicated that they had foodservice certifications such as ServSafe (37%), food handler (32%), and SNA/ASFSA certification (20%); however, 17% indicated that they did not have any certification at the time of HACCP training. Twenty-six percent reported that high school was their highest level of education, 33% had completed some college, 15% had an associate’s degree, 15% had a bachelor’s degree, 5% had completed some graduate education, and 6% had a Master’s degree. Many respondents reported working in several schools or centralized kitchens that served more than one school. Eleven percent reported working only at high schools, 18% were working exclusively at elementary schools, and 34% reported working with preschool-aged children and were employed at preschools, daycare centers, and Head Start.

The foodservice employees also reported improvements (p<0.001) in their perceived knowledge of HACCP principles when comparing pre-HACCP training and post-HACCP training (Table 2). The largest improvements were reported in “overall knowledge of HACCP principles,” “reducing liabilities,” “corrective action,” and “critical limits.” Relatively small improvements in knowledge gained were reported in “employee hygiene” and “equipment maintenance.” The foodservice employee group may have felt they already had basic knowledge in hygiene and equipment maintenance and scored lower their knowledge attainment of those two areas. The significant results in comparing pre- and post-HACCP training may be because the majority (76%) of the foodservice employees indicated no prior HACCP training. Overall, the magnitude of the changes in perceived knowledge for the foodservice employees was greater than that for the Trainers (Table 1), which, again, may have been due to 76% of the foodservice employees reporting that they had no prior training in HACCP and 44 Trainers reporting that the training they received at the NFSMI was the only training they received in HACCP.

The Trainers were asked, “How likely are you to recommend NFSMI HACCP training to others?” Using a five-point rating scale (very unlikely=1 to very likely=5), 43 reported “very likely.” Sixty-three Trainers indicated that the overall quality of the NFSMI HACCP training was very good to excellent. Other questions and items used for an overall assessment and evaluation by the Trainers and the foodservice employees are presented in Table 3. Results indicated that both groups of survey respondents evaluated their HACCP training positively. There was no difference between Trainers and foodservice employees’ perceived overall quality of the HACCP training and the overall satisfaction with their HACCP training. Both groups agreed that the overall quality of the HACCP training was very good. The two groups did not agree on four statements that had a difference (p<0.05) and are listed in Table 3. The HACCP training delivered by the Trainers to the foodservice employees was perceived as less satisfactory than the HACCP training delivered by the NFSMI. The foodservice employee group was less likely (p<0.01) to recommend their HACCP training compared to the Trainers (Table 3).

Conclusions And Applications

The results of this research can be useful in developing and delivering HACCP education and training programs as schools implement food safety programs based on the principles of HACCP using the Process Approach. Forty-four of the Trainers and 76% of the foodservice employees reported that the NFSMI and individuals trained by the NFSMI, respectively, were the only sources of HACCP training they had received. Both the Trainers and the foodservice employees reported significant improvements in their perceived knowledge of a variety of HACCP principles. The largest improvements for both groups were “overall knowledge of HACCP principles,” “corrective action,” and “critical limits.” These results indicate both a need for the information and that the NFSMI is successfully providing needed information. However, more time probably should be allowed for teaching the principles of HACCP since 11 Trainers indicated that they were still unclear about the principles of HACCP.

Applying the Process Approach to HACCP in school foodservice should simplify and clarify future training efforts. Additionally, while both groups reported a high level of satisfaction with their HACCP training, they indicated that more group discussions and group projects would have been helpful in their training. Structured group discussions and group projects may assist participants in identifying and addressing HACCP implementation in their foodservice operations.

Thirty-two Trainers indicated that more demonstrations of teaching strategies would have improved their HACCP training. This is reflected in the lower satisfaction rating reported by the foodservice employees as compared to the Trainers’ high level of satisfaction reported for their HACCP training received from the NFSMI. Perhaps future NFSMI training could include more sessions on teaching skills and effective instructional methods.

Acknowledgments

This research was funded by the NFSMI with federal funds from the USDA Food and Nutrition Service. The authors thank the staff at the NFSMI for their technical assistance and the respondents for their participation.