Evaluation Tool Development for Food Literacy Programs

Andrea Begley, Ellen Paynter, Satvinder S Dhaliwal, Andrea Begley, Ellen Paynter, Satvinder S Dhaliwal

Abstract

Food literacy is described as the behaviours involved in planning, purchasing, preparing, and eating food and is critical for achieving healthy dietary intakes. Food literacy programs require valid and reliable evaluation measures. The aim of this paper is to describe the development and validation of a self-administered questionnaire to measure food literacy behaviours targeted by the Food Sensations® for Adults program in Western Australia. Validity and reliability tests were applied to questionnaire item development commencing with (a) a deductive approach using Australian empirical evidence on food literacy as a construct along with its components and (b) adapting an extensively-tested food behaviour checklist to generate a pool of items for investigation. Then, an iterative process was applied to develop a specific food literacy behaviour checklist for program evaluation including exploratory factor analysis. Content, face, and construct validity resulted in a 14-item food behaviour checklist. Three factors entitled Plan & Manage, Selection, and Preparation were evident, resulting in Cronbach's alpha 0.79, 0.76, and 0.81, respectively, indicating good reliability of each of these factors. This research has produced a validated questionnaire, is a useful starting point for other food literacy programs, and has applications globally.

Keywords: evaluation; food literacy; internal consistency; public health; questionnaire; validity.

Conflict of interest statement

The authors declare no conflict of interest. The Funders had no role in the design of the study, collection, analysis, or interpretation of the data, and they had a role in approving the results for publication.

Figures

Figure 1
Figure 1
2-Dimensional plots of the three food literacy behaviours factors and the factor loadings of the items (a) Plan & Manage and Preparation and (b) Selection and Preparation.
Figure 1
Figure 1
2-Dimensional plots of the three food literacy behaviours factors and the factor loadings of the items (a) Plan & Manage and Preparation and (b) Selection and Preparation.

References

    1. Vidgen H.A., Gallegos D. Defining food literacy and its components. Appetite. 2014;76:50–59. doi: 10.1016/j.appet.2014.01.010.
    1. Truman E., Lane D., Elliott C. Defining food literacy: A scoping review. Appetite. 2017;116:365–371. doi: 10.1016/j.appet.2017.05.007.
    1. Azevedo Perry E., Thomas H., Samra H.R., Edmonstone S., Davidson L., Faulkner A., Petermann L., Manafò E., Kirkpatrick S.I. Identifying attributes of food literacy: A scoping review. Public Health Nutr. 2017;20:2406–2415. doi: 10.1017/S1368980017001276.
    1. Gréa Krause C., Beer-Borst S., Sommerhalder K., Hayoz S., Abel T. A short food literacy questionnaire (SFLQ) for adults: Findings from a swiss validation study. Appetite. 2018;120:275–280. doi: 10.1016/j.appet.2017.08.039.
    1. Palumbo R., Annarumma C., Adinolfi P., Vezzosi S., Troiano E., Catinello G., Manna R. Crafting and applying a tool to assess food literacy: Findings from a pilot study. Trends Food Sci. Technol. 2017;67:173–182. doi: 10.1016/j.tifs.2017.07.002.
    1. McGowan L., Pot G.K., Stephen A.M., Lavelle F., Spence M., Raats M., Hollywood L., McDowell D., McCloat A., Mooney E., et al. The influence of socio-demographic, psychological and knowledge-related variables alongside perceived cooking and food skills abilities in the prediction of diet quality in adults: A nationally representative cross-sectional study. Int. J. Behav. Nutr. Phys. Act. 2016;13:111. doi: 10.1186/s12966-016-0440-4.
    1. Garcia A.L., Reardon R., McDonald M., Vargas-Garcia E.J. Community interventions to improve cooking skills and their effects on confidence and eating behaviour. Curr. Nutr. Rep. 2016;5:315–322. doi: 10.1007/s13668-016-0185-3.
    1. Lahne J., Wolfson J.A., Trubek A. Development of the cooking and food provisioning action scale (CAFPAS): A new measurement tool for individual cooking practice. Food Qual. Prefer. 2017;62:96–105. doi: 10.1016/j.foodqual.2017.06.022.
    1. Reicks M., Trofholz A.C., Stang J.S., Laska M.N. Impact of cooking and home food preparation interventions among adults: Outcomes and implications for future programs. J. Nutr. Educ. Behav. 2014;46:259–276. doi: 10.1016/j.jneb.2014.02.001.
    1. Reicks M., Kocher M., Reeder J. Impact of cooking and home food preparation interventions among adults: A systematic review (2011–2016) J. Nutr. Educ. Behav. 2018;50:148–172. doi: 10.1016/j.jneb.2017.08.004.
    1. Contento I.R., Randell J.S., Basch C.E. Review and analysis of evaluation measures used in nutrition education intervention research. J. Nutr. Educ. Behav. 2002;34:2–25. doi: 10.1016/S1499-4046(06)60220-0.
    1. Townsend M.S. Evaluating food stamp nutrition education: Process for development and validation of evaluation measures. J. Nutr. Educ. Behav. 2006;38:18–24. doi: 10.1016/j.jneb.2005.11.008.
    1. Begley A., Gallegos D., Vidgen H. Effectiveness of australian cooking skill interventions. Br. Food J. 2017;119:973–991. doi: 10.1108/BFJ-10-2016-0451.
    1. Townsend M.S., Sylva K., Martin A., Metz D., Wooten-Swanson P. Improving readability of an evaluation tool for low-income clients using visual information processing theories. J. Nutr. Educ. Behav. 2008;40:181–186. doi: 10.1016/j.jneb.2007.06.011.
    1. Banna J.C., Vera Becerra L.E., Kaiser L.L., Townsend M.S. Using qualitative methods to improve questionnaires for spanish speakers: Assessing face validity of a food behavior checklist. J. Am. Diet. Assoc. 2010;110:80–90. doi: 10.1016/j.jada.2009.10.002.
    1. Murphy S.P., Kaiser L.L., Townsend M.S., Allen L.H. Evaluation of validity of items for a food behavior checklist. J. Am. Diet. Assoc. 2001;101:751–761. doi: 10.1016/S0002-8223(01)00189-4.
    1. Kristal A.R., Abrams B.F., Thornquist M.D., Disogra L., Croyle R.T., Shattuck A.L., Henry H.J. Development and validation of a food use checklist for evaluation of community nutrition interventions. Am. J. Public Health. 1990;80:1318–1322. doi: 10.2105/AJPH.80.11.1318.
    1. Branscum P., Sharma M., Kaye G., Succop P. An evaluation of the validity and reliability of a food behavior checklist modified for children. J. Nutr. Educ. Behav. 2010;42:349–352. doi: 10.1016/j.jneb.2009.12.005.
    1. Blackburn M.L., Townsend M.S., Kaiser L.L., Martin A.C., West E.A., Turner B.J., Joy A.B. Food behavior checklist effectively evaluates nutrition education. Calif. Agric. 2006;60:20–24. doi: 10.3733/ca.v060n01p20.
    1. Condrasky M.D., Williams J.E., Catalano P.M., Griffin S.F. Development of psychosocial scales for evaluating the impact of a culinary nutrition education program on cooking and healthful eating. J. Nutr. Educ. Behav. 2011;43:511–516. doi: 10.1016/j.jneb.2010.09.013.
    1. Pinard C.A., Uvena L.M., Quam J.B., Smith T.M., Yaroch A.L. Development and testing of a revised cooking matters for adults survey. Am. J. Health Behav. 2015;39:866–873. doi: 10.5993/AJHB.39.6.14.
    1. Auld G., Baker S., McGirr K., Osborn K.S., Skaff P. Confirming the reliability and validity of others’ evaluation tools before adopting for your programs. J. Nutr. Educ. Behav. 2017;49:441–450. doi: 10.1016/j.jneb.2017.02.006.
    1. Barton K.L., Wrieden W.L., Anderson A.S. Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions. J. Hum. Nutr. Diet. 2011;24:588–595. doi: 10.1111/j.1365-277X.2011.01180.x.
    1. Hutchinson J., Watt J.F., Strachan E.K., Cade J.E. Evaluation of the effectiveness of the ministry of food cooking programme on self-reported food consumption and confidence with cooking. Public Health Nutr. 2016:1–11. doi: 10.1017/S1368980016001476.
    1. Flego A., Herbert J., Gibbs L., Swinburn B., Keating C., Waters E., Moodie M. Methods for the evaluation of the jamie oliver ministry of food program, australia. BMC Public Health. 2013;13:411. doi: 10.1186/1471-2458-13-411.
    1. Schuster E. EFNEP (expanded food and nutrition program) Am. J. Public Health. 1988;78:719–720. doi: 10.2105/AJPH.78.6.719.
    1. Dollahite J., Olson C., Scott-Pierce M. The impact of nutrition education on food insecurity among low-income participants in EFNEP. Fam. Consum. Sci. Res. J. 2009;32:127–139. doi: 10.1177/1077727X03032002003.
    1. Bradford T., Serrano E.L., Cox R.H., Lambur M. Development and testing of a nutrition, food safety, and physical activity checklist for EFNEP and FSNE adult programs. J. Nutr. Educ. Behav. 2010;42:123–130. doi: 10.1016/j.jneb.2008.11.005.
    1. Townsend M.S., Kaiser L.L., Allen L.H., Joy A.B., Murphy S.P. Selecting items for a food behavior checklist for a limited-resource audience. J. Nutr. Educ. Behav. 2003;35:69–77. doi: 10.1016/S1499-4046(06)60043-2.
    1. Hoerr S., Abdulkadri A., Miller S., Waltersdorf C., LaShore M., Martin K., Newkirk C. Improving measurement of the EFNEP outcomes using factor analysis of the behavior checklist. J. Ext. 2011;49:n4.
    1. Anliker J., Willis W., Montogomery S. The Development and Testing of the Behavioural Checklist Questions for the Efnep Evaluation/Reporting System. [(accessed on 20 February 2016)]; Available online: .
    1. Townsend M.S., Ganthavorn C., Neelon M., Donohue S., Johns M.C. Improving the quality of data from efnep participants with low literacy skills: A participant-driven model. J. Nutr. Educ. Behav. 2014;46:309–314. doi: 10.1016/j.jneb.2013.10.004.
    1. Foley R., Pollard C. Food cent$—Implementing and evaluating a nutrition education project focusing on value for money. Aust. N. Z. J. Public Health. 1998;22:494–501. doi: 10.1111/j.1467-842X.1998.tb01420.x.
    1. Butcher L., Chester M.R., Aberle L.M., Bobongie V.J.-A., Davies C., Godrich S.L., Milligan R.A.K., Tartaglia J., Thorne L.M., Begley A. Foodbank of western australia’s healthy food for all. Br. Food J. 2014;116:1490–1505. doi: 10.1108/BFJ-01-2014-0041.
    1. Begley A., Coelho G., Brooks N. Adult Food Literacy: Best Practice Criteria. Department of Health, Western Australia; Perth, Australia: 2015. Unpublished Work.
    1. Stokols D. Translating social ecological theory into guidelines for community health promotion. Am. J. Health Promot. AJHP. 1996;10:282–298. doi: 10.4278/0890-1171-10.4.282.
    1. Coleman M.T., Pasternak R.H. Effective strategies for behavior change. Prim. Care. 2012;39:281–305. doi: 10.1016/j.pop.2012.03.004.
    1. DeVillis R. Scale Development Theory and Applications. Sage; Hemet, CA, USA: 2012.
    1. Phelps J., Brite-Lane A., Crook T., Hakkak R., Fuller S. Demonstrating impact through replicable analysis: Implications of an evaluation of arkansas’s expanded food and nutrition education program. J. Ext. 2017;55:n2.
    1. Australian Bureau of Statistics Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA) [(accessed on 15 April 2016)]; Available online: .
    1. Pollard C., Harray A., Daly A., Kerr D. Nutrition Monitoring Survey Series 2012 Key Findings. [(accessed on 3 June 2016)]; Available online: .
    1. Tomlin S., Joyce S. Health and Wellbeing of Adults in Western Australia 2012, Overview and Trends. [(accessed on 20 February 2016)]; Available online: .
    1. Tomlin S., Joyce S., Radomilijac A. Health and Wellbeing of Adults in Western Australia 2015, Overview and Trends. [(accessed on 20 February 2016)]; Available online: .
    1. Friedman D.B., Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ. Behav. 2006;33:352–373. doi: 10.1177/1090198105277329.
    1. Williams B., Onsman A., Brown T. Exploratory factor analysis: A five-step guide for novices. Aust. J. Paramed. 2010;8:3.
    1. Stevens J. Applied Multivariate Statistics for the Social Sciences. Routledge; Oxford, UK: 2009.
    1. Gliem J.A., Gliem R.R. Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. [(accessed on 15 May 2018)]; Available online: .
    1. Wilson M. Readability and patient education materials used for low-income populations. Clin. Nurse Spec. CNS. 2009;23:33–40. doi: 10.1097/01.NUR.0000343079.50214.31.
    1. Fisher E. Low literacy levels in adults: Implications for patient education. J. Contin. Educ. Nurs. 1999;30:56–61.
    1. Australian Bureau of Statistics Census of Population and Housing, Australia 2016. [(accessed on 27 May 2018)]; Available online: .
    1. Rees R., Hinds K., Dickson K., O’Mara-Eves A., Thomas J. Communities That Cook: A Systematic Review of the Effectiveness and Appropriateness of Interventions to Introduce Adults to Home Cooking. [(accessed on 4 September 2015)]; Available online: .
    1. Murray E.K., Auld G., Baker S.S., Barale K., Franck K., Khan T., Palmer-Keenan D., Walsh J. Methodology for developing a new EFNEP food and physical activity behaviors questionnaire. J. Nutr. Educ. Behav. 2017;49:777–783. doi: 10.1016/j.jneb.2017.05.341.
    1. Carbonneau E., Bradette-Laplante M., Lamarche B., Provencher V., Begin C., Robitaille J., Desroches S., Vohl M.C., Corneau L., Lemieux S. Development and validation of the food liking questionnaire in a french-canadian population. Nutrients. 2017;9:1337. doi: 10.3390/nu9121337.
    1. Huisken A., Orr S.K., Tarasuk V. Adults’ food skills and use of gardens are not associated with household food insecurity in canada. Can. J. Public Health. 2017;107:e526–e532. doi: 10.17269/cjph.107.5692.
    1. Medeiros L.C., Hillers V.N., Kendall P.A., Mason A. Food safety education: What should we be teaching to consumers? J. Nutr. Educ. 2001;33:108–113. doi: 10.1016/S1499-4046(06)60174-7.
    1. Wardlaw M., Baker S. Long-term evaluation of EFNEP and snap-ed. Forum Fam. Consum. Issues (FFCI) 2012;17:2.
    1. Short F. Domestic cooking skills—What are they? HEIA J. 2003;10:13–22.
    1. Garcia A., Reardon R., Hammond E., Parrett A., Gebbie-Diben A. Evaluation of the “eat better feel better” cooking programme to tackle barriers to healthy eating. Int. J. Environ. Res. Public Health. 2017;14:380. doi: 10.3390/ijerph14040380.
    1. Perkins L., Cohen J. Meeting patient needs in the hospital setting—Are written nutrition education resources too hard to understand? Nutr. Diet. 2008;65:216–221. doi: 10.1111/j.1747-0080.2008.00273.x.
    1. (ABS), Australian Bureau of Statistics Programme for the International Assessment of Adult Competencies, Australia, 2011–12. [(accessed on 23 May 2014)]; Available online: .
    1. Clark L.A., Watson D. Constructing validity: Basic issues in objective scale development. Psychol. Assess. 1995;7:309–319. doi: 10.1037/1040-3590.7.3.309.
    1. Subar A.F., Freedman L.S., Tooze J.A., Kirkpatrick S.I., Boushey C., Neuhouser M.L., Thompson F.E., Potischman N., Guenther P.M., Tarasuk V., et al. Addressing current criticism regarding the value of self-report dietary data. J. Nutr. 2015;145:2639–2645. doi: 10.3945/jn.115.219634.
    1. Dhaliwal S.S., Howat P., Bejoy T., Welborn T.A. Self-reported weight and height for evaluating obesity control programs. Am. J. Health Behav. 2010;34:489–499. doi: 10.5993/AJHB.34.4.10.
    1. Braun E., Woodley A., Richardson J., Leidner B. Self-rated competences qusestionnaires from a design perspective. Educ. Res. Rev. 2012;7:1–18. doi: 10.1016/j.edurev.2011.11.005.
    1. King M.F., Bruner G.C. Social desirability bias: A neglected aspect of validity testing. Psychol. Mark. 2000;17:79–103. doi: 10.1002/(SICI)1520-6793(200002)17:2<79::AID-MAR2>;2-0.
    1. Slater J.J., Mudryj A.N. Self-perceived eating habits and food skills of canadians. J. Nutr. Educ. Behav. 2016;48:486–495. doi: 10.1016/j.jneb.2016.04.397.
    1. Bongoni R., Verkerk R., Dekker M., Steenbekkers B. Evaluation of research methods to study domestic food preparation. Br. Food J. 2015;117:7–21. doi: 10.1108/BFJ-09-2013-0273.
    1. Hand R.K., Perzynski A.T. Ecologic momentary assessment: Perspectives on applications and opportunities in research and practice regarding nutrition behaviors. J. Nutr. Educ. Behav. 2016 doi: 10.1016/j.jneb.2016.05.004.

Source: PubMed

赞助者和合作者

医疗条件

药物干预

3
订阅