Measuring Comprehensive, General Health Literacy in the General Adult Population: The Development and Validation of the HLS19-Q12 Instrument in Seventeen Countries

Jürgen M Pelikan, Thomas Link, Christa Straßmayr, Karin Waldherr, Tobias Alfers, Henrik Bøggild, Robert Griebler, Maria Lopatina, Dominika Mikšová, Marie Germund Nielsen, Sandra Peer, Mitja Vrdelja, HLS19 Consortium of the WHO Action Network M-POHL, Jürgen M Pelikan, Thomas Link, Christa Straßmayr, Karin Waldherr, Tobias Alfers, Henrik Bøggild, Robert Griebler, Maria Lopatina, Dominika Mikšová, Marie Germund Nielsen, Sandra Peer, Mitja Vrdelja, HLS19 Consortium of the WHO Action Network M-POHL

Abstract

Background: For improving health literacy (HL) by national and international public health policy, measuring population HL by a comprehensive instrument is needed. A short instrument, the HLS19-Q12 based on the HLS-EU-Q47, was developed, translated, applied, and validated in 17 countries in the WHO European Region.

Methods: For factorial validity/dimensionality, Cronbach alphas, confirmatory factor analysis (CFA), Rasch model (RM), and Partial Credit Model (PCM) were used. For discriminant validity, correlation analysis, and for concurrent predictive validity, linear regression analysis were carried out.

Results: The Cronbach alpha coefficients are above 0.7. The fit indices for the single-factor CFAs indicate a good model fit. Some items show differential item functioning in certain country data sets. The regression analyses demonstrate an association of the HLS19-Q12 score with social determinants and selected consequences of HL. The HLS19-Q12 score correlates sufficiently highly (r ≥ 0.897) with the equivalent score for the HLS19-Q47 long form.

Conclusions: The HLS19-Q12, based on a comprehensive understanding of HL, shows acceptable psychometric and validity characteristics for different languages, country contexts, and methods of data collection, and is suitable for measuring HL in general, national, adult populations. There are also indications for further improvement of the instrument.

Keywords: HLS19; HLS19-Q12; health literacy measurement; instrument development; national adult population; validation.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Percentages of respondents who responded with “very difficult” or “difficult” to the HLS19-Q12 items (ordered by the overall mean), for each country.
Figure 2
Figure 2
Histograms of the HLS19-Q12 scores (type D), for all countries.
Figure 3
Figure 3
Histograms of the HLS19-Q12 scores (type P), for all countries.
Figure 4
Figure 4
Distribution of the HLS19-Q12 levels for general HL, based on type D scores, for each country and the mean for all countries.
Figure 5
Figure 5
Distribution of the HLS19-Q12 levels for general HL, based on type P scores, for each country and the mean for all countries.

References

    1. Kickbusch I., Pelikan J.M., Apfel F., Tsouros A.D. Health Literacy: The Solid Facts. World Health Organization (WHO) Regional Office for Europe; Copenhagen, Denmark: 2013.
    1. Quaglio G., Sørensen K., Rübig P., Bertinato L., Brand H., Karapiperis T., Dinca I., Peetso T., Kadenbach K., Dario C. Accelerating the health literacy agenda in Europe. Health Promot. Int. 2016;32:1074–1080. doi: 10.1093/heapro/daw028.
    1. HLS-EU Consortium . Comparative Report on Health Literacy in Eight EU Member States. The European Health Literacy Survey HLS-EU; Wien, Austria: 2012. Second Extended and Revised Version, Date 22 July 2014.
    1. Sørensen K., Pelikan J.M., Röthlin F., Ganahl K., Slonska Z., Doyle G., Fullam J., Kondilis B., Agrafiotis D., Uiters E., et al. Health literacy in Europe: Comparative results of the European health literacy survey (HLS-EU) Eur. J. Public Health. 2015;25:1053–1058. doi: 10.1093/eurpub/ckv043.
    1. M-POHL . Concept Note. For a WHO Action Network on Measuring Population and Organizational Health Literacy (M-POHL Network) within the European Health Information Initiative (EHII) M-POHL; Vienna, Austria: 2018.
    1. Dietscher C., Pelikan J., Bobek J., Nowak P. The action network on measuring population and organizational health literacy (M-POHL) a network under the umbrella of the WHO European health information initiative (EHII) Public Health Panor. 2019;5:71.
    1. M-POHL . The Vienna Statement on the Measurement of Population and Organizational Health Literacy in Europe. M-POHL; Vienna, Austria: 2018.
    1. Sørensen K., Van den Broucke S., Fullam J., Doyle G., Pelikan J., Slonska Z., Brand H., (HLS-EU) Consortium Health Literacy Project European Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health. 2012;12:80. doi: 10.1186/1471-2458-12-80.
    1. Sørensen K., van den Broucke S., Pelikan J.M., Fullam J., Doyle G., Slonska Z., Kondilis B., Stoffels V., Osborne R.H., Brand H. Measuring health literacy in populations: Illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q) BMC Public Health. 2013;13:948. doi: 10.1186/1471-2458-13-948.
    1. Pelikan J.M., Ganahl K. Measuring Health Literacy in General Populations: Primary Findings from the HLS-EU Consortium’s Health Literacy Assessment Effort. Stud. Health Technol. Inf. 2017;240:34–59. doi: 10.3233/978-1-61499-790-0-34.
    1. Pelikan J.M., Straßmayr C., Ganahl K. Health Literacy Measurement in General and Other Populations: Further Initiatives and Lessons Learned in Europe (and Beyond) In: Logan G.D., Siegel E.R., editors. Health Literacy in Clinical Practice and Public Health. IOS Press; Amsterdam, The Netherlands: 2020. pp. 170–191. New Initiatives and Lessons Learned at the Intersection with other Disciplines.
    1. The HLS19 Consortium of the WHO Action Network M-POHL . International Report on the Methodology, Results, and Recommendations of the European Health Literacy Population Survey 2019-2021 (HLS19) of M-POHL. Austrian National Public Health Institute; Vienna, Austria: 2021.
    1. Schaeffer D., Vogt D.o., Berens E.-M., Messer M., Quenzel G., Hurrelmann K. Health literacy in Deutschland. In: Schaeffer D., Pelikan J.M., editors. Health Literacy. Forschungsstand und Perspektiven. Hogrefe; Bern, Switzerland: 2017. p. 144.
    1. Schaeffer D., Vogt D., Berens E.-M., Hurrelmann K. Gesundheitskompetenz der Bevölkerung in Deutschland: Ergebnisbericht. Universität Bielefeld, Fakultät für Gesundheitswissenschaften; Bielefeld, Germany: 2016.
    1. Pelikan J.M., Ganahl K. Die europäische gesundheitskompetenz-studie: Konzept, instrument und ausgewählte Ergebnisse. In: Schaeffer D., Pelikan J.M., editors. Health Literacy. Forschungsstand und Perspektiven. Hogrefe; Bern, Switzerland: 2017. p. 125.
    1. Pelikan J.M., Ganahl K., van den Broucke S., Sørensen K. In: Measuring health literacy in Europe: Introducing the European Health Literacy Survey Questionnaire (HLS-EU-Q) In International Handbook of Health Literacy. Research, Practice and Policy across the Life-Span. Okan O., Bauer U., Pinheiro P., Levin-Zamir D., Sorensen K., editors. Policy Press; Bristol, UK: 2019. p. 138.
    1. Pelikan J.M., Röthlin F., Ganahl K., Peer S. Measuring comprehensive health literacy in general populations–the HLS-EU instruments; Proceedings of the International Conference of Health Literacy and Health Promotion; Taipei, Taiwan. 6 October 2014; p. 57.
    1. Röthlin F., Pelikan J.M., Ganahl K. Die Gesundheitskompetenz der 15-jährigen Jugendlichen in Österreich. Abschlussbericht der österreichischen Gesundheitskompetenz Jugendstudie im Auftrag des Hauptverbands der Österreichischen Sozialversicherungsträger. Ludwig Boltzmann Institut für Health Promotion Research; Wien, Austria: 2013.
    1. Ganahl K., Dahlvik J., Pelikan J.M. Health literacy and experienced communication in health care among immigrants in Austria. Findings from the Austrian health literacy immigrant study; Proceedings of the 25th International Conference on Health Promoting Hospitals and Health Services; Vienna, Austria. 12–14 April 2017.
    1. Ganahl K., Dahlvik J., Röthlin F., Alpagu F., Sikic-Fleischhacker A., Peer S., Pelikan J.M. Ergebnisse Einer Quantitativen und Qualitativen Studie. Ludwig Boltzmann Institut Forschungsbericht; Wien, Austria: 2016. Gesundheitskompetenz bei personen mit migrationshintergrund aus der türkei und ex-jugoslawien in Österreich.
    1. Wangdahl J.M., Dahlberg K., Jaensson M., Nilsson U. Psychometric validation of Swedish and Arabic versions of two health literacy questionnaires, eHEALS and HLS-EU-Q16, for use in a Swedish context: A study protocol. BMJ Open. 2019;9:e029668. doi: 10.1136/bmjopen-2019-029668.
    1. Gele A.A., Pettersen K.S., Torheim L.E., Kumar B. Health literacy: The missing link in improving the health of Somali immigrant women in Oslo. BMC Public Health. 2016;16:1134. doi: 10.1186/s12889-016-3790-6.
    1. Wangdahl J.M., Lytsy P., Mårtensson L., Westerling R. Health literacy and refugees’ experiences of the health examination for asylum seekers–a Swedish cross-sectional study. BMC Public Health. 2015;15:1162. doi: 10.1186/s12889-015-2513-8.
    1. Lorini C., Santomauro F., Grazzini M., Mantwill S., Vettori V., Lastrucci V., Bechini A., Boccalini S., Bussotti A., Bonaccorsi G. Health literacy in Italy: A cross-sectional study protocol to assess the health literacy level in a population-based sample, and to validate health literacy measures in the Italian language. BMJ Open. 2017;7:e017812. doi: 10.1136/bmjopen-2017-017812.
    1. Lorini C., Lastrucci V., Mantwill S., Vettori V., Bonaccorsi G. Measuring health literacy in Italy: A validation study of the HLS-EU-Q16 and of the HLS-EU-Q6 in Italian language, conducted in Florence and its surroundings. Ann. dell’Ist. Super. Sanità. 2019;55:10–18. doi: 10.4415/ANN_19_01_04.
    1. Soenaryati S., Rachmani E. Media use behavior and health literacy on high school students in Semarang. Adv. Sci. Lett. 2017;23:3493–3496. doi: 10.1166/asl.2017.9145.
    1. Gerich J., Moosbrugger R. Subjective estimation of health literacy—What is measured by the HLS-EU scale and how is it linked to empowerment? Health Commun. 2016;33:254–263. doi: 10.1080/10410236.2016.1255846.
    1. Rouquette A., Nadot T., Labitrie P., Broucke S.V.D., Mancini J., Rigal L., Ringa V. Validity and measurement invariance across sex, age, and education level of the French short versions of the European Health Literacy Survey Questionnaire. PLoS ONE. 2018;13:e0208091. doi: 10.1371/journal.pone.0208091.
    1. Rouquette A., Rigal L., Mancini J., Guillemin F., Broucke S.V.D., Allaire C., Azogui-Levy S., Ringa V., Hassler C. Health Literacy throughout adolescence: Invariance and validity study of three measurement scales in the general population. Patient Educ. Couns. 2021;105:996–1003. doi: 10.1016/j.pec.2021.07.044.
    1. Emiral G.Ö., Atalay B., Aygar H., Göktas S. Health literacy scale-European union-Q16: A validity and reliability study in Turkey. Int. Res. J. Med. Sci. 2018;6:1–7.
    1. Uemura K., Yamada M., Okamoto H. Effects of active learning on health literacy and behavior in older adults: A randomized controlled trial. J. Am. Geriatr. Soc. 2018;66:1721–1729. doi: 10.1111/jgs.15458.
    1. Gustafsdottir S.S., Sigurdardottir A.K., Arnadottir S.A., Heimisson G.T., Mårtensson L. Translation and cross-cultural adaptation of the European Health Literacy Survey Questionnaire, HLS-EU-Q16: The Icelandic version. BMC Public Health. 2020;20:61. doi: 10.1186/s12889-020-8162-6.
    1. Coman M.A., Forray A.I., Broucke S.V.D., Chereches R.M. Measuring health literacy in Romania: Validation of the HLS-EU-Q16 survey questionnaire. Int. J. Public Health. 2022;67:1604272. doi: 10.3389/ijph.2022.1604272.
    1. Mohamad E.M.W., Kaundan M.K., Hamzah M.R., Azlan A.A., Ayub S.H., Tham J.S., Ahmad A.L. Establishing the HLS-M-Q18 short version of the European health literacy survey questionnaire for the Malaysian context. BMC Public Health. 2020;20:580. doi: 10.1186/s12889-020-08704-7.
    1. Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests. Danmarks Pædagogiske Institut; Copenhagen, Denmark: 1960. Studies in mathematical psychology: I; p. 80.
    1. Van Duong T., Chang P.W., Yang S.-H., Chen M.-C., Chao W.-T., Chen T., Chiao P., Huang H.-L. A new comprehensive short-form health literacy survey tool for patients in general. Asian Nurs. Res. 2017;11:30–35. doi: 10.1016/j.anr.2017.02.001.
    1. Duong T.V., Aringazina A., Kayupova G., Nurjanah, Pham T.V., Pham K.M., Truong T.Q., Nguyen K.T., Oo W.M., Su T.T., et al. Development and validation of a new short-form health literacy instrument (HLS-SF12) for the general public in six Asian countries. HLRP Health Lit. Res. Pract. 2019;3:e91–e102. doi: 10.3928/24748307-20190225-01.
    1. Van Duong T., Nguyen T.T.P., Pham K.M., Nguyen K.T., Giap M.H., Tran T.D.X., Nguyen C.X., Yang S.-H., Su C.-T. Validation of the short-form health literacy questionnaire (HLS-SF12) and its determinants among people living in rural areas in Vietnam. Int. J. Environ. Res. Public Health. 2019;16:3346. doi: 10.3390/ijerph16183346.
    1. Finbråten H.S., Wilde-Larsson B., Nordström G., Pettersen K.S., Trollvik A. Guttersrud, establishing the HLS-Q12 short version of the European health literacy survey questionnaire: Latent trait analyses applying Rasch modelling and confirmatory factor analysis. BMC Health Serv. Res. 2018;18:506. doi: 10.1186/s12913-018-3275-7.
    1. Waldherr K., Alfers T., Peer S., Pelikan J.M. Development and Validation of the HLS-EU-Q12; Vienna, Austria. 2019. [(accessed on 1 September 2022)]. Available online: .
    1. Masters G.N. A rasch model for partial credit scoring. Psychometrika. 1982;47:149–174. doi: 10.1007/BF02296272.
    1. Storms H., Claes N., Aertgeerts B., Broucke S.V.D. Measuring health literacy among low literate people: An exploratory feasibility study with the HLS-EU questionnaire. BMC Public Health. 2017;17:1–10. doi: 10.1186/s12889-017-4391-8.
    1. Domanska O.M., Firnges C., Bollweg T.M., Sørensen K., Holmberg C., Jordan S. Do adolescents understand the items of the European Health Literacy Survey Questionnaire (HLS-EU-Q47)–German version? Findings from cognitive interviews of the project “Measurement of Health Literacy Among Adolescents” (MOHLAA) in Germany. Arch. Public Health. 2018;76:46. doi: 10.1186/s13690-018-0276-2.
    1. Finbråten H.S., Pettersen K.S., Wilde-Larsson B., Nordström G., Trollvik A., Guttersrud Ø. Validating the european health literacy survey questionnaire in people with type 2 diabetes: Latent trait analyses applying multidimensional Rasch modelling and confirmatory factor analysis. J. Adv. Nurs. 2017;73:2730–2744. doi: 10.1111/jan.13342.
    1. Huang Y.-J., Lin G.-H., Lu W.-S., Tam K.-W., Chen C., Hou W.-H., Hsieh C.-L. Validation of the European health literacy survey questionnaire in women with breast cancer. Cancer Nurs. 2018;41:E40–E48. doi: 10.1097/NCC.0000000000000475.
    1. IBM Corp . IBM SPSS Statistics for Windows, Version 27.0. IBM Corp.; Armonk, NY, USA: 2020.
    1. R Core Team . R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing; Vienna, Austria: 2020.
    1. Eisinga R., Grotenhuis M.t., Pelzer B. The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? Int. J. Public Health. 2013;58:637–642. doi: 10.1007/s00038-012-0416-3.
    1. Kline R.B. Principles and Practice of Structural Equation Modeling. Guilford Publications; New York, NY, USA: 2015.
    1. Zumbo B.D., Gadermann A.M., Zeisser C. Ordinal versions of coefficients alpha and theta for Likert rating scales. J. Mod. Appl. Stat. Methods. 2007;6:29. doi: 10.22237/jmasm/1177992180.
    1. Chalmers R.P. On misconceptions and the limited usefulness of ordinal alpha. Educ. Psychol. Meas. 2017;78:1056–1071. doi: 10.1177/0013164417727036.
    1. Zumbo B.D., Kroc E. A measurement is a choice and stevens’ scales of measurement do not help make it: A response to chalmers. Educ. Psychol. Meas. 2019;79:1184–1197. doi: 10.1177/0013164419844305.
    1. Revelle W. Psych: Procedures for Psychological, Psychometric, and Personality Research, Northwestern University: Evanston, Illinois, United States. 2021. [(accessed on 3 October 2022)]. Available online: .
    1. Prudon P. Confirmatory factor analysis: A brief introduction and critique. Compr. Psychol. 2015;4:18. doi: 10.2466/03.CP.4.10.
    1. Roussel Y. lavaan: An R package for structural equation modeling. [(accessed on 3 October 2022)];J. Stat. Softw. 2012 48:36. Available online:
    1. Beaujean A.A. Latent Variable Modeling Using R: A Step-by-Step Guide. Routledge; London, UK: 2014.
    1. Rosseel Y. The Lavaan Tutorial. Ghent University; Ghent, Belgium: 2021.
    1. Waldherr K., Alfers T., Peer S. Development and Validation of the HLS-EU-Q12. M-POHL; Vienna, Austria: 2021. [(accessed on 1 September 2022)]. Available online: .
    1. Guttersrud Ø., Le C., Pettersen K.S., Finbråten H.S. Rasch Analyses of Data Collected in 17 Countries. A Technical Report to Support Decision-Making within the M-POHL Consortium. Oslo, Norway. 2021. [(accessed on 1 September 2022)]. Available online: .
    1. Fischer G.H. Rasch Models. In: Rao C.R., Sinharay S., editors. Handbook of Statistics, Psychometrics. Volume 26. Elsevier Science Publishers; Amsterdam, The Netherlands: 2006. pp. 515–585.
    1. Mair P., Hatzinger R., Maier M.J. eRm: Extended Rasch Modeling. 1.0-1. 2020. [(accessed on 3 October 2022)]. Available online: .
    1. Sveidqvist K., Bostock M., Pettitt C., Daines M., Kashcha A., Iannone R. DiagrammeR: Create Graph Diagrams and Flowcharts Using R. R Package Version 0.9. 0. 2017. [(accessed on 11 March 2022)]. Available online: .
    1. Chalmers R.P. mirt: A multidimensional item response theory package for the R environment. J. Stat. Softw. 2012;48:1–29. doi: 10.18637/jss.v048.i06.
    1. Adams R.J. Reliability as a measurement design effect. Stud. Educ. Evaluation. 2005;31:162–172. doi: 10.1016/j.stueduc.2005.05.008.
    1. Maydeu-Olivares A. Goodness-of-fit assessment of item response theory models. Meas. Interdiscip. Res. Perspect. 2013;11:71–101. doi: 10.1080/15366367.2013.831680.
    1. Hagell P. Testing rating scale unidimensionality using the principal component analysis (PCA)/t-test protocol with the Rasch model: The primacy of theory over statistics. Open J. Stat. 2014;4:456–465. doi: 10.4236/ojs.2014.46044.
    1. Smith E.V., Jr. Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. J. Appl. Meas. 2002;3:205–231.
    1. Agresti A., Coull B.A. Approximate is better than “exact” for interval estimation of binomial proportions. Am. Stat. 1998;52:119. doi: 10.2307/2685469.
    1. Yen W.M. Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Appl. Psychol. Meas. 1984;8:125–145. doi: 10.1177/014662168400800201.
    1. Linacre J., Wright B. Rasch Measurement Transactions. MESA Press; Chicago, IL, USA: 1995.
    1. Smith A.B., Rush R., Fallowfield L.J., Velikova G., Sharpe M. Rasch fit statistics and sample size considerations for polytomous data. BMC Med. Res. Methodol. 2008;8:33. doi: 10.1186/1471-2288-8-33.
    1. Wright B., Linacre J. Reasonable mean-square fit values. Rasch Meas. Trans. 1994;8:371.
    1. Thissen D., Cai L., Bock R. The nominal categories model. In: Nering M., Ostini R., editors. Handbook of Polytomous Item Response Theory Models. Routledge; London, UK: 2010. pp. 43–75.
    1. Andersen E.B. A goodness of fit test for the rasch model. Psychometrika. 1973;38:123–140. doi: 10.1007/BF02291180.
    1. Fischer G.H. On the existence and uniqueness of maximum-likelihood estimates in the Rasch model. Psychometrika. 1981;46:59–77. doi: 10.1007/BF02293919.
    1. Fischer G.H., Scheiblechner H. Algorithms and programs for the probabilistic test model of Rasch. Psychol. Beiträge. 1970;12:23–51.
    1. Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests. 2nd ed. University of Chicago Press; Chicago, IL, USA: 1980.
    1. Ponocny I. Nonparametric goodness-of-fit tests for the rasch model. Psychometrika. 2001;66:437–459. doi: 10.1007/BF02294444.
    1. Baccolini V., Rosso A., Di Paolo C., Isonne C., Salerno C., Migliara G., Prencipe G.P., Massimi A., Marzuillo C., De Vito C., et al. What is the prevalence of low health literacy in european union member states? A systematic review and meta-analysis. J. Gen. Intern. Med. 2021;36:753–761. doi: 10.1007/s11606-020-06407-8.
    1. Dormann C.F., Elith J., Bacher S., Buchmann C., Carl G., Carré G., Marquéz J.R.G., Gruber B., Lafourcade B., Leitão P.J., et al. Collinearity: A review of methods to deal with it and a simulation study evaluating their performance. Ecography. 2013;36:27–46. doi: 10.1111/j.1600-0587.2012.07348.x.
    1. Lumley T. Complex Surveys: A Guide to Analysis Using R. Volume 565 John Wiley & Sons; Hoboken, NJ, USA: 2011.
    1. Hu L.T., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. A Multidiscip. J. 1999;6:1–55. doi: 10.1080/10705519909540118.
    1. García-Pérez M.A. Order-constrained estimation of nominal response model parameters to assess the empirical order of categories. Educ. Psychol. Meas. 2017;78:826–856. doi: 10.1177/0013164417714296.
    1. Pelikan J.M., Röthlin F., Ganahl K. Measuring health literacy in general populations in Europe. Selected results from the HLS-EU Study; Proceedings of the 21 IUHPE World Conference on Health Promotion; Pattaya, Thailand. 27 August 2013.
    1. Osborne R., Elsworth G., Hawkins M., Cheng C., Elmer S. Measurement of Health Literacy: Assumptions and Potential Consequences Locally and Globally. Swinburne University of Technology; Melbourne, Australia: 2021.

Source: PubMed

Sponsors and Collaborators

Medical Conditions

Drug Interventions

3
Subscribe