Monitoring patient care through health facility exit interviews: an assessment of the Hawthorne effect in a trial of adherence to malaria treatment guidelines in Tanzania

Baptiste Leurent, Hugh Reyburn, Florida Muro, Hilda Mbakilwa, David Schellenberg, Baptiste Leurent, Hugh Reyburn, Florida Muro, Hilda Mbakilwa, David Schellenberg

Abstract

Background: Survey of patients exiting health facilities is a common way to assess consultation practices. It is, however, unclear to what extent health professionals may change their practices when they are aware of such interviews taking place, possibly paying more attention to following recommended practices. This so-called Hawthorne effect could have important consequences for interpreting research and programme monitoring, but has rarely been assessed.

Methods: A three-arm cluster-randomised trial of interventions to improve adherence to guidelines for the use of anti-malarial drugs was conducted in Tanzania. Patient interviews were conducted outside health facilities on two randomly-selected days per week. Health workers also routinely documented consultations in their ledgers. The Hawthorne effect was investigated by comparing routine data according to whether exit interviews had been conducted on three key indicators of malaria care. Adjusted logistic mixed-effects models were used, taking into account the dependencies within health facilities and calendar days.

Results: Routine data were collected on 19,579 consultations in 18 facilities. The odds of having a malaria rapid diagnostic test (RDT) result reported were 11 % higher on days when exit surveys were conducted (adjusted odds ratio 95 % CI: 0.98-1.26, p = 0.097), 17 % lower for prescribing an anti-malarial drug to patients with a negative RDT result (0.56-1.23, p = 0.343), and 27 % lower for prescribing an anti-malarial when no RDT result was reported (0.53-1.00, p = 0.052). The effect varied with time, with a U-shaped association over the study period (p < 0.001). We also observed a higher number of consultations recorded on days when exit-interviews were conducted (adjusted mean difference = 2.03, p < 0.001).

Conclusions: Although modest, there was some suggestion of better practice by health professionals on days when exit interviews were conducted. Researchers should be aware of the potential Hawthorne effect, and take into account assessment methods when generalising findings to the 'real word' setting. This effect is, however, likely to be context dependent, and further controlled evaluation across different settings should be conducted.

Trial registration: ClinicalTrials.gov: NCT01292707 . Registered on 29th January 2011.

Figures

Fig. 1
Fig. 1
Hawthorne effect on reporting a RDT result, by study period. Odds ratio of reporting a RDT result for survey days compared to non-survey days. Estimates from a three-level hierarchical model (with health facility and calendar day as random effects) adjusted for day of the week, and stratified by study period. RDT = Rapid diagnostic test, CI Confidence Interval

References

    1. Gillespie R. Manufacturing knowledge: a history of the Hawthorne experiments. Cambridge: Cambridge University Press; 1993.
    1. Roethlisberger FJ, Dickson WJ. Management and the Worker Cambridge, Mass: Harvard University Press; 1939.
    1. Kompier MAJ. The “Hawthorne effect” is a myth, but what keeps the story going? Scand J Work Environ Health. 2006;32(5):402–12. doi: 10.5271/sjweh.1036.
    1. Levitt SD, List JA. Was there really a hawthorne effect at the Hawthorne Plant? An analysis of the original illumination experiments. Am Econ J Appl Econ. 2011;3(1):224–38. doi: 10.1257/app.3.1.224.
    1. McCambridge J, Kypri K, Elbourne D. Research participation effects: a skeleton in the methodological cupboard. J Clin Epidemiol. 2014;67(8):845–9. doi: 10.1016/j.jclinepi.2014.03.002.
    1. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67(3):267–77. doi: 10.1016/j.jclinepi.2013.08.015.
    1. Cundill B, Mbakilwa H, Chandler CI, Mtove G, Mtei F, Willetts A, et al. Prescriber and patient-oriented behavioural interventions to improve use of malaria rapid diagnostic tests in Tanzania: facility-based cluster randomised trial. BMC Med. 2015;13(1):118. doi: 10.1186/s12916-015-0346-z.
    1. London School of Hygiene and Tropical Medicine. A Cluster-randomised Trial of Interventions to Improve Antimalarial Prescribing With Malaria Tests (TACT). In: . National Library of Medicine (US): Bethesda (MD); 2011. . Accessed 11 Feb 2015.
    1. NMCP . Training guide and facilitators manual for malaria rapid diagnostic tests. Tanzania: Ministry of Health and Social Welfare; 2010.
    1. Leurent B, Reyburn H, Schellenberg D. Analysis Plan - Hawthorne effect in TACT study. .
    1. McCulloch CE, Searle SR. Generalized, Linear, and Mixed Models. New York: John Wiley & Sons, Ltd; 2001.
    1. Adams AS, Soumerai SB, Lomas J, Ross-Degnan D. Evidence of self-report bias in assessing adherence to guidelines. Int J Qual Health Care. 1999;11(3):187–92. doi: 10.1093/intqhc/11.3.187.
    1. Baiden F, Owusu-Agyei S, Bawah J, Bruce J, Tivura M, Delmini R, et al. An evaluation of the clinical assessments of under-five febrile children presenting to primary health facilities in rural Ghana. PLoS One. 2011;6(12):e28944. doi: 10.1371/journal.pone.0028944.
    1. Mukanga D, Babirye R, Peterson S, Pariyo GW, Ojiambo G, Tibenderana JK, et al. Can lay community health workers be trained to use diagnostics to distinguish and treat malaria and pneumonia in children? Lessons from rural Uganda. Trop Med Int Health. 2011;16(10):1234–42. doi: 10.1111/j.1365-3156.2011.02831.x.
    1. Font F, Alonso Gonzalez M, Nathan R, Kimario J, Lwilla F, Ascaso C, et al. Diagnostic accuracy and case management of clinical malaria in the primary health services of a rural area in south‐eastern Tanzania. Trop Med Int Health. 2001;6(6):423–8. doi: 10.1046/j.1365-3156.2001.00727.x.
    1. Schellenberg JRA, Adam T, Mshinda H, Masanja H, Kabadi G, Mukasa O, et al. Effectiveness and cost of facility-based Integrated Management of Childhood Illness (IMCI) in Tanzania. Lancet. 2004;364(9445):1583–94. doi: 10.1016/S0140-6736(04)17311-X.
    1. Beehler GP, Finnell DS, Foltz K, Edwards D. Patient exit interviews for assessing fidelity to the VA VISN 2 Integrated Primary Care Model: Results from a pilot feasibility study. J Interprof Care. 2010;24(3):323–6. doi: 10.3109/13561820902921894.
    1. Mbacham WF, Mangham-Jefferies L, Cundill B, Achonduh OA, Chandler CI, Ambebila JN, et al. Basic or enhanced clinician training to improve adherence to malaria treatment guidelines: a cluster-randomised trial in two areas of Cameroon. Lancet Glob Health. 2014;2(6):e346–e58. doi: 10.1016/S2214-109X(14)70201-3.
    1. Reyburn H, Mbakilwa H, Mwangi R, Mwerinde O, Olomi R, Drakeley C, et al. Rapid diagnostic tests compared with malaria microscopy for guiding outpatient treatment of febrile illness in Tanzania: randomised trial. BMJ. 2007;334(7590):403. doi: 10.1136/.
    1. Leonard K, Masatu MC. Outpatient process quality evaluation and the Hawthorne effect. Soc Sci Med. 2006;63(9):2330–40. doi: 10.1016/j.socscimed.2006.06.003.
    1. Mangione‐Smith R, Elliott MN, McDonald L, McGlynn EA. An observational study of antibiotic prescribing behavior and the Hawthorne effect. Health Serv Res. 2002;37(6):1603–23. doi: 10.1111/1475-6773.10482.
    1. Rowe S, Olewe M, Kleinbaum D, McGowan J, Jr, McFarland D, Rochat R, et al. The influence of observation and setting on community health workers’ practices. Int J Qual Health Care. 2006;18(4):299–305. doi: 10.1093/intqhc/mzl009.
    1. Eisele TP, Silumbe K, Yukich J, Hamainza B, Keating J, Bennett A, et al. Measuring coverage in MNCH: accuracy of measuring diagnosis and treatment of childhood malaria from household surveys in Zambia. PLoS Med. 2013;10(5):e1001417. doi: 10.1371/journal.pmed.1001417.
    1. Cook DL. The Hawthorne effect in educational research. Phi Delta Kappan. 1962;44:116–22.
    1. Wickström G, Bendix T. Commentary. Scand J Work Environ Health. 2000;26(4):363–7. doi: 10.5271/sjweh.555.
    1. Herrera S, Vallejo AF, Quintero JP, Arévalo-Herrera M, Cancino M, Ferro S. Field evaluation of an automated RDT reader and data management device for Plasmodium falciparum/Plasmodium vivax malaria in endemic areas of Colombia. Malar J. 2014;13(1):87. doi: 10.1186/1475-2875-13-87.

Source: PubMed

Sponsorzy i współpracownicy

Warunki medyczne

Interwencje lekowe

3
Subskrybuj