Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects

Jim McCambridge, John Witton, Diana R Elbourne, Jim McCambridge, John Witton, Diana R Elbourne

Abstract

Objectives: This study aims to (1) elucidate whether the Hawthorne effect exists, (2) explore under what conditions, and (3) estimate the size of any such effect.

Study design and setting: This systematic review summarizes and evaluates the strength of available evidence on the Hawthorne effect. An inclusive definition of any form of research artifact on behavior using this label, and without cointerventions, was adopted.

Results: Nineteen purposively designed studies were included, providing quantitative data on the size of the effect in eight randomized controlled trials, five quasiexperimental studies, and six observational evaluations of reporting on one's behavior by answering questions or being directly observed and being aware of being studied. Although all but one study was undertaken within health sciences, study methods, contexts, and findings were highly heterogeneous. Most studies reported some evidence of an effect, although significant biases are judged likely because of the complexity of the evaluation object.

Conclusion: Consequences of research participation for behaviors being investigated do exist, although little can be securely known about the conditions under which they operate, their mechanisms of effects, or their magnitudes. New concepts are needed to guide empirical studies.

Keywords: Assessment; Hawthorne effect; Observation; Reactivity; Research methods; Research participation.

Copyright © 2014 The Authors. Published by Elsevier Inc. All rights reserved.

Figures

Fig. 1
Fig. 1
PRISMA flowchart.
Fig. 2
Fig. 2
Binary outcome data. RR, relative risk; CI, confidence interval.

References

    1. Parsons H. What happened at Hawthorne? Science. 1974;183:922–932.
    1. Sommer R. The Hawthorne dogma. Psychol Bull. 1968;70(6 Pt 1):592–595.
    1. Wickstrom G., Bendix T. The "Hawthorne effect"-what did the original Hawthorne studies actually show? Scand J Work Environ Health. 2000;26(4):363–367.
    1. Gale E. The Hawthorne studies-a fable for our times? QJM. 2004;97:439–449.
    1. Chiesa M., Hobbs S. Making sense of social research: how useful is the Hawthorne effect? Eur J Soc Psychol. 2008;38(1):67–74.
    1. Mayo E. MacMillan; New York, NY: 1933. The human problems of an industrial civilization.
    1. Roethlisberger F.J., Dickson W.J. Cambridge University Press; Cambridge, MA: 1939. Management and the worker.
    1. Gillespie R. Cambridge University Press; Cambridge, MA: 1991. Manufacturing knowledge: a history of the Hawthorne experiments.
    1. French J.R.P. Experiments in field settings. In: Festinger L., Katz D., editors. Research methods in the behavioral sciences. Holt, Rinehart & Winston; New York, NY: 1953.
    1. Bramel D., Friend R. Hawthorne, the myth of the docile worker, and class bias in psychology. Am Psychol. 1981;36(8):867–878.
    1. Holden J.D. Hawthorne effects and research into professional practice. J Eval Clin Pract. 2001;7:65–70.
    1. Adair J.G., Sharpe D., Huynh C.L. Hawthorne control procedures in educational experiments: a reconsideration of their use and effectiveness. Rev Educ Res. 1989;59(2):215–228.
    1. Cook D.L. Ohio State University; Columbus, OH: 1967. The impact of the Hawthorne effect in experimental designs in educational research. Final report.
    1. Adair J.G. The Hawthorne effect-a reconsideration of the methodological artifact. J Appl Psychol. 1984;69(2):334–345.
    1. Bouchet C., Guillemin F., Briancon S. Nonspecific effects in longitudinal studies: impact on quality of life measures. J Clin Epidemiol. 1996;49:15–20.
    1. McCarney R., Warner J., Iliffe S., van Haselen R., Griffin M., Fisher P. The Hawthorne effect: a randomised, controlled trial. BMC Med Res Methodol. 2007;7:30.
    1. DeAmici D., Klersy C., Ramajoli F., Brustia L., Politi P. Impact of the Hawthorne effect in a longitudinal clinical study: the case of anaesthesia. Control Clin Trials. 2000;21:103–114.
    1. Jones S.R.G. Was there a Hawthorne effect? Am J Sociol. 1992;98:451–468.
    1. Pitcher B.L. The Hawthorne experiments: statistical evidence for a learning hypothesis. Soc Forces. 1981;60(1):133–149.
    1. Franke R.H., Kaul J.D. The Hawthorne experiments: first statistical interpretation. Am Sociol Rev. 1978;43(5):623–643.
    1. Carey A. The Hawthorne studies: a radical criticism. Am Sociol Rev. 1967;32(3):403–416.
    1. Hansson M., Wigblad R. Recontextualizing the Hawthorne effect. Scand J Manage. 2006;22(2):120–137.
    1. Olson R., Hogan L., Santos L. Illuminating the history of psychology: tips for teaching students about the Hawthorne studies. Psychol Learn Teach. 2006;5(2):110–118.
    1. Kompier M.A.J. The "Hawthorne effect" is a myth, but what keeps the story going? Scand J Work Environ Health. 2006;32(5):402–412.
    1. Leonard K.L. Is patient satisfaction sensitive to changes in the quality of care? An exploitation of the Hawthorne effect. J Health Econ. 2008;27(2):444–459.
    1. Higgins J.P.T., Thompson S.G. Quantifying heterogeneity in meta-analysis. Stat Med. 2002;21:1539–1558.
    1. Granberg D., Holmberg S. The Hawthorne effect in election studies-the impact of survey participation on voting. Br J Polit Sci. 1992;22(2):240–247.
    1. van Rooyen S., Godlee F., Evans S., Smith R., Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998;280:234–237.
    1. Feil P.H., Grauer J.S., Gadbury-Amyot C.C., Kula K., McCunniff M.D. Intentional use of the Hawthorne effect to improve oral hygiene compliance in orthodontic patients. J Dent Educ. 2002;66(10):1129–1135.
    1. O'Sullivan I., Orbell S., Rakow T., Parker R. Prospective research in health service settings: health psychology, science and the 'Hawthorne' effect. J Health Psychol. 2004;9(3):355–359.
    1. Kypri K., McAnally H.M. Randomized controlled trial of a web-based primary care intervention for multiple health risk behaviors. Prev Med. 2005;41:761–766.
    1. Kypri K., Langley J.D., Saunders J.B., Cashell-Smith M.L. Assessment may conceal therapeutic benefit: findings from a randomized controlled trial for hazardous drinking. Addiction. 2007;102(1):62–70.
    1. McCambridge J., Day M. Randomized controlled trial of the effects of completing the Alcohol Use Disorders Identification Test questionnaire on self-reported hazardous drinking. Addiction. 2008;103(2):241–248.
    1. Evans R., Joseph-Williams N., Edwards A., Newcombe R.G., Wright P., Kinnersley P. Supporting informed decision making for prostate specific antigen (PSA) testing on the web: an online randomized controlled trial. J Med Internet Res. 2010;12(3):e27.
    1. Murray M., Swan A., Kiryluk S., Clarke G. The Hawthorne effect in the measurement of adolescent smoking. J Epidemiol Community Health. 1988;42:304–306.
    1. Malotte C.K., Morisky D.E. Using an unobstrusively monitored comparison study-group in a longitudinal design. Health Educ Res. 1994;9(1):153–159.
    1. McCusker J., Karp E., Yaffe M.J., Cole M., Bellavance F. Determining detection of depression in the elderly by primary care physicians: chart review or questionnaire? Prim Care Psychiatry. 1996:217–221.
    1. Ertem I.O., Votto N., Leventhal J.M. The timing and predictors of the early termination of breastfeeding. Pediatrics. 2001;107(3):543–548.
    1. Fernald D.H., Coombs L., DeAlleaume L., West D., Parnes B. An assessment of the Hawthorne effect in practice-based research. J Am Board Fam Med. 2012;25(1):83–86.
    1. Campbell J.P., Maxey V.A., Watson W.A. Hawthorne effect-implications for prehospital research. Ann Emerg Med. 1995;26(5):590–594.
    1. Mangione-Smith R., Elliott M.N., McDonald L., McGlynn E.A. An observational study of antibiotic prescribing behavior and the Hawthorne effect. Health Serv Res. 2002;37:1603–1623.
    1. Eckmanns T., Bessert J., Behnke M., Gastmeier P., Ruden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol. 2006;27:931–934.
    1. Leonard K., Masatu M.C. Outpatient process quality evaluation and the Hawthorne effect. Soc Sci Med. 2006;63(9):2330–2340.
    1. Maury E., Moussa N., Lakermi C., Barbut F., Offenstadt G. Compliance of health care workers to hand hygiene: awareness of being observed is important. Intensive Care Med. 2006;32(12):2088–2089.
    1. Fox N.S., Brennan J.S., Chasen S.T. Clinical estimation of fetal weight and the Hawthorne effect. Eur J Obstet Gynecol Reprod Biol. 2008;141(2):111–114.
    1. French D.P., Sutton S. Reactivity of measurement in health psychology: how much of a problem is it? What can be done about it? Br J Health Psychol. 2010;15(Pt 3):453–468.
    1. Solomon R.L. An extension of control group design. Psychol Bull. 1949;46(2):137–150.
    1. McCambridge J., Kypri K., Elbourne D.R. A surgical safety checklist. N Engl J Med. 2009;360:2373–2374. author reply 2374-5.
    1. McCambridge J., Butor-Bhavsar K., Witton J., Elbourne D. Can research assessments themselves cause bias in behaviour change trials? A systematic review of evidence from Solomon 4-group studies. PLoS One. 2011;6(10):e25223.
    1. McCambridge J., Kypri K. Can simply answering research questions change behaviour? Systematic review and meta analyses of brief alcohol intervention trials. PLoS One. 2011;6(10):e23748.
    1. McCambridge J., de Bruin M., Witton J. The effects of demand characteristics on research participant behaviours in non-laboratory settings: a systematic review. PLoS One. 2012;7(6):e39116.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever