Method for appraising model validity of randomised controlled trials of homeopathic treatment: multi-rater concordance study

Robert T Mathie, Helmut Roniger, Michel Van Wassenhoven, Joyce Frye, Jennifer Jacobs, Menachem Oberbaum, Marie-France Bordet, Chaturbhuja Nayak, Gilles Chaufferin, John A Ives, Flávio Dantas, Peter Fisher, Robert T Mathie, Helmut Roniger, Michel Van Wassenhoven, Joyce Frye, Jennifer Jacobs, Menachem Oberbaum, Marie-France Bordet, Chaturbhuja Nayak, Gilles Chaufferin, John A Ives, Flávio Dantas, Peter Fisher

Abstract

Background: A method for assessing the model validity of randomised controlled trials of homeopathy is needed. To date, only conventional standards for assessing intrinsic bias (internal validity) of trials have been invoked, with little recognition of the special characteristics of homeopathy. We aimed to identify relevant judgmental domains to use in assessing the model validity of homeopathic treatment (MVHT). We define MVHT as the extent to which a homeopathic intervention and the main measure of its outcome, as implemented in a randomised controlled trial (RCT), reflect 'state-of-the-art' homeopathic practice.

Methods: Using an iterative process, an international group of experts developed a set of six judgmental domains, with associated descriptive criteria. The domains address: (I) the rationale for the choice of the particular homeopathic intervention; (II) the homeopathic principles reflected in the intervention; (III) the extent of homeopathic practitioner input; (IV) the nature of the main outcome measure; (V) the capability of the main outcome measure to detect change; (VI) the length of follow-up to the endpoint of the study. Six papers reporting RCTs of homeopathy of varying design were randomly selected from the literature. A standard form was used to record each assessor's independent response per domain, using the optional verdicts 'Yes', 'Unclear', 'No'. Concordance among the eight verdicts per domain, across all six papers, was evaluated using the kappa (κ) statistic.

Results: The six judgmental domains enabled MVHT to be assessed with 'fair' to 'almost perfect' concordance in each case. For the six RCTs examined, the method allowed MVHT to be classified overall as 'acceptable' in three, 'unclear' in two, and 'inadequate' in one.

Conclusion: Future systematic reviews of RCTs in homeopathy should adopt the MVHT method as part of a complete appraisal of trial validity.

References

    1. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses. Lancet. 1999;354:1896–1900. doi: 10.1016/S0140-6736(99)04149-5.
    1. Moher D, Liberati A, Tetzlaff J, Altman DG. PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006–1012. doi: 10.1016/j.jclinepi.2009.06.005.
    1. Higgins JPT, Altman DG, Sterne JAC, editor. The Cochrane Collaboration. 2011. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]
    1. Bornhöft G, Maxion-Bergemann S, Wolf U, Kienle GS, Michalsen A, Vollmar HC, Gilbertson S, Matthiessen PF. Checklist for the qualitative evaluation of clinical studies with particular focus on external validity and model validity. BMC Med Res Methodol. 2006;6:56. doi: 10.1186/1471-2288-6-56.
    1. Akobeng AK. Assessing the validity of clinical trials. J Pediatr Gastroenterol Nutr. 2008;47:277–282. doi: 10.1097/MPG.0b013e31816c749f.
    1. Jüni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ. 2001;323:42–46. doi: 10.1136/bmj.323.7303.42.
    1. Verhoef MJ, Lewith G, Ritenbaugh C, Boon H, Fleishman S, Leis A. Complementary and alternative whole systems research: Beyond identification of inadequacies of the RCT. Complement Ther Med. 2005;13:206–212. doi: 10.1016/j.ctim.2005.05.001.
    1. Wein C. Qualitätsaspekte klinischer Studien zur Homöopathie [Quality aspects of clinical trials of homeopathy] Essen: KVC Verlag; 2002.
    1. Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, Jonas WB. Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet. 1997;350:834–843. doi: 10.1016/S0140-6736(97)02293-9.
    1. Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005;366:726–732. doi: 10.1016/S0140-6736(05)67177-2.
    1. Jonas WB, Anderson RL, Crawford CC, Lyons JS. A systematic review of the quality of homeopathic clinical trials. BMC Complement Altern Med. 2001;1:12. doi: 10.1186/1472-6882-1-12.
    1. Thompson TDB, Weiss M. Homeopathy - what are the active ingredients? An exploratory study using the UK Medical Research Council's framework for the evaluation of complex interventions. BMC Complement Altern Med. 2006;6:37. doi: 10.1186/1472-6882-6-37.
    1. Bergmann J, Luft B, Boehmann S, Runnebaum B, Gerhard I. Die Wirksamkeit des Komplexmittels Phyto-Hypophyson® L bei weiblicher, hormonell bedingter Sterilität [The efficacy of the complex medication Phyto-Hypophyson® L in female, hormone-related sterility] Forsch Komplementärmed Klass Naturheilkd. 2000;7:190–199. doi: 10.1159/000021343.
    1. Kraft K. Commentary: a questionably positive effect of a homoeopathic complex medication in female infertility. Focus Altern Complement Ther. 2001;6:23.
    1. White A, Slade P, Hunt C, Hart A, Ernst E. Individualised homeopathy as an adjunct in the treatment of childhood asthma: a randomised placebo controlled trial. Thorax. 2003;58:317–321. doi: 10.1136/thorax.58.4.317.
    1. Brien SB, Lewith G. Letter to the Editor: Homeopathy in childhood asthma. Thorax. 2003;58:626–627.
    1. Kim LS, Riedlinger JE, Baldwin CM, Hilli L, Khalsa SV, Messer SA, Waters RF. Treatment of seasonal allergic rhinitis using homeopathic preparation of common allergens in the southwest region of the US: a randomized, controlled clinical trial. Ann Pharmacother. 2005;39:617–624. doi: 10.1345/aph.1E387.
    1. O'Mathúna DP, Horgan JM. Seasonal allergic rhinitis study results of questionable relevance to homeopathy. Ann Pharmacother. 2005;39:736–738. doi: 10.1345/aph.1G095.
    1. Dean ME, Coulter MK, Fisher P, Jobst K, Walach H. Reporting data on homeopathic treatments (RedHot): a supplement to CONSORT. Homeopathy. 2007;96:42–45. doi: 10.1016/j.homp.2006.11.006.
    1. MacPherson H, White A, Cummings M, Jobst K, Rose K, Niemtzow R. Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations. Complement Ther Med. 2001;9:246–249. doi: 10.1054/ctim.2001.0488.
    1. Schulz KF, Altman DG, Moher D. CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: 10.1136/bmj.c332.
    1. Bornhöft G, Wolf U, Ammon K, Righetti M, Maxion-Bergemann S, Baumgartner S, Thurneysen A, Matthiessen PF. Effectiveness, safety and cost-effectiveness of homeopathy in general practice - summarized health technology assessment. Forsch Komplementmed. 2006;13(Suppl 2):19–29.
    1. Ferley JP, Zmirou D, D'Adhemar D, Balducci F. A controlled evaluation of a homoeopathic preparation in the treatment of influenza like syndromes. Br J Clin Pharmacol. 1989;27:329–335.
    1. Jacobs J, Jimenez LM, Gloyds SS, Gale JL, Crothers D. Treatment of acute childhood diarrhea with homeopathic medicine; a randomized clinical trial in Nicaragua. Pediatrics. 1994;93:719–725.
    1. Schmidt JM, Ostermayr B. Does a homeopathic ultramolecular dilution of Thyroidinum 30cH affect the rate of body weight reduction in fasting patients? A randomized placebo-controlled double-blind clinical trial. Homeopathy. 2002;91:197–206. doi: 10.1054/homp.2002.0049.
    1. Lewith G, Watkins AD, Hyland ME. Use of ultramolecular potencies of allergen to treat asthmatic people allergic to house dust mite: double blind randomized controlled clinical trial. BMJ. 2002;324:520–523. doi: 10.1136/bmj.324.7336.520.
    1. Kainz JT, Kozel G, Haidvogl M, Smolle J. Homoeopathic versus placebo therapy of children with warts on the hands: a randomized, double-blind clinical trial. Dermatology. 1996;193:318–320. doi: 10.1159/000246277.
    1. McCutcheon LE. Treatment of anxiety with a homeopathic remedy. J Appl Nutr. 1996;48:2–6.
    1. Bell I, Lewis D, Brooks A, Schwartz G, Lewis S, Walsh B, Baldwin C. Improved clinical status in fibromyalgia patients treated with individualized homeopathic remedies versus placebo. Rheumatology (Oxford) 2004;43:577–582. doi: 10.1093/rheumatology/keh111.
    1. Robertson A, Suryanarayanan R, Banerjee A. Homeopathic Arnica montana for post-tonsillectomy analgesia: a randomised placebo control trial. Homeopathy. 2007;96:17–21. doi: 10.1016/j.homp.2006.10.005.
    1. Adler UC, Paiva NM, Cesar AT, Adler MS, Molina A, Padula AE, Calil HM. Homeopathic individualized Q-potencies versus fluoxetine for moderate to severe depression: double-blind, randomized non-inferiority trial. Evid Based Complement Alternat Med. 2009. in press Epub ahead of print.
    1. World Health Organization. Towards a Common Language for Functioning, Disability and Health. ICF: The International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2002.
    1. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–382.
    1. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310.
    1. Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Saida JK, Klassen TP. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009;339:b4012. doi: 10.1136/bmj.b4012.
    1. Jonas WB, Crawford CC. Commentary: there is no clinical benefit of homoeopathic isopathy over placebo in asthma. Focus Altern Complement Ther. 2002;7:263.
    1. Bell IR, Koithan M, Gorman MM, Baldwin CM. Homeopathic practitioner views of changes in patients undergoing constitutional treatment for chronic disease. J Altern Complement Med. 2003;9:39–50. doi: 10.1089/107555303321222937.
    1. Bell IR, Koithan M, Pincus D. Methodological implications of nonlinear dynamical systems models for whole systems of complementary and alternative medicine. Forsch Komplementmed. 2012;19(suppl 1):15–21.

Source: PubMed

赞助者和合作者

医疗条件

药物干预

3
订阅