Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool

Julia Abelson, Kathy Li, Geoff Wilson, Kristin Shields, Colleen Schneider, Sarah Boesveld, Julia Abelson, Kathy Li, Geoff Wilson, Kristin Shields, Colleen Schneider, Sarah Boesveld

Abstract

Objectives: Only rudimentary tools exist to support health system organizations to evaluate their public and patient engagement (PPE) activities. This study responds to this gap by developing a generic evaluation tool for use in a wide range of organizations.

Methods: The evaluation tool was developed through an iterative, collaborative process informed by a review of published and grey literature and with the input of Canadian PPE researchers and practitioners. Over a 3-year period, structured e-mail, telephone and face-to-face exchanges, including a modified Delphi process, were used to produce an evaluation tool that includes core principles of high-quality engagement, expected outcomes for each principle and three unique evaluation questionnaires that were tested and revised with input from 65 end users.

Results: The tool is structured around four core principles of 'quality engagement': (i) integrity of design and process; (ii) influence and impact; (iii) participatory culture; and (iv) collaboration and common purpose. Three unique questionnaires were developed to assess each of these four evaluation domains from the following perspectives: (i) those who participate in PPE activities; (ii) those who plan, execute or sponsor PPE activities within organizations; and (iii) those who provide the leadership and capacity for PPE within their organizations.

Conclusions: This is the first known collaboration of researchers and practitioners in the co-design of a comprehensive PPE evaluation tool aimed at three distinct respondent groups and for use in a wide range of health system organization settings.

Keywords: public and patient engagement evaluation; public and patient engagement in health system decision making; public and patient involvement.

© 2015 The Authors. Health Expectations Published by John Wiley & Sons Ltd.

References

    1. Rowe G, Frewer LJ. A typology of public engagement mechanisms. Science, Technology & Human Values, 2005; 30: 251–290.
    1. Tambuyzer E, Pieters G, van Audenhove C. Patient involvement in mental health care: one size does not fit all. Health Expectations, 2011; 17: 138–150.
    1. Christiaens W, Kohn L, Léonard C, Denis A, Daue F, Cleemput I. Models for Citizen and Patient Involvement in Health Care Policy – Part I: Exploration of Their Feasibility and Acceptability. KCE Report 195C. Brussels: Belgian Health Care Knowledge Centre, 2012.
    1. Carman KL, Dardess P, Maurer M et al Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Affairs, 2013; 32: 223–231.
    1. Sarrami‐Foroushani P, Travaglia J, Debono D, Braithwaite J. Implementing strategies in consumer and community engagement in health care: results of a large‐scale, scoping meta‐review. BMC Health Services Research, 2014; 14: 402.
    1. Crawford MJ, Rutter D, Manley C et al Systematic review of involving patients in the planning and development of health care. British Medical Journal, 2002; 325: 1263–1265.
    1. OECD . Evaluating Public Participation in Policy Making. France: OECD Publications, 2005. Available at: , accessed 21 January 2015.
    1. Mitton C, Smith N, Peacock S, Evoy B, Abelson J. Public participation in health care priority setting: a scoping review. Health Policy, 2009; 91: 219–228.
    1. Conklin A, Morris Z, Nolte E. What is the evidence base for public involvement in health‐care policy? Results of a systematic scoping review. Health Expectations, 2012; 18: 153–165. doi:.
    1. Mockford C, Staniszewska S, Griffiths F, Herron‐Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. International Journal for Quality in Health Care, 2012; 24: 28–38.
    1. Brett J, Staniszewska S, Mockford C et al Mapping the impact of patient and public involvement in health and social care research: a systematic review. Health Expectations, 2012; 17: 637–650. doi:.
    1. Staley K, Buckland S, Hayes H, Tarpey M. The missing links: understanding how context and mechanism influence the impact of public involvement in research. Health Expectations., 2012; 17: 755–764. doi: .
    1. Popay J, Collins M, (editors) with the PiiAF Study Group . The Public Involvement Impact Assessment Framework Guidance. Universities of Lancaster, Liverpool and Exeter 2014. Available at: , accessed 10 April 2015.
    1. Evans D, Coad J, Cottrell K et al Public involvement in research: assessing impact through a realist evaluation. Health Services and Delivery Research, 2014; 2. doi:.
    1. Rowe G, Frewer LJ. Public participation methods: a framework for evaluation. Science, Technology & Human Values, 2000; 25: 3–29.
    1. Rowe G, Frewer LJ. Evaluating public participation exercises: a research agenda. Science, Technology & Human Values, 2004; 29: 512–556.
    1. Abelson J, Gauvin F‐P. Assessing the impacts of public participation: Concepts, evidence and policy implications. Research Report P06 Ontario: Canadian Policy Research Networks, 2006. Available at: , accessed 21 January 2015.
    1. Motsi G. Evaluating citizen engagement in policy making. Paper presented at: Canadian Evaluation Society Conference; 2009 June 2; Ottawa, Ontario. [Place unknown]: Institute on Governance, 2009. Available at: , accessed 21 January 2015.
    1. Burton P. Conceptual, theoretical and practical issues in measuring the benefits of public participation. Evaluation, 2009; 15: 264–284.
    1. Hendricks A. Evaluation Framework for Community Engagement Based on the United Nations Brisbane Declaration. Australia: Darzin Sortware, 2009. Available at: , accessed 21 January 2015.
    1. South J, Fairfax P, Green E. Developing an assessment tool for evaluating community involvement. Health Expectations, 2005; 8: 64–73.
    1. Jabbar AM, Abelson J. Development of a framework for effective community engagement in Ontario, Canada. Health Policy, 2011; 101: 59–69.
    1. Mass LBP. Engaging with Impact: Targets for Successful Community Engagement by Ontario's Local Health Integration Networks. Ontario: MASS LBP, 2009. Available at: , accessed 21 January 2015.
    1. Government of British Columbia . Primary Health Care Charter: A Collaborative Approach. British Columbia: Government of British Columbia, 2007. Available at: , accessed 2 February 2015.
    1. Government of Ontario . Service Ontario e‐Laws: Local Health System Integration Act. Ontario: Government of Ontario, 2006. Available at: , accessed 21 January 2015.
    1. Government of Quebec . An Act respecting the Health and Welfare Commissioner. Quebec: Official Publisher of Quebec, 2005. Available at: , accessed 21 January 2015.
    1. Government of Ontario . LHIN community Engagement Guidelines and Toolkit. Ontario: Ministry of Health and Long‐term Care, 2011. Available at: , accessed 21 January 2015.
    1. Vancouver Coastal Health . Community Engagement Framework. Community Engagement. Vancouver: Vancouver Coastal Health, 2011. Available at: , accessed 21 January 2015.
    1. Capital Health . Engagement Framework and Toolkit. Nova Scotia: Capital District Health Authority; 2011. Available at: , accessed 21 January 2015. [note: current web version has been updated as of 2014].
    1. Abelson J, Montessanti S, Li K , Gauvin F‐P , Martin E. Effective strategies for interactive public engagement in the development of healthcare policies and programs. Ontario: Canadian Health Services Research Foundation, 2010. Available at: , accessed 21 January 2015.
    1. Webler T. “Right” Discourse in Citizen Participation: An Evaluative Yardstick. Boston: Kluwer Academic Press, 1995.
    1. Nabatchi T. A Manager's guide to evaluating citizen participation. IBM Centre for the Business Government: Fostering Transparency and Democracy Series, 2012. [cited 2015 Jan 21]. Available at: , accessed 21 January 2015.
    1. Capital Health . Involving Patients and Citizens: A Review of Capital Health's Engagement Policy Achievements, 2010–2014. 2014. Available at: ? accessed 6 February 2015.
    1. Vancouver Coastal Health . What do you know about Patient and Public Engagement work at VCH?, 2014. Available at: , accessed 6 February 2015.

Source: PubMed

Patrocinadores e Colaboradores

Condições médicas

Intervenções de drogas

3
Se inscrever