The Program Sustainability Assessment Tool: a new instrument for public health programs

Douglas A Luke, Annaliese Calhoun, Christopher B Robichaux, Michael B Elliott, Sarah Moreland-Russell, Douglas A Luke, Annaliese Calhoun, Christopher B Robichaux, Michael B Elliott, Sarah Moreland-Russell

Abstract

Introduction: Public health programs can deliver benefits only if they are able to sustain programs, policies, and activities over time. Although numerous sustainability frameworks and models have been developed, there are almost no assessment tools that have demonstrated reliability or validity or have been widely disseminated. We present the Program Sustainability Assessment Tool (PSAT), a new and reliable instrument for assessing the capacity for program sustainability of various public health and other programs.

Methods: A measurement development study was conducted to assess the reliability of the PSAT. Program managers and staff (n = 592) representing 252 public health programs used the PSAT to rate the sustainability of their program. State and community-level programs participated, representing 4 types of chronic disease programs: tobacco control, diabetes, obesity prevention, and oral health.

Results: The final version of the PSAT contains 40 items, spread across 8 sustainability domains, with 5 items per domain. Confirmatory factor analysis shows good fit of the data with the 8 sustainability domains. The subscales have excellent internal consistency; the average Cronbach's α is 0.88, ranging from 0.79 to 0.92. Preliminary validation analyses suggest that PSAT scores are related to important program and organizational characteristics.

Conclusion: The PSAT is a new and reliable assessment instrument that can be used to measure a public health program's capacity for sustainability. The tool is designed to be used by researchers, evaluators, program managers, and staff for large and small public health programs.

Figures

Figure 1
Figure 1
Density plot (frequency) of the variability of Program Sustainability Assessment Tool (PSAT) scores across 252 public health programs participating in tests of the PSAT.
Figure 2
Figure 2
Program Sustainability Assessment Tool domain scores by level of program and type of program among programs participating in tests of the tool.

References

    1. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009;36(1):24–34. 10.1007/s10488-008-0197-4
    1. Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York (NY): Oxford University Press; 2012.
    1. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health 2011;101(11):2059–67. 10.2105/AJPH.2011.300193
    1. Schell SF, Luke DA, Schooley MW, Elliott MB, Herbers SH, Mueller NB, et al. Public health program capacity for sustainability: a new framework. Implement Sci 2013;8:15.
    1. Hutchinson K. Literature review of program sustainability assessment tools. Vancouver (BC): The Capture Project, Simon Fraser University; 2010. . Accessed January 9, 2013.
    1. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation 2005;26(3):320–47. 10.1177/1098214005278752
    1. Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res 1998;13(1):87–108. 10.1093/her/13.1.87
    1. Evashwick C, Ory M. Organizational characteristics of successful innovative health care programs sustained over time. Fam Community Health 2003;26(3):177–93. 10.1097/00003727-200307000-00003
    1. Sustainability Framework. Atlanta (GA): Georgia Health Policy Center; 2011. . Accessed December 13, 2012.
    1. Trochim WM. An introduction to concept mapping for planning and evaluation. Eval Program Plann 1989;12(1):1–16. 10.1016/0149-7189(89)90016-5
    1. Rosseel Y. Lavaan: an R package for structural equation modeling. J Stat Softw 2012;48(2):1–36. . Accessed July 30, 2013.
    1. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess 1995;7(3):286–99. 10.1037/1040-3590.7.3.286
    1. Luke DA, Ribisi KM, Walton MA, Davidson WS. Assessing the diversity of personal beliefs about addiction: development of the Addiction Belief Inventory. Subst Use Misuse 2002;37(1):89–120. 10.1081/JA-120001498
    1. Brown T. Confirmatory factor analysis for applied research. New York (NY): Guilford Press: 2006.
    1. Hu LT, Bentler PM. Evaluating model fit. In: Hoyle RH, editor. Structural equation modeling: concepts, issues, and applications. Thousand Oaks (CA): Sage Publications; 1995. p 76–99.
    1. Hu LT, Bentler PM. Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychol Methods 1998;3(4):424–53. 10.1037/1082-989X.3.4.424
    1. Kenny DA, McCoach DB. Effect of the number of variables on measures of fit in structural equation modeling. Struct Equ Modeling 2003;10(3):333–51. 10.1207/S15328007SEM1003_1
    1. Schmitt N. Uses and abuses of coefficient alpha. Psychol Assess 1996;8(4):350–3. 10.1037/1040-3590.8.4.350
    1. Calhoun A, Mainor A, Moreland-Russell S, Maier RC, Brossart L, Luke DA. Using the Program Sustainability Assessment Tool to assess and plan for sustainability. Prev Chronic Dis 2014;11:130185. 10.5888/pcd11.130185
    1. Best A, Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-system transformation in health care: a realist review. Milbank Q 2012;90(3):421–56. 10.1111/j.1468-0009.2012.00670.x
    1. Scheirer MA. Linking sustainability research to intervention types. Am J Public Health 2013;103(4):e73–80. 10.2105/AJPH.2012.300976

Source: PubMed

Sponzoři a spolupracovníci

Zdravotní podmínky

Drogové intervence

3
Předplatit