Commentary on “Development and application of the RE-AIM QuEST mixed methods framework for program evaluation.”
Commentary: In their recent Preventive Medicine Reports article, Jane Forman and her colleagues extend RE-AIM, one of the most widely used frameworks in implementation science, by integrating qualitative assessments into the framework. Glasgow and colleagues 1 developed RE-AIM to aid researchers in assessing key dimensions of the application of interventions in practice: Reach, Effectiveness, Adoption, Implementation and Maintenance. In its original conception, RE-AIM included quantitative measures to retrospectively evaluate these dimensions. For example, an intervention’s reach might be assessed by identifying the number of eligible patients receiving the intervention. Researchers have largely adhered to these quantitative measures: A systematic review of studies using RE-AIM found that 69 percent were quantitative (30 percent were mixed method, and one study used qualitative methods). 2 The quantitative measures originally proposed for RE-AIM studies allowed researchers to understand whether an intervention reached patients or providers, was effective, adopted, implemented or maintained, and the extent to which this varied across contexts; however, the quantitative measures do not allow researchers to understand why or how these outcomes were achieved. Recognizing this, Kessler and colleagues 3recommended that “fully developed use” of RE-AIM should include qualitative methods.
To address the need for qualitative methods to understand why and how interventions achieve RE-AIM outcomes, Forman and colleagues propose RE-AIM QuEST mixed methods framework for program evaluation, an extension of RE-AIM. The authors propose generic, open-ended questions to complement quantitative measures of each RE-AIM outcome. They then demonstrate QuEST’s application to a specific intervention, adapting the open-ended questions to formatively and retrospectively evaluate the implementation of the Adherence and Intensification of Medications program, a pragmatic cluster randomized controlled trial intended to improve patient adherence to blood pressure medication.
Forman and colleagues identify several benefits of QuEST, including its ability to understand why and how RE-AIM outcomes are achieved, to formatively and retrospectively evaluate implementation, and to explain the influence of context on implementation. The authors acknowledge that this deeper understanding allows researchers to address barriers in real-time and may help to inform intervention design. QuEST has broader impact than its developers acknowledge. By expanding RE-AIM to incorporate qualitative methods, Forman and colleagues make two substantial contributions to the field of implementation science. First, in developing QuEST, Forman and colleagues make an existing implementation framework more useful. With QuEST in hand, researchers can use RE-AIM to answer questions about whether as well as why and how implementation outcomes are achieved. By expanding research opportunities with an existing implementation framework, Forman and colleagues combat the proliferation of theories and frameworks in the field. 4-5 Second, the article offers guidance for the application of QuEST. As described in our Resource of the Month, guidance for the application of theories and frameworks in implementation science has been limited to date. Efforts such as those of Forman and colleagues and Atkins and colleagues contribute to a substantial need in the field. Guidance for the application of theories and frameworks in implementation science has the potential to promote more consistent and appropriate use of theories and frameworks in the field. Future efforts should build upon these promising efforts.
Read the abstract.
1Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the re-aim framework. Am J Public Health. 1999;89:1322–7. doi: 10.2105/AJPH.89.9.1322.
2 Harden S.M., Gaglio B., Shoup J.A. Fidelity to and comparative results across behavioral interventions evaluated through the RE-AIM framework: a systematic review. Syst. Rev. 2015;4:155.
3Kessler R.S., Purcell E.P., Glasgow R.E., Klesges L.M., Benkeser R.M., Peek C.J. What does it mean to “employ” the RE-AIM model? Eval. Health Prof. 2013;36(1):44–66.
4 Noar SM, Zimmerman RS. Health Behavior Theory and cumulative knowledge regarding health behaviors: are we moving in the right direction? Health Educ Res. 2005;20:275–90.
5 Walshe K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. Int J Qual Health Care. 2009;21:153–9.