Continuing the Conversation on Rigor, Relevance, and Rapidity – January 2020 – CIS Associate Editor Commentary
Following the 12th Annual Conference on the Science of Dissemination and Implementation in Health hosted in Arlington, VA last month, several of this month’s featured articles move forward the conference theme of rigor, relevance, and rapidity. This theme was, in my opinion, a refreshing call to action to deeply examine what matters in the field of implementation, challenging us to re-consider what we are focusing on, how, and why.
It was with that lens that I viewed many of the articles featured in this month’s newsletter. Several of the articles focused on furthering methods (MOST trials1 and systematic reviews2), while others focused on the reality of getting things done (applying RE-AIM framework in the “real world”3 and practical methods for identifying core functions and forms of interventions4). Two of the featured articles placed an emphasis on rigor,1-2 while others focused on relevance.3-4
Gallis et al. discuss detailed recommendations for randomization in MOST trials, or multiphase optimization strategy, an engineering-inspired framework for quickly identifying and optimizing which intervention components are most effective via factorial designs. MOST trials also allow efficiencies in study design, allowing researchers to test many components efficiently and save on sample size. In this sense, the MOST design balances rapidity, rigor, and relevance, as it promotes rigorous methods for identifying “what works” in a way that is quicker than a traditional two-arm randomized controlled trial (rapidity) and gets us to learning what matters (relevance) faster. Tierney et al. present a feasibility study on integrating implementation outcomes into evidence synthesis or systematic review projects. The authors note that typical reviews focus on “effectiveness outcomes,” such as clinical/health services outcomes, but often leave out implementation outcomes. To address this gap, Tierney et al. conducted an initial feasibility study where they attempted to extract data on 10 implementation outcomes from a sample of 29 physician communication skills intervention studies, balancing priorities of rigor (integrating implementation outcomes into syntheses) with relevance (is this feasible to do?).
Finally, Kwan et al. and Kirk et al. had a large focus on relevance with their research on how things work in practice. Kwan et al. investigated how researchers are using the RE-AIM framework in the “real world,” asking researchers which components of the RE-AIM framework are most often used, rating its understandability and usefulness, and discussing benefits and challenges. Kirk et al. present practical methods for identifying core functions and forms of interventions post hoc, as core functions are often left unspecified by intervention developers. Both of these articles focus on relevance: proposing creative methods to apply some rigor within the messy reality of our work (Kirk et al.) and underscoring the importance of circling back to key frameworks (or methods and measures) to formally assess their applicability and relevance in the real world (Kwan et al.).
As we move into 2020, I challenge each of you to carry forward the emphasis on rigor, relevance, and rapidity, and how to balance them in our research and practice. Although some methods, questions, and models may lend themselves more to one than the other, all three are needed if we are going to be successful in truly re-visiting what matters in the field of implementation and meaningfully reconsidering what we are focusing on, how, and why.
1. Gallis J. A. B. G. G. Steinberg D. M. Askew S. Turner E. L. (2019). Randomization procedures for multicomponent behavioral intervention factorial trials in the multiphase optimization strategy framework: challenges and recommendations. Transl Behav Med96) 1047-1056. doi:10.1093/tbm/iby131
2. Tierney A. Haverfield C. McGovern P. Zulman M. (2019). Advancing Evidence Synthesis from Effectiveness to Implementation: Integration of Implementation Measures into Evidence Reviews. J Gen Intern Med. doi:10.1007/s11606-019-05586-3
3. Kwan B. M. M. H. L. Ory M. G. Estabrooks P. A. Waxmonsky J. A. Glasgow R. E. (2019). RE-AIM in the Real World: Use of the RE-AIM Framework for Program Planning and Evaluation in Clinical and Community Settings. Front Public Health7 345. doi:10.3389/fpubh.2019.00345
4. Kirk M. A. H. E. R. Rokoske F. S. Powell B. J. Weinberger M. Hanson L. C. Birken S. A. (2019). A case study of a theory-based method for identifying and reporting core functions and forms of evidence-based interventions. Transl Behav Med. doi:10.1093/tbm/ibz178