Commentary on “Structuring Process Evaluation To Forecast Use And Sustainability Of An Intervention: Theory And Data From The Efficacy Trial For Lunch Is In The Bag”

Commentary: In a recent issue of Health Education and Behavior, Roberts-Gray and colleagues published findings about an innovative approach to structuring process evaluations for predicting sustained use of interventions using a case-study of the Lunch is in the Bag intervention.¹ This intervention is designed to increase healthy foods in parent-packed lunches consumed by children at early childhood care and education centers. The investigators used the Fit-or-Fix model to guide implementation, and the process evaluation included several aims at different phases of implementation. Before launching the Lunch is in the Bag efficacy trial, the investigators used a readiness assessment and simulations of user experience with the intervention to adjust the intervention and implementation strategy prior to launch of the efficacy trial. They also used this information to forecast use of the intervention by sites. During the trial, investigators used several process evaluation instruments (e.g., logs, observations, questionnaires) to continue to tweak the forecast of sustained use. Finally, investigators used debriefing interviews post-trial to assess readiness of the intervention for wider dissemination and implementation. The pre-trial assessment predicted only a modest likelihood of sustained intervention use and this forecast was borne out by no sustained use at any sites post-trial. Although the lack of sustainment is a disappointing implementation finding, the ability to accurately predict sustained use with a relatively low pre-trial effort is a useful finding. However, it raises interesting dilemmas about what to do with forecasts that predict a low likelihood of sustainment. On the one hand, a forecast that suggests a low likelihood of sustainment could direct investigators to rethink the design or implementation strategy before proceeding with efficacy testing. After all, an effective intervention that can’t be sustained in typical settings may not be a good investment of limited research resources. On the other hand, important information may still be gleaned from implementing interventions with a low likelihood of sustainment, particularly when it can inform iterative adaptations of the intervention that might lead to a higher likelihood of sustainment. In conclusion, this evaluation by Roberts-Gray and colleagues is an excellent reminder for intervention developers to consider feasibility when designing interventions and to collect robust data to characterize the implementation continuum from adoption to sustainment.

¹ Roberts-Gray C, Sweitzer SJ, Ranjit N, Potratz C, Rood M, Romo-Palafox MJ, Byrd-Williams CE, Briley ME, Hoelscher DM. Structuring Process Evaluation to Forecast Use and Sustainability of an Intervention: Theory and Data From the Efficacy Trial for Lunch Is in the Bag. Health Educ Behav. 2016 Nov 17. pii:1090198116676470. [Epub ahead of print] PubMed PMID: 27864471.