Generating meta-inferences for program improvement recommendations: mixed methods integration in joint displays for program evaluation

Public health research and program evaluation are increasingly relying on mixed methods approaches to tackle complex social and medical crises, such as the ongoing U.S. opioid epidemic. A recent methodological study published in Frontiers in Psychology highlights a critical evolution in how researchers process mixed qualitative and quantitative data. Titled "Generating meta-inferences for program improvement recommendations: mixed methods integration in joint displays for program evaluation," the paper is authored by L. Monkman, D. Ratnapradipa, S. Fàbregues, and Timothy C. Guetterman. The work introduces a framework to expand "meta-inferences"—the overarching conclusions drawn from merging qualitative and quantitative insights—to directly generate actionable program recommendations, moving beyond simple data validation.

The Context and Background of the Overdose Crisis

To demonstrate this methodological shift, the research team drew from an ongoing evaluation of a local overdose prevention program in Douglas County, Nebraska. This initiative is funded by the Centers for Disease Control and Prevention (CDC) through its flagship Overdose Data to Action (OD2A) program. Since its inception in 2019, the OD2A program has directed multi-year cooperative agreements to 90 state and local health departments nationwide, empowering jurisdictions to craft localized, data-responsive prevention and surveillance strategies.

A core tenet of the OD2A framework is breaking down historical silos between public health entities, healthcare providers, and public safety organizations. The initiative prioritizes linking individuals who use drugs to comprehensive care networks, including evidence-based substance use recovery treatments, naloxone distribution, harm reduction education, and essential wraparound services like stable housing and free transportation. In Douglas County, this mission is heavily anchored in a peer navigation model, utilizing certified personnel with lived experience of substance use disorders to guide vulnerable clients through recovery pathways.

Chronology of the Mixed Methods Evaluation

The evaluation was structured around a rigorous multi-stage mixed methods design that integrated quantitative performance metrics with qualitative stakeholder perspectives. The timeline of data collection and implementation unfolded over several phases:

• September 2024: The evaluation officially synchronized with the rollout of the OD2A local performance measure reporting tracker, mandating monthly quantitative data submissions from participating community organizations.
• September 2024 to March 2025: Quantitative tracking recorded baseline referral metrics, revealing significant gaps in specific care linkages—notably, only nine total referrals for medications for opioid use disorder (MOUD) across the initial months.
• April to June 2025: To allow sufficient time for program start-up and service delivery, the evaluation team administered investigator-developed surveys to 17 peer navigators across three partner organizations. These surveys focused on role history, training, self-confidence, burnout, and daily responsibilities.
• Mid-2025: Informed by the survey findings and quantitative tracking data, researchers launched semi-structured qualitative interviews with six peer navigators. These interviews probed daily workflows, systemic challenges, and resource limitations.
• Late 2025 to Early 2026: The team employed side-by-side joint displays to merge the datasets, collaborating closely with an Evaluation-Community Advisory Board (ECAB) to synthesize findings and draft practical recommendations.

Key Findings and Data Insights

The integration of quantitative and qualitative data illuminated complex operational realities on the ground. While survey results indicated that 76% of peer navigators routinely provided emotional support and 68% frequently assisted with basic needs like food and shelter, critical gaps emerged regarding specialized treatments.

Specifically, quantitative tracking revealed low referral rates for harm reduction resources and MOUD, such as methadone or buprenorphine. When qualitative interviews were integrated via joint displays, researchers uncovered the underlying causes: individual-level knowledge gaps, personal biases among navigators favoring abstinence-only recovery, and widespread role ambiguity. For example, while some navigators embraced comprehensive care coordination, others viewed case management as outside their professional scope. Furthermore, 71% of surveyed navigators expressed an acute need for specialized training to effectively support distinct demographic groups, particularly a growing local Spanish-speaking population and justice-involved individuals re-entering society.

Expanding the Framework of Meta-Inferences

Traditionally, mixed methods integration focuses on determining "fit"—assessing whether quantitative and qualitative results are concordant, discordant, or mutually expanding. However, the authors argue that stopping at fit limits the utility of applied research. Building upon recent typologies established by qualitative scholarship, which categorized meta-inferences into relational, predictive, causal, comparative, and elaborative forms, the study proposes adding a sixth category: program recommendation meta-inferences.

By utilizing visual joint displays to juxtapose side-by-side data points, the researchers demonstrated how evaluators can transition directly from academic observation to operational reform. For instance, combining low MOUD referral counts with qualitative accounts of navigator stigma led to a concrete recommendation: implement targeted educational modules for peer support specialists to demystify evidence-based medications and align practices with clinical realities.

Official Responses and Advisory Collaboration

To ensure the evaluation translated into real-world utility, the study relied on continuous collaboration with the Evaluation-Community Advisory Board. Comprising community members, local organization staff, and individuals with lived experience of substance abuse, the ECAB actively guided data interpretation and refined final program recommendations. Their involvement ensured that proposed interventions—such as developing culturally tailored training modules for Spanish-speaking clients and fostering inter-agency coordination meetings—reflected the authentic needs of the Douglas County community rather than top-down bureaucratic assumptions.

Broader Implications for Public Health and Methodology

The implications of this methodological advancement extend well beyond Nebraska’s public health sector. By formalizing program recommendation meta-inferences, researchers have been provided with a structured roadmap for making mixed methods evaluations immediately actionable for program managers and policymakers.

As health departments nationwide continue to combat the multi-faceted opioid crisis, bridging the gap between statistical performance metrics and the lived experiences of frontline workers remains paramount. This study underscores that when mixed methods integration is purposefully directed toward practical problem-solving, it can significantly enhance program fidelity, dismantle institutional silos, and ultimately save lives by optimizing systems of care.