Introduction
Healthy marriage and relationship education (HMRE) services are designed to help participants build and sustain strong families. HMRE services for youth between the ages of 14 and 24 focus on preparing participants for positive, healthy relationships in adulthood and educating them about the social and emotional aspects of relationships (Alamillo et al. 2021; Simpson et al. 2018). Studies have generally found positive impacts on short-term outcomes related to youths’ relationship attitudes and beliefs. To date, however, little evidence has emerged on the effects HMRE services for youth have on longer-term outcomes (Alamillo et al. 2021; Simpson et al. 2018). To achieve their intended effects, HMRE service providers might need support to address key implementation challenges related to recruitment, retention, and content engagement.
The Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF), with funding from the Office of Family Assistance (OFA), contracted with Mathematica and its partner, Public Strategies, to conduct the Strengthening the Implementation of Marriage and Relationship Education Programs (SIMR) project. This project aims to identify key implementation challenges facing HMRE grantees and, in close collaboration with HMRE grantees and their staff, develop and test strategies to address those challenges using rapid cycle learning techniques. This report shares lessons and insights from the testing phase of the project, focusing on the five youth-serving HMRE grantees that participated in SIMR.
Purpose
In the SIMR project, Mathematica and its partner, Public Strategies, collaborated with 10 HMRE grantees—five youth-serving grantees and five adult-serving grantees—to conduct iterative rapid cycle testing aimed at strengthening their services. SIMR focused on common implementation challenges related to recruitment, retention, and content engagement. SIMR had two main goals:
- to improve the service delivery of these grantees
- to develop lessons for the broader HMRE field about promising practices for addressing common implementation challenges.
This report describes the rapid cycle learning process and findings for the five youth-serving HMRE grantees that participated in SIMR. It shares how each grantee addressed implementation challenges and improved services through participation in SIMR and insights that can help other HMRE grantees strengthen their own service delivery.
Key Findings and Highlights
In SIMR, each grantee developed and tested improvement strategies tailored to their specific needs, service populations, and individual contexts, using an approach to program improvement and rapid cycle learning known as Learn, Innovate, Improve (LI2). Through their work with the SIMR team, grantees:
- Addressed pressing implementation challenges: One grantee focused on improving recruitment and four focused on topics related to improving content engagement
- Increased their capacity to collect and use data to inform decision-making: Through rapid cycle learning, grantee staff administered feedback surveys to participants, tracked recruitment data, and analyzed social media analytics. They reviewed these data with the SIMR team and developed insights to refine their improvement strategies.
- Developed skills for identifying and responding to emerging implementation challenges: At the end of each learning cycle, the SIMR team met with grantees to review and interpret data and determine next steps. When new challenges emerged, grantees were able to pivot to address them in later learning cycles.
- Strengthened capacity and developed tools and strategies to support strong implementation through the rest of the grant period: Grantees developed promising tools and strategies to support facilitators, enhance case management, recruit youth from rural areas, and encourage participant relationships. At the end of SIMR, the grantees planned to continue using these tools and strategies.
Methods
Grantees and the SIMR team used the LI2 framework to guide rapid cycle learning. LI2 is an analytic and evidence-based approach to managing program improvement (Derr et al. 2017). Throughout the three phases of LI2, researchers collaborate with practitioners to identify the root causes of a challenge (Learn); create innovative program improvement strategies that are participant-centered, informed by science, and sustainable (Innovate); and use rapid cycle learning methods to test and refine strategies (Improve).
Youth-serving grantees conducted between two and four learning cycles in SIMR. They collected different types of data to assess the success of the strategies they were testing, including interviews, focus groups, and surveys of staff and participants, workshop observations, program data, and data from nFORM (Information, Family Outcomes, Reporting, and Management), the management information system sponsored by ACF that grantees use to record participants’ characteristics and participation in services, monitor service use, and make decisions that are informed by data.
Recommendations
Through their collaboration as part of SIMR rapid cycle learning, the SIMR team and the five READY4Life grantees that participated generated insights and lessons to inform strong service delivery that are relevant to other HMRE grantees. The tools and strategies that grantees developed provide starting points for other organizations that want to strengthen their own HMRE services. Grantees interested in adopting any strategies presented in this report can do so using a continuous quality improvement (CQI) process to adapt the strategy to their specific context and then iteratively test it on a small scale to refine the strategy design and implementation:
- Provide supports and tools for facilitators to successfully lead HMRE workshops, such as strategies to manage sources of stress and plan and debrief lessons.
- Look for innovative ways to reinforce workshop content, such as by engaging participants with curriculum content on social media and helping youth set and make progress on goals through case management.
- Prioritize relationship-building to engage participants, by establishing a safe and supportive classroom environment and using technology in intentional, innovative ways.
Citation
Baumgartner, Scott, Caroline O’Callahan, Hannah McInerney, Heather Gordon, Katie Hunter, Annie Buonaspina, Daniel Friend, and Robert G. Wood (2023). “Conducting Rapid Cycle Learning with Healthy Marriage and Relationship Education Programs for Youth: Findings from the Strengthening the implementation of Marriage Programs (SIMR) Project.” OPRE Report #2023-034 . Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- HMRE:
- Healthy Marriage and Relationship Education
- Learning cycle:
- One of the short, iterative testing periods involved in rapid cycle learning. Each learning cycle includes a period when grantee staff implement a program improvement strategy and participate in data collection, followed by a period in which grantee staff and researchers review data and determine changes to the strategy for the next learning cycle.
- LI^2:
- Learn, Innovate, Improve, the framework the SIMR team used to guide rapid cycle learning.
- nFORM:
- Information, Family Outcomes, Reporting, and Management, a management information system sponsored by ACF that grantees use to record participants’ characteristics and participation in services, monitor service use, and make decisions that are informed by data.
- Rapid cycle learning:
- An iterative process in which data on short-term outcomes are collected and used to implement and repeatedly refine a strategy until co-created goals are met.
- SIMR:
- The Strengthening the Implementation of Marriage and Relationship Programs project.
- SIMR team:
- Mathematica and Public Strategies staff who worked closely with grantee staff to develop and test program improvement strategies.
- Youth:
- The READY4Life grant defines a youth as anyone between the ages of 14 and 24. Grantees in SIMR focused rapid cycle learning either on services for youth in high schools (typically younger than 18) or services for older youth in community settings (typically between the ages of 18 and 24).