Introduction
Research Questions
- Does Work Success affect participants’ outcomes related to pursuing their goals and developing other skills associated with labor market success?
- Does Work Success affect participants’ employment and economic security outcomes?
- How do the impacts of Work Success change over time?
- Is Work Success more effective for some groups of participants than others?
Many adults with low incomes face complex challenges that make it difficult for them to find and keep a job or advance in their careers. Coaching by trained staff as an approach to help them become economically secure has attracted increasing interest from policymakers and operators of employment programs.
Work Success is a novel program for adults with low incomes. It combines intensive group and individual coaching with workshops on topics such as resume writing, job search strategies, and interviewing skills.
The Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families, U.S. Department of Health and Human Services, is sponsoring an evaluation of Work Success as part of the Next Generation of Enhanced Employment Strategies (NextGen) Project. Mathematica is conducting this evaluation to explore the program’s implementation and its impact on the well-being of study participants. This report describes the design and analysis plan for the impact study of Work Success.
Purpose
This report describes the design of and analysis plans for the impact evaluation of Work Success. It discusses employment coaching and explains why Work Success is expected to affect participants’ employment and economic security. It details the process for conducting random assignment and the data sources. Finally, it describes the analytic approach to estimating the program’s impacts.
Key Findings and Highlights
The approach to research design and analysis will provide rigorous estimates of the effectiveness of Work Success, a coaching program for adults with low incomes.
Methods
The study uses an experimental research design to assess the effectiveness of Work Success in improving employment-related outcomes, economic security, self-regulation, and other measures of economic well-being. Eligible employment center customers who consented to participate in the study were randomly assigned either to a program group that was offered access to Work Success or a control group that was not offered access to Work Success. The control group was instead given a list of services available from the employment center and in the community. Both groups could access any other services available in the center or community. The effectiveness of Work Success will be assessed based on differences in the outcomes of members of the program and control groups.
Confirmatory outcomes—the key test of Work Success’s effectiveness—include skills in setting and attaining goals (a primary measure of self-regulation skills), earnings, and measures of economic hardship such as needing to go without food to pay bills. Outcomes will be measured at two points in time: six months after study enrollment and 24 months after study enrollment. The outcomes were collected from (1) two follow-up surveys of study participants, (2) administrative employment and unemployment insurance records from the National Directory of New Hires, and (3) administrative records pertaining to TANF and Supplemental Nutrition Assistance Program benefit receipt.
Citation
Wu, April Yanyuan, Owen Schochet, Kristen Joyce, and Nicardo McInnis (2023). “Design and Analysis Plan for the Impact Study of Work Success,” OPRE Report #2023-212, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- ACF:
- Administration for Children and Families
- OPRE:
- Office of Planning, Research, and Evaluation