Introduction
Research Questions
- What is the impact of HPOG 2.0 on receipt of training (in general and for healthcare professions) and support services, on employment and earnings (in general and in healthcare professions), and on broader measures of well-being?
- How does the impact of HPOG 2.0 vary with study members’ baseline characteristics (e.g., age, education, TANF receipt)?
- How does the impact of HPOG 2.0 vary by program and program features?
The Health Profession Opportunity Grants (HPOG) Program is designed to deliver high-quality education, training, and support services to eligible individuals for occupations in the healthcare field. The Office of Family Assistance of the Administration for Children and Families (ACF), within the U.S. Department of Health and Human Services, awarded a second round of 32 five-year grants (“HPOG 2.0”) in 2015. ACF’s Office of Planning, Research, and Evaluation (OPRE) funded an evaluation to assess the success of the HPOG 2.0 Program.
This Analysis Plan describes the methodology for answering the Impact Evaluation’s key research questions at the second follow-up point for HPOG participants. It serves as a guide for the statistical and programming staff conducting the analyses. The document also improves the transparency and replicability of study findings by committing the research team to make consequential decisions prior to inspecting estimates of program effects. This Analysis Plan has a primary focus on the operationalization of outcome measures and provides key details of the methods to be used in this work. It also provides additional details on questions that were left open in the overall design plan.
Purpose
The purpose of this report is to describe a plan for answering the Impact Evaluation’s key research questions using three-year outcomes. The Analysis Plan operationalizes key outcome measures, briefly describes key sources of data for the study, and provides details of the methods that will be used. By specifying these details in advance, this document serves as a public commitment to the planned analysis.
Key Findings and Highlights
Highlights of this Analysis plan include:
- The Impact Evaluation’s estimation of impact will build on random assignment conducted by all 27 non-Tribal HPOG 2.0 grantees.
- The Analysis Plan describes the outcomes that will be examined for program impact; how these outcomes will be measured; how impacts on multiple outcomes will be prioritized in the summary of program effectiveness; and how COVID-19 disruptions to the economy and data collection will be addressed.
- It also provides a draft outline and report schedule for the Intermediate-Term Impact Report (ITIR).
Methods
Through September 2021, eligible applicants are being assigned randomly either to a treatment group that is offered HPOG 2.0 training and services or to a control group that does not have access to HPOG during the study period, but does have access to other training and services in the community. Data sources for the Impact Evaluation include baseline data collected about participants at study enrollment; follow-up participant surveys initiated about 15, 36, and 66 months after random assignment; and national administrative data on participants’ employment and earnings and their educational enrollment and degree receipt.
Citation
Judkins, David Ross, Sarah Prenovitz, Gabriel Durham, Jacob Alex Klerman, and Gretchen Locke. (2021). Analysis Plan for the HPOG 2.0 National Evaluation Intermediate-Term Impact Report, OPRE Report 2021-176. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- ACF:
- Administration for Children and Families
- OPRE:
- Office of Planning, Research, and Evaluation
- HPOG:
- Health Profession Opportunity Grants
- IEDP:
- Impact Evaluation Design Plan
- STIR:
- Short-Term Impact Report
- ITIR:
- Intermediate-Term Impact Report