Introduction
Research Questions
- What impacts do HPOG programs have on the outcomes of participants and their families?
- To what extent do impacts vary across selected subpopulations?
In 2010, the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services awarded the first round of grants from the Health Profession Opportunity Grants (HPOG 1.0) Program to 32 organizations in 23 states, including five Tribal organizations. The purpose of the HPOG Program is to provide education, training, and supportive services to Temporary Assistance for Needy Families (TANF) recipients and other low-income individuals to prepare them for occupations in the healthcare field that pay well and are expected to either experience labor shortages or be in high demand.
The ACF Office of Planning, Research, and Evaluation is using a multipronged research and evaluation strategy to assess the success of the HPOG Program. HPOG 1.0 is being evaluated using an experimental design in which program applicants were assigned at random to a “treatment” group that could access the program or to a “control” group that could not. The outcomes of these respective groups are being compared in the short term (about 15 to 18 months after randomization), intermediate term (after three years), and long term (after six years).
This report presents the plan for evaluating the six-year impacts of the HPOG 1.0 grants on various outcomes. It describes the operationalization of outcome measures, including their source data.
Purpose
The purpose of this report is to describe a plan for answering the study’s key research questions using six-year outcomes. By specifying these details in advance, this document serves as a public commitment to the planned analysis.
Key Findings and Highlights
- The report identifies three confirmatory outcomes, which are indicators of HPOG 1.0’s progress toward its goals after six years. These are: (1) receipt of a postsecondary credential requiring one year or more of training; (2) current employment in a healthcare occupation; and (3) average quarterly earnings over the 23rd and 24th quarters after random assignment.
- The report also identifies secondary and exploratory outcomes. Secondary outcomes are additional important outcomes identified in the HPOG logic model. Exploratory outcomes include additional outcomes of interest and alternative measures of the confirmatory and secondary outcomes.
- The report describes the new data sources that will be used to carry out the six-year impact analysis, including the Six-Year Follow-Up Survey and administrative data from the National Student Clearinghouse (NSC).
- The report describes planned analysis of the influence of COVID-19 on the evaluation and on its findings.
Methods
The impact evaluation of HPOG 1.0 uses an experimental evaluation design. As elaborated in prior analysis plans, the evaluation estimates the impact of HPOG 1.0 using a regression model that adjusts the difference between average outcomes for treatment versus control group members by controlling for characteristics measured at baseline. This Analysis Plan describes items that are new to the six-year analysis, including data sources and measures; previously established analytic procedures are described in prior project documents.
Citation
Litwok, Daniel, Douglas Walton, and Laura R. Peck. (2021). Health Profession Opportunity Grants (HPOG) Impact Study’s Six-Year Follow-Up Analysis Plan. OPRE Report 2021-26, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- ACF:
- Administration for Children and Families
- HPOG 1.0:
- First round of Health Profession Opportunity Grants
- TANF:
- Temporary Assistance for Needy Families