Introduction
Many social programs are intended to generate long-term benefits for their participants, but evaluations of those programs have historically not had access to the necessary resources to measure those outcomes over the long run—for 5, 10, or 15 years or longer. Administrative data—data that are created and stored to enable government administration, or as a by-product of it—present a potentially low-cost opportunity for tracking the long-term effects of policy or program interventions. However, the procedures for gaining access to these data are often idiosyncratic or time-intensive. In addition, little documentation is available about how to access and use these data for research purposes, and researchers are likely to encounter unique data-quality challenges in so doing.
Purpose
This guide is a resource to assist program evaluation project teams—including funders, sponsors, and evaluation research partners—in assessing the feasibility and potential value of examining long-term outcomes using administrative data. It describes common steps that are involved in linking evaluation data and administrative data, how to assess the quality of linked study and administrative data, as well as how to assess the quality of linked study and administrative data.
The guide is directed primarily toward research teams considering whether to examine long-term outcomes for evaluations, particularly those whose work has been completed. This guide also includes valuable information that may enable research on long-term outcomes for new or ongoing evaluations. Examples and case studies throughout the guide generally highlight efforts to research long-term, employment-related interventions, but the concepts presented should be applicable to a variety of social interventions.
Key Findings and Highlights
The guide will help teams tackle topics such as:
- how to identify worthwhile, policy-relevant opportunities for extending evaluation follow-up
- what study data and infrastructure are required to enable extended follow-up
- factors to consider in selecting suitable administrative data sources
- navigating the legal and ethical requirements that are commonly associated with pursuing extended follow-up
- special considerations for matching study and administrative data
Methods
This Guide for Using Administrative Data to Examine Long-Term Outcomes in Program Evaluation is being produced as part of the “From Theory to Practice” (T2P) project. This guide complements federal efforts to expand the use of administrative data for building evidence—in this case, evidence about the long-term effectiveness of federally funded programs and interventions. Through T2P, the Administration for Children and Families’ Office of Planning, Research, and Evaluation (ACF/OPRE) in the U.S. Department of Health and Human Services is developing resources to support these interests and explore the benefits and limitations of linking study and administrative data for long-term research.
Citation
Bigelow, Jonathan, Alexandra Pennington, Kelsey Schaberg, and Deondre’ Jones 2021. A Guide for Using Administrative Data to Examine Long-Term Outcomes in Program Evaluation. OPRE Report 2021-145. Washington, DC:
Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.