Welfare and Family Self-Sufficiency Research and Evaluation Snapshot

When PRWORA replaced the Aid to Families with Dependent Children (AFDC) program with TANF, the law provided States with flexibility to implement a variety of rule changes, many of which had been piloted in a series of reforms prior to 1996 through waivers of AFDC program rules authorized by Section 1115 of the Social Security Act. These changes included various forms of benefit time limits, strengthened training and work requirements, and other administrative rule modifications. During this transition, ACF’s research and evaluation portfolio shifted from individual state AFDC demonstration studies to initiatives focused on testing program models that TANF programs had interest in implementing. Early post-AFDC learning included studies to understand the circumstances of people leaving TANF and evaluations of state interventions to improve employment retention among applicants and recipients subject to program participation requirements. As states broadened the types of employment activities they implemented, ACF launched tests to study particular barriers to employment and related questions.

ACF’s recent TANF-related research and evaluation has sought to understand and inform how TANF and other programs that serve TANF or TANF-eligible populations can best support their self-sufficiency and economic well-being. Rigorous studies funded by ACF and others have demonstrated that different types of interventions can improve labor market outcomes for disadvantaged groups, with variation in the magnitude and duration of impacts. For example, subsidized employment can increase employment and earnings in the short-term, and earnings supplements can increase both employment and income, at least while supplements remain in place. Career pathways models can improve educational progress and employment in the occupational sector targeted, and in some cases have been found to produce lasting earnings gains for individuals who meet program entrance criteria.[1]

A recent OPRE study used a quantitative meta-analysis to draw lessons from across more than 200 rigorous studies of 93 distinct employment and training interventions. The analysis found that interventions bundling multiple strategies are more effective than interventions using a single strategy. It also pointed to the importance of implementation quality, regardless of the intervention’s specific design.^[2]

Still, many questions remain. For example, the most successful training programs are generally inaccessible to individuals with low literacy or numeracy levels or significant personal barriers. Interventions targeting individuals who face complex barriers to employment, even when these programs boost employment and earnings, typically leave most participants with low earnings or inconsistent employment. Current and future ACF research will seek to explore these issues. Additionally, ACF has research underway to better understand how TANF and other ACF programs related to self-sufficiency are delivered and could be improved, and to explore how broader social context and environmental factors, including public policy and economic conditions, can impact the path to self-sufficiency.

[1] Fishman, M., et al., (2020). Employment and Training Programs Serving Low-Income Populations: Next Steps for Research (OPRE Report #2020-72). Washington, D.C.: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. /opre/resource/employment-and-training-programs-serving-low-income-populations-next-steps-for-research

[2] Vollmer, L., et al. (2017). The Right Tool for the Job: A Meta-Regression of Employment Strategies’ Effects on Different Outcomes - Appendices (OPRE Report #2017-40B). Washington, D.C.: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. https://www.acf.hhs.gov/sites/default/files/documents/opre/eser_ib_meta_analysis_appendix_042817_508.pdf (PDF)

In setting welfare and family self-sufficiency research and evaluation priorities, ACF takes into account legislative requirements and Congressional interests; the interest and needs of ACF, HHS, and Administration leadership; program office staff and leadership; ACF partners; the populations served; researchers; and other stakeholders. ACF routinely interacts with these stakeholders through a variety of engagement activities. These activities inform our ongoing research and evaluation planning processes.

Who

• State, territory, tribal, local, and non-profit TANF and other human services administrators and staff
• TANF training and technical assistance providers
• Employers
• Individuals and families served by TANF and other human services programs
• Federal partners in HHS and other agencies, such as the Office of the Assistant Secretary for Planning and Evaluation (ASPE), the Substance Abuse and Mental Health Services Administration (SAMHSA), the Office of Policy Development and Research at the Department of Housing and Urban Development (HUD/PD&R), the Chief Evaluation Office at the Department of Labor (DOL/CEO), the Employment and Training Administration at the Department of Labor (DOL/ETA), the Office of Disability Employment Policy at the Department of Labor (DOL/ODEP), the Food and Nutrition Service at the Department of Agriculture (USDA/FNS), the Institute of Education Sciences at the Department of Education (ED/IES), the Office of Career, Technical, and Adult Education at the Department of Education (ED/OCTAE), and the Office of Research, Demonstration, and Employment Support at the Social Security Administration (SSA/ORDES)
• Researchers and policy experts
• National organizations, such as the National Governors Association, the National Conference of State Legislatures, and the American Public Human Services Association
• Partners in other human services fields that serve families with low income

How

• Conferences and meetings, such as the Research and Evaluation Conference on Self-Sufficiency (RECS), National Association of Welfare Research and Statistics (NAWRS) conference, Association for Public Policy Analysis and Management (APPAM) conference, and periodic topic-specific meetings such as the Next Steps for Employment and Training Research Roundtable
• TANF and Tribal TANF Summits
• Surveys, focus groups, interviews, and other activities conducted as part of research and evaluation studies
• Interagency collaborations, such as the Federal Employment, Training, and Education Working Group (FETE)
• Structured mechanisms for broad stakeholder engagement, such as the Family Self-Sufficiency Research Technical Working Group

1. Human Capital Development — Which interventions build or improve individuals’ occupational and non-occupational skills in order to help them obtain employment and advance toward economic self-sufficiency?
2. Employment Attainment and Retention — How do individual programs and approaches move TANF recipients and other low-income individuals into jobs and help them retain employment?
3. Social Services Delivery Systems — How are TANF and other ACF programs related to self-sufficiency delivered; how does TANF coordinate with or otherwise intersect with other social services programs; and how could service delivery within these programs be improved?
4. Social Context and Environment — How does the broader social context, including public policies, labor market forces, and economic conditions, affect low-income families’ ability to achieve economic self-sufficiency?
5. Evaluation Capacity Building — How can training and technical assistance build human services providers’ capacity to learn from and contribute to the evidence base about what works to improve family self-sufficiency?
6. Dissemination and Knowledge Building — How can ACF ensure that research on programs and policies to advance family self-sufficiency is accessible and actionable for key stakeholders?

• Behavioral Interventions to Advance Self-Sufficiency (BIAS): This portfolio of multi-site studies explores the application of behavioral economics, which combines findings from psychology and economics, to improve outcomes among ACF’s programs and target populations. (#3)
• Building Evidence on Employment Strategies for Low-Income Families (BEES) and Next Generation of Enhanced Employment Strategies (NextGen) Project: These projects are evaluating the effectiveness of innovative programs designed to boost employment and earnings among individuals with low income. BEES has a special interest in programs that are state-initiated and programs that serve adults whose employment prospects have been affected by opioid use disorder, abuse of other substances, or mental health conditions. The NextGen Project has a special interest in interventions that are market-oriented and/or employer-driven. BEES and the NextGen Project are closely coordinated, and both projects are working closely with the Social Security Administration to incorporate a focus on employment-related early interventions for individuals with current or foreseeable disabilities who have limited work history and are at-risk of applying for Supplemental Security Income. (#2)
• Career Pathways Research Portfolio: This portfolio of work examines the implementation and effectiveness of career pathways programs and is comprised of the Pathways for Advancing Careers and Education (PACE) project and the evaluations of the Health Profession Opportunity Grants (HPOG) Program.[1] (#1)
• Evaluation of Employment Coaching for TANF and Related Populations: This project is evaluating interventions that apply coaching practices to promote job entry and retention among TANF populations and other individuals with low income. It is examining the impact of coaching on self-regulation skills and the role of these skills in generating impacts on employment outcomes. (#1)
• Human-Centered Design for Human Services: This project explores the application of human-centered design (HCD) across the delivery of human services programs at the federal, state, and local levels. The project is defining HCD in the context of human services; identifying programs that have experience implementing HCD approaches in social welfare contexts; and conducting a pilot study to help understand the feasibility of this approach in human services programs. (#3)
• Job Search Assistance Strategies Evaluation (JSA): This project is evaluating the relative impact of different TANF job search approaches on short-term labor market outcomes. (#2)
• Next Steps for Rigorous Research on Two-Generation Approaches: This project builds on earlier efforts that investigated the design and evaluability of integrated approaches to alleviating poverty that address the needs of both parents and children. The project is conducting formative research to prepare promising programs for evaluations of effectiveness; building the evaluation capacity of programs and researchers to conduct rigorous evaluations of integrated approaches to supporting child development and improving family economic security; and addressing measurement issues to promote learning across evaluations and a better understanding of processes and outcomes of two-generation programs. (#3)
• Pathways to Work Evidence Clearinghouse: The What Works Clearinghouse of Proven and Promising Approaches to Move Welfare Recipients to Work: This Congressionally-mandated clearinghouse provides a comprehensive, transparent, and systematic assessment of the effectiveness of interventions designed to help job seekers with low income succeed in the labor market. It shares the results on a user-friendly website and will create and disseminate synthesis reports regarding the overall state of evidence in the field. It builds on the previously completed Employment Strategies for Low-Income Adults Evidence Review by including newer and wider reaching research and assessing the effectiveness of the interventions reviewed. (#6)
• Promoting and Supporting Innovation in TANF Data: This project supports innovation and improved effectiveness of state TANF programs by enhancing the use of data from TANF and related human services programs. This work may include encouraging and strengthening state integrated data systems, promoting proper payments and program integrity, and enabling data analytics for TANF program improvement. Across its activities, the project supports the use of data for understanding the broad impact that TANF has on families and improving knowledge of how the federal government and state partners can use data to more efficiently and effectively serve TANF clients. (#5)
• Racial and Ethnic Disparities in Human Services Analysis Execution Project (RED-X): This project builds on a  predecessor study, which helped build a base of knowledge to inform ACF’s identification and understanding of ethnic and racial differences in access to and take-up of human services; in the nature and quality of services received; and in the outcomes of services. RED-X is analyzing program administrative and survey data from two ACF programs—TANF and the Child Care Development Fund—to identify and analyze potential racial and ethnic disparities. (#3)
• State TANF Case Studies: This project is identifying promising and innovative employment and training programs for individuals with low income that include TANF recipients, and producing case studies of the selected programs. The programs profiled include approaches that provide or link families to child care services and other work support services. (#3)
• State TANF Policies Database: The State TANF Policies Database, also known as the Welfare Rules Database, serves as a single location where information on TANF program rules can be researched across states and/or across years. (#6)
• Subsidized and Transitional Employment Demonstration (STED): This project is evaluating subsidized employment strategies designed to address two distinct goals: 1) to provide work-based income support for individuals who are not able to find regular, unsubsidized jobs; and; 2) to improve the employability of disadvantaged groups. The project is conducting impact and implementation evaluations in eight sites and is closely coordinating with the Department of Labor’s Enhanced Transitional Jobs Demonstration. (#2)
• Supporting Partnerships to Advance Research and Knowledge (SPARK): This project builds on existing technical assistance (TA) efforts to support state, local, and tribal TANF agencies with evaluating program changes or innovations and sharing findings, lessons learned, and resources with other sites that may benefit from their experiences. The project also will document the landscape of current or previous evaluation TA activities in human services program contexts, what can be learned from other fields that have examined this question, and what lessons can be drawn to inform future federal evaluation capacity-building efforts. (#5)
• Understanding Poverty: Childhood and Family Experiences: This study is examining the experiences and perspectives of children and families living in poverty, including those who apply for and access TANF and other safety net benefits. (#4)
• Understanding Poverty: TANF Office Culture: This project is exploring how organizational culture, office design, and office procedures contribute to shaping clients’ experiences with TANF, the services provided to them, and potentially their outcomes. (#3)

The broad questions listed above will continue to drive much of ACF’s research and evaluation activity in this area. Future activities will also be informed by emerging findings from ongoing research and evaluation activities, other learning activities, and continued engagement with welfare and family self-sufficiency stakeholders.

Examples of activities planned for the next few years include:

• Conducting rigorous evaluations of interventions designed to improve employment outcomes for individuals with complex barriers to employment including physical and mental health conditions, substance use disorder, opioid dependency, criminal history, or limited work skills and experience
• Designing and testing interventions that apply behavioral science principles to challenges facing human services programs
• Analyzing labor market information to inform service planning and job placement for populations with low income
• Conducting research activities to identify racial and ethnic disparities across human services delivery systems
• Supporting the provision of technical assistance to strengthen the evaluation capacity of human services agencies
• Supporting state-initiated research and evaluation on innovative approaches for increasing economic self-sufficiency and reducing public assistance dependency
• Collecting information on the effects of the COVID-19 pandemic on the support service needs, employment, and economic and personal well-being of families with low income
• Convening the Research and Evaluation Conference on Self-Sufficiency to stimulate discussion on cutting-edge research from evaluations of programs, policies, and services that support families with low income on the path to economic self-sufficiency