Introduction
Research Questions
- How can Human-Computer Interaction design approaches to developing digital employment tools support jobseekers, particularly those with low incomes, in job search and skill building?
- How can studies of digital employment tools created with Human-Computer Interaction design approaches inform opportunities for future E&T research and evaluation?
The Office of Planning, Research, and Evaluation (OPRE), within the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services, has a long history of supporting rigorous research and evaluation on the broad range of human services programs that fall under ACF’s auspices. Many of ACF’s programs support employment among low-income populations, and, consequently, OPRE regularly supports numerous evaluations of employment and training (E&T) programs for low-income populations. Though many E&T programs for populations with low incomes have historically been heavily influenced by the field of Economics, OPRE looks more broadly for theories and approaches to inform its work.
OPRE funded MEF Associates, in partnership with MDRC, to organize and facilitate a roundtable and prepare a series of white papers to explore future research topics related to employment and training programs for low-income populations. The first white paper discusses the current knowledge gaps and suggested areas for further research on designing effective E&T programs for low-income populations (Fishman et al., 2020). The second white paper discusses ongoing trends in the labor market and their potential effects on the nature of work over the next 10 to 15 years for low-income populations (Miller, 2021). The last two white papers—of which this brief is one—seek to identify approaches from fields not typically drawn on in E&T research that present opportunities to potentially strengthen the design, delivery, and effectiveness of E&T programs through research and evaluation (Fumia and Yau, 2021; Putnam et al., 2021).
This brief focuses on digital employment tools created with Human-Computer Interaction (HCI) design approaches. Digital employment tools and technology are embedded in the modern labor market but often fail to meet the needs of historically underserved and marginalized communities. This brief describes the role of digital employment tools for low-income populations and details how each phase of the HCI design approach differs from other approaches to developing digital employment tools, including relevant research within each phase. It concludes with implications of this work for future E&T research and evaluation.
Purpose
Existing digital employment tools and technologies often fail to meet the needs of historically underserved and marginalized communities, who experience barriers of access and digital literacy related to digital employment tools. These barriers might exacerbate existing inequities in the labor market for underserved and marginalized populations. This brief discusses how HCI design approaches differ from other approaches and can engage jobseekers as designers to create equitable digital employment tools that support job search and skill building. It identifies pathways for further research and evaluation that could inform directions for OPRE’s and other federal agencies’ research and evaluation portfolios and for the broader field of researchers and practitioners.
Key Findings and Highlights
- The HCI design approach challenges the status quo of digital employment tool design that often marginalizes low-income jobseekers. HCI design principles focus on end users’ strengths, challenges, and needs; acknowledge the historical marginalization of end users; and center end users as the designers. This approach can create more equitable tools tailored to jobseekers with low incomes and their specific needs.
- Examples of preliminary digital employment tools created for historically underserved and marginalized jobseekers can serve as models for future digital employment tool development.Through the design process of these tools, jobseekers identify their strengths and tools are designed to build on these strengths.
- HCI design approaches are a relatively new field and evidence is limited. There has been no large-scale, rigorous testing of HCI design approaches to developing digital employment tools nor the digital employment tools themselves. Wider use of these design principles will concurrently allow for more rigorous research on the effects of using HCI for digital tool development and of the tools themselves.
Methods
This brief was developed through an iterative and exploratory process. An initial literature scan identified seven promising academic disciplines that provide new perspectives and approaches to the field of E&T.0F The disciplines included Public Health, Business and Management, Human-Computer Interaction, Sociology, Geography, Psychology, and Anthropology. A broad scan of research and scholarship between 2010 and 2020 within those disciplines revealed topics and experts with a focus on E&T activities or low-income populations. The topic of this brief, technology-based support for jobseekers with low incomes, emerged as one of the focal topics during conversations with experts within those disciplines who are advancing research on interventions and approaches related to E&T that could support improved outcomes for low-income and vulnerable populations served by ACF programs.
Recommendations
- Conduct descriptive research to explore the implementation of digital employment tools developed with HCI design approaches and embedded into existing E&T programs.
- Conduct comparative impact studies of the effect of HCI-informed enhancements to existing digital employment tools, such as providing additional technical assistance to build digital literacy skills for tools currently in use, like online resume builders.
- Design new digital employment tools created with HCI design approaches and compare them to existing approaches in E&T to understand the relative merits of these approaches. Large scale evaluation, including randomized controlled trials and complementary implementation and cost studies, can help build the evidence base for which digital employment tools work, why they work, and at what cost.
Citation
Putnam, Marisa, Tawanna Dillahunt, and Asaph Glosser. 2021. New Directions in Employment and Training Research and Evaluation: Digital Employment Tools Created with Approaches from Human-Computer Interaction. OPRE Report 2021-241, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.