Introduction
Research Questions
- Did the randomization process produce treatment and control groups with similar characteristics at baseline?
- For young adults randomized to the treatment group, what was the rate of enrollment in LifeSet?
- For young adults randomized to the treatment group, what characteristics distinguish young adults who enrolled in LifeSet from young adults who did not?
- How does LifeSet differ from usual services?
- How are eligible young adults identified, recruited, and enrolled in the LifeSet program?
- Is LifeSet being delivered as intended?
Each year, around 20,000 young adults age out of foster care, meaning they left foster care solely due to their age (Annie E. Casey Foundation 2021). Many young adults who age out experience challenges such as homelessness, unemployment, lack of education, incarceration, and untreated mental health and substance use problems (Courtney et al. 2020; Dworsky et al. 2011). Unfortunately, the evidence base for interventions that effectively meet the needs of young adults leaving foster care is extremely limited.
As part of a larger project aimed at increasing the number of evidence-supported interventions for the child welfare population, the Supporting Evidence Building in Child Welfare project selected the LifeSet program for evaluation as it had positive findings from a prior evaluation (Skemer and Valentine 2016). LifeSet provides young adults leaving foster care with intensive community-based support and guidance to help them make a successful transition to adulthood. The core of the program is a therapeutic case management relationship between the young adults and LifeSet specialists who work with young adults to develop and move towards young adults-defined goals in multiple domains of independent living. We designed the current evaluation to rigorously test the impact of the program on longer-term outcomes.
To investigate the LifeSet program, we used a randomized controlled trial (RCT) design. The RCT design allows us to know whether young adults who receive LifeSet have better outcomes than young adults who receive usual services. This report provides a description of young adults randomized in the study at baseline, characteristics of treatment young adults who enrolled in LifeSet, and perceptions of the LifeSet program from young adults, program staff, and state child welfare administrators.
Purpose
The purpose of this report is to share early findings from an evaluation of the LifeSet program in New Jersey. The evaluation used an RCT design to learn whether LifeSet produces better outcomes than services as usual for young adults in extended foster care. This report provides a description of young adults randomized in the study and an initial analysis of baseline equivalence using child welfare administrative data. We also present early findings on the treatment group’s enrollment in LifeSet, including enrollment rates and characteristics of treatment group young adults who enrolled in the program. Finally, we summarize information about LifeSet’s implementation gathered through discussions with program staff and administrators, state child welfare administrators, and young adults in the study.
Key Findings and Highlights
Survey responses suggest young adults in the study experience challenges but also have strengths to build upon.
Of young adults who completed the survey, almost one-third indicated at least one instance of housing instability at baseline.
Among young adults who did not have a high school diploma, over half were currently enrolled in education or training at the time they completed the survey.
Fifty-seven percent of survey respondents reported working at a full or part-time job.
About 45 percent of young adults reported receiving mental health treatment in the past year.
The randomization process in New Jersey resulted in treatment and control groups that had similar characteristics at baseline.
Age at randomization was the only characteristic of statistical non-equivalence. The treatment group was about two months younger than the control group at randomization.
The treatment and control groups were statistically equivalent on all measures of child welfare history, such as total number of removals, lifetime months in placement, and age at most recent removal.
For young adults randomized to the treatment group, two-thirds enrolled in LifeSet.
Nearly one year after randomization ended, 68 percent of treatment group young adults had enrolled in LifeSet.
There were no significant differences in enrollment rates across implementing organizations.
Young adults who enrolled in LifeSet were slightly older and spent less time placed with kin than young adults who did not enroll.
Young adults who enrolled in LifeSet were about two months older at the time of randomization than young adults who chose not to enroll and this difference was statistically significant.
There were no significant differences between enrollees and nonenrollees with regard to sex and race/ethnicity.
Young adults who enrolled in LifeSet spent significantly less time placed with kin during their most recent removal than young adults who did not enroll. Enrolled young adults spent an average of 5 months placed with kin compared to an average of 9 months for nonenrolled young adults. There were no other significant differences related to young adults’ child welfare history.
LifeSet differs from usual services in several key ways.
Staff felt that LifeSet’s scope and flexibility, program goals, higher frequency of communication with young adults, and well-defined model were key features that distinguish it from services as usual.
In our focus groups, young adults perceived LifeSet as being more attentive to their goals and wants than other programs. Young adults we spoke with often described LifeSet as a program that “listens to them” in terms of case planning and goal setting.
Staff used multiple methods to recruit and enroll eligible young adults in the LifeSet program.
The New Jersey Department of Children and Families (DCF) identified which young adults were eligible for LifeSet through a centralized referral process.
LifeSet supervisors most often assigned cases based on which LifeSet specialist had openings on their caseload.
Some specialists would contact a young adults’ DCF caseworker to get more information about the young adults’ needs and goals. Other specialists preferred to have a “blank slate” of sorts when entering conversations with young adults.
A common recruitment tactic was focusing on how LifeSet is unique and not a duplication of other services the young adults may have received.
The LifeSet model encourages specialists to do unannounced visits to young adults’ homes (referred to as “pop-ups”) to engage referred young adults who had not enrolled. Staff we spoke with expressed mixed feelings about doing pop-ups. Some felt the pressure to conduct pop-ups, especially multiple times after young adults had declined LifeSet, was at odds with the program’s philosophy of being young adults driven.
LifeSet is being delivered as Youth Villages intended, with some minor modifications during the COVID-19 pandemic.
Youth Villages, the developer of LifeSet, conducted four certification reviews of each implementing organization in New Jersey. For all four certifications, the New Jersey organizations met the minimum thresholds with overall scores ranging from 84 percent to 95 percent, suggesting LifeSet is being delivered as intended.
Program descriptions given to staff and treatment group young adults during focus groups align with the core elements of the program. This includes weekly meetings with young adults, working on young adults-defined goals, and following the program’s staffing and supervision requirements.
Implementation of LifeSet was modified to allow for virtual sessions due to the COVID-19 pandemic. The program returned to requiring in-person sessions in 2023. However, specialists stated they would sometimes give in to young adults’ preference to meet virtually, viewing a virtual session as better than no session.
Methods
To evaluate the LifeSet program, we used an RCT design. An RCT was chosen because the design provides the best evidence as to whether a program works and because the target population for the LifeSet program included many more individuals who were eligible for the intervention than could be served by the program’s limited slots. The evaluation therefore allocated the limited program slots by lottery, which is a fair way to allocate scarce resources.
Young adults aged 17-21 who had an open services case with DCF, were living in the community, and did not have a disqualifying condition were eligible for the study. Randomization of young adults occurred from August 2021 through March 2023. Only young adults randomized to the treatment group received a referral to LifeSet. Control group young adults continued to receive referrals to other services as usual. We used child welfare administrative data from DCF to analyze baseline equivalence of the treatment and control groups and describe young adults who enrolled in LifeSet. We used LifeSet program data from Youth Villages to analyze program enrollment rates.
The project team is conducting a 24-month follow-up survey through grant funding from Youth Villages. They may also receive administrative data from additional sources to assess outcomes such as incarceration, receipt of public benefits, education, and employment. They will conduct their impact analyses in accordance with the preregistered analysis plan. They expect to publish the final impact study report in late 2026.
Citation
Brewsaugh, Katrina, Mark E. Courtney, Michael Pergamit, and Annelise Loveless (2024). LifeSet Evaluation Baseline Data Analysis, OPRE Report #2024-329, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Glossary
- Baseline equivalence:
- When there are no statistically significant differences between the treatment and control groups on key measures (e.g., demographics, child welfare history, or other characteristics) at randomization. This shows that the treatment and control groups are similar at baseline.
- DCF:
- Department of Children and Families is the child welfare agency in New Jersey.
- RCT:
- Randomized controlled trial is a study which randomly assigns people to either the treatment group or the control group. The treatment group gets the intervention while the control group receives the usual services.
- Youth Villages:
- Developer of the LifeSet program. They train and certify other organizations to provide LifeSet.