Absence of Conflict of Interest
This study was conducted by staff from Abt Associates, which administers CLEAR. The review of this study was conducted by ICF Incorporated, which also administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.
Citation
Highlights
- The study's objective was to examine the impact of the Back to Work (B2W) and Independent Job Search (IJS) programs on employment, earnings, and public benefits receipt outcomes.
- The study used a randomized controlled trial to evaluate the difference in outcomes between participants in the B2W and IJS programs. Using data from the National Directory of New Hires, administrative data from New York City, and survey data, the authors conducted statistical models to compare differences in outcomes between applicants assigned to each of the two programs.
- The study found that participation in the B2W program reduced the receipt of cash assistance and SNAP benefits compared to the IJS program.
- The study receives a high evidence rating. This means we are confident that the estimated effects are attributable to the B2W and IJS programs, and not to other factors.
Intervention Examined
Back to Work (B2W) and Independent Job Search (IJS)
Features of the Intervention
The Temporary Assistance for Needy Families (TANF) program helps low-income adults become more self-sufficient by providing cash assistance and employment services. As a condition of benefit receipt, the TANF program requires participation in a specified set of work activities, including job search assistance (JSA). The Back to Work (B2W) and Independent Job Search (IJS) programs were developed to assess the effectiveness of various JSA strategies offered and to identify what aspects have the strongest impacts on increasing employment and sustaining a living wage for applicants.
The B2W program required a total of 35 hours/week for daily meetings with staff and participation in job search activities at the program office. The services included both group classes and one-on-one assistance to help applicants with job search skills and find them jobs. The IJS program had the same overall weekly requirement of 35 hours, but applicants were required to conduct independent job searches. The program also did not have restrictions on timing or location and required only one weekly meeting with staff at the program office. Both interventions were delivered to job ready cash assistance applicants in the New York City area across two boroughs (Brooklyn and Queens). Both programs were offered by two vendors in each borough (America Works and Goodwill) and operated during the period after an applicant applied for cash assistance benefits for a period of up to six weeks.
Features of the Study
The study used a randomized controlled trial to evaluate the impact of two different intervention programs. All eligible applicants were randomly assigned to either the B2W or IJS programs. The research team worked with vendor offices to design and implement the random assignment procedures, which used a six-step process. Participants were from two boroughs in NYC across four study sites (America Works Brooklyn, America Works Queens, Goodwill Brooklyn, and Goodwill Queens). A total of 2,699 applicants attended orientation, were screened as job ready, and consented to the study. Of the eligible applicants, 1,345 applicants were assigned to the B2W program group and 1,354 were assigned to the IJS program group. Data for both programs were collected during the evaluation period between October 2015 and October 2016. The study used several data sources including the National Directory of New Hires that provided earnings and employment outcomes data, administrative data from New York City on cash assistance and SNAP benefit receipt, and data from a follow-up survey administered 6-months after random assignment. The authors used statistical models to compare differences in outcomes between applicants assigned to each of the two intervention programs.
Findings
Earnings and wages
- The study did not find a significant difference in cumulative earnings over the two-quarter follow-up period between participants in the B2W and IJS programs.
Employment
- The study did not find a significant difference in employment in the second quarter after random assignment between participants in the B2W and IJS programs.
Public Benefit Receipt
- The study found that significantly fewer participants in the B2W program received Family assistance and Safety Net cash assistance during the study period compared to IJS participants (59 percent vs. 68 percent, respectively). No significant differences were found for receipt of SNAP benefits.
- The study also found that BRW participants received $99 less in Family assistance and Safety Net cash assistance and $90 less in SNAP benefits during the study period compared to IJS participants. These differences were significant.
Considerations for Interpreting the Findings
The study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Back to Work (B2W) and Independent Job Search (IJS) programs, and not to other factors.