Citation
Molina, F., Cheng, W., and Hendra, R. (2008). The Employment Retention and Advancement project: Results from the Valuing Individual Success and Increasing Opportunities Now (VISION) program in Salem, Oregon. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Highlights
- The study’s objective was to examine the impact of the Salem, Oregon, Valuing Individual Success and Increasing Opportunities Now (VISION) program on employment, earnings, and public benefits receipt among eligible single parents. VISION provided job search assistance and post-employment services to unemployed Temporary Assistance for Needy Families (TANF) applicants.
- The study used a randomized controlled trial to assign TANF applicants eligible for welfare-to-work programs to either VISION or the state’s typical welfare-to-work program, Job Opportunities and Basic Skills Training (JOBS), from May 2002 to May 2004. To estimate differences in outcomes between the two study groups, the authors retrieved data from unemployment insurance wage records, TANF and Food Stamps administrative records, and a 12-month follow-up survey.
- The study found that people in the VISION group received, on average, $365 more in TANF benefits and $160 more in Food Stamps than those in the JOBS group during the first year after random assignment.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to VISION, and not to other factors.
Intervention Examined
The Salem Valuing Individual Success and Increasing Opportunities Now (VISION) Program
Features of the Intervention
The VISION program in Salem provided job search assistance and post-employment services to unemployed TANF recipients. The program was run by the local welfare agency and a community college and operated out of a One-Stop Career Center. Participation was typically mandatory and enforced by sanctions. Job search assistance included job placement and workshops on job retention and career paths. Staff helped clients develop and regularly revisit a personal development plan to identify and prepare for a job that interested them. A case manager from the local welfare office could authorize supportive services. VISION staff reportedly devoted most of their time to pre-employment services and helped clients gain access to public assistance.
Features of the Study
At two welfare offices, evaluators randomly assigned TANF applicants eligible for welfare-to-work programs to either VISION or the state’s typical welfare-to-work program, JOBS, from May 2002 to May 2004. The study randomly assigned 1,820 people during the two years. The authors restricted the analysis sample to 1,504 single parents. Data on employment and earnings were available from unemployment insurance wage records for all study participants for the first year after random assignment. Data on public assistance receipt from TANF and Food Stamps administrative records during the first year were available only for the 977 participants randomly assigned by September 2003. The evaluation included a survey 12 months after random assignment administered to all participants who met the following criteria: randomly assigned from May to October 2003, 18 years old or older, and spoke English or Spanish. The fielded survey sample included 418 participants, of whom 300 responded to the survey. The authors used regression analysis to estimate differences in outcomes across the VISION and JOBS groups.
Findings
- The study found that people in the VISION group received, on average, $365 more in TANF benefits and $160 more in Food Stamps during the first year after random assignment than did the JOBS group. These differences were significant at the 5 percent level.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to short-term benefits receipt. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the VISION program, and not to other factors.