This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Schochet, P., Burghardt, J., & McConnell, S. (2006). National Job Corps study and longer-term follow-up study: Impact and benefit-cost findings using survey and summary earnings records data. Washington, DC: Employment and Training Administration, U.S. Department of Labor.
Highlights
- The National Job Corps Study included several reports, including this longer-term impact report. The report’s objective was to examine the impact of the Job Corps program on participants’ long-term earnings, employment, and other outcomes.
- Job Corps offered intensive academic classroom instruction and vocational skills training to economically disadvantaged youth. Its effectiveness was evaluated using a randomized controlled trial conducted in 48 states and the District of Columbia.
- The study found a number of statistically significant, positive impacts of Job Corps on earnings and employment in the first four follow-up years. However, there were no significant differences between the youth who were offered Job Corps and the control group on employment rate or earnings in the fifth through ninth follow-up years, as measured by administrative data.
- The quality of causal evidence presented in this report is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the effects estimated in the study are solely attributable to Job Corps, and not to other factors.
Intervention Examined
Job Corps
Features of the Intervention
Job Corps is a federally funded program that has operated since 1964. It offers individualized academic education, vocational training, counseling, and job placement assistance to economically disadvantaged youth ages 16 to 24. Services are delivered at Job Corps centers, where most Job Corps students reside while participating in the program. Most participants do not have a general equivalency degree (GED) or high school diploma upon program entry and they participate in the program for an average of eight months.
Features of the Study
The study took place at 105 Job Corps outreach and admissions agencies operating in the 48 contiguous United States and the District of Columbia. From November 1994 to December 1995, 15,386 eligible youth were randomly assigned to either the treatment group, which could enter the Job Corps program, or the control group, which was not able to enroll in Job Corps but could access other available programs. The randomization procedure assigned about 7 percent of the eligible applicants to the comparison group. All of these youth were in the research sample. About 13 percent of the youth in the treatment group were randomly sampled to participate in a follow-up survey.
Data sources for this study included a follow-up survey conducted four years after random assignment and Social Security earnings record data for nine years after random assignment. To estimate program impacts, the authors compared the means of the treatment and control groups, weighted for sample and survey design.
Findings
- Using administrative data, the study found that youth randomly assigned to Job Corps were significantly more likely to be employed during each of the first four follow-up years after random assignment than youth in the control group. For example, in the first year after random assignment, 89 percent of those assigned to the Job Corps group were employed, compared with 78 percent of those in the control group. In the fourth year after random assignment, 85 percent of those assigned to the Job Corps group were employed, compared with 83 percent in the control group. However, there were no statistically significant differences between the Job Corps and control group in the likelihood of being employed during the fifth through ninth years of follow-up.
- Using the survey data, which authors examined quarterly rather than annually, there were no statistically significant differences in the employment rates of the two groups during any quarter in the first four years of the follow-up period. (Survey data did not examine years five and later.)
- Both data sources agreed that youth offered Job Corps had significantly lower earnings compared with the control group in the first year following random assignment, when the Job Corps group would have been engaging in the program and thus not actively participating in the labor force. In the second through fourth years after random assignment, the youth offered Job Corps earned significantly more than the control group, according to at least one of the data sources. However, there were no statistically significant differences in earnings between the groups in follow-up years five through nine, as measured by administrative data.
- Finally, using survey data, the study found that the youth offered Job Corps received $3,696 in public benefits four years after random assignment, whereas the control group received $4,156 in public benefits. This difference was statistically significant.
Considerations for Interpreting the Findings
The authors noted that the impacts on earnings using survey data were notably greater than those found when using the administrative data. To investigate this, they estimated impacts using administrative data only for those youth who also responded to the follow-up survey and then compared those impacts with those estimated for youth who did not respond to the survey. The estimated impact was much larger for youth who responded to the survey than for youth who did not, suggesting that the survey results might overstate the true impact of the program on the full sample.
The authors estimated multiple related impacts on outcomes related to employment and earnings. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-conducted randomized controlled trial. This means we are confident that the effects estimated in the study are solely attributable to Job Corps, and not to other factors.
Additional Sources
Schochet, P., Burghardt, J., & Glazerman, S. (2001). National Job Corps study: The impacts of Job Corps on participants’ employment and related outcomes. Washington, DC: Employment and Training Administration, U.S. Department of Labor.