Absence of conflict of interest.
Citation
Eyster, L., Nightingale, D.S., Barnow, B., O'Brien, C., Trutko, J. & Kuehn, D. (2010) Implementation and early training outcomes of the High Growth Job Training Initiative: Final report. Washington, D.C.: Urban Institute Center on Labor, Human Services, and Population.
Highlights
- The study’s objective was to examine the impact of the High Growth Job Training Initiative (HGJTI) grant on earnings. This summary focuses on the apprenticeship program at South Texas College (STC).
- This nonexperimental study used propensity score matching and difference-in-differences models to compare the earnings outcomes of workers who participated in the apprenticeship program to those who did not.
- The study found that participation in the apprenticeship program was significantly related to higher earnings.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the South Texas College (STC) apprenticeship program; other factors are likely to have contributed.
Intervention Examined
South Texas College (STC) Advanced Manufacturing Apprenticeship Program
Features of the Intervention
Between 2001 and 2008, the Department of Labor’s Employment and Training Administration funded more than 160 High Growth Job Training Initiative (HGJTI) grants. The Lower Rio Grande Valley Workforce Development Board in partnership with South Texas College (STC) received an HGJTI grant that ran from 2005 to 2008. The Lower Rio Grande Valley Workforce Development Board was the fiscal agent and STC conducted the grant activities. STC is a community college serving four counties in South Texas near McAllen, Texas.
Due to shortages of skilled manufacturing workers, STC focused on advanced manufacturing training in industrial maintenance, tool and die, plastics, and machining. STC advanced manufacturing training included a three- to four-year registered apprenticeship program for incumbent workers from local manufacturing firms. The apprenticeship program included 600-800 hours of training and 6,000-8,000 hours of on-the-job training. Training occurred on STC’s campus with the on-the-job component of the apprenticeship located at the employer site. The apprenticeship programs enrolled 270 incumbent workers of small to medium sized advanced manufacturing firms in South Texas. The workers in the apprenticeship program were all Hispanic males.
Features of the Study
The study used propensity score matching and difference-in-differences models to estimate the impact of the apprenticeship program on earnings outcomes. The treatment group included 130 workers with complete data who were part of the apprenticeship program (in machining, plastics, tool and die, or industrial maintenance) between 2005 and 2008. The comparison group was drawn from Workforce Investment Act (WIA) Adult and Dislocated Workers program participants who received core or intensive services from local One-Stop Centers during the same time period. All women and non-Hispanic men were removed from the WIA comparison pool to match the STC enrollees, resulting in a pool of 13,813 for matching. The authors used five matching strategies to create the comparison group but noted that data were not collected on demographics such as age or education which limited the variables that could be used for matching.
Data sources included STC college records, the Texas Workforce Commission, the South Texas Manufacturers Association, and the Ray Marshall Center for Studies in Human Resources at the University of Texas at Austin. The authors conducted statistical models to compare the changes in quarterly earnings (two quarters post program to quarters five and six of pre-program earnings and quarters seven and eight of pre-program earnings) between the treatment group and the comparison group. Variables used for matching and controlled for in the models included race/ethnicity, prior labor force status, and prior earnings. Unemployment rates were also included in the models as a measure of changing economic conditions over time.
Findings
- For comparison of the post-program earnings to pre-program quarters five and six, the study found that participation in the apprenticeship program was significantly related to higher earnings when compared to one of the five matched samples.
- For comparison of the post-program earnings to pre-program quarters seven and eight, the study found that participation in the apprenticeship program was significantly related to higher earnings when compared to one of the five matched samples. However, the matched sample was not balanced.
Considerations for Interpreting the Findings
Although the authors accounted for some baseline demographic characteristics between the treatment and comparison group, they did not match or control for age or a preintervention measure of education in their analytic model which is required by the review protocol. The preexisting differences between the groups on these variables—and not the STC apprenticeship program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the South Texas College (STC) apprenticeship program; other factors are likely to have contributed.