Absence of conflict of interest.
Citation
Hendra, R., Greenberg, D. H., Hamilton, G., Oppenheim, A. Pennington, A. Schaberg, K., and Tessler, B. L. (2016). Encouraging evidence on a sector-focused advancement strategy. New York: MDRC. [Madison Strategies Group]
Highlights
- The study’s objective was to examine the impact of the WorkAdvance sectoral training program at the Madison Strategies Group site on employment, earnings, education, and training from 2011 to 2015. The authors investigated similar research questions with three other sites, the profiles of which are available here.
- The study used a randomized controlled trial design to compare the treatment group, which was able to access the WorkAdvance program at the Madison Strategies Group site, and the control group, which was not eligible for WorkAdvance services but could access other services in the community. The authors collected data from two sources: a follow-up survey and unemployment insurance (UI) wage and employment data.
- Using survey data, the authors found that those in the treatment group were significantly more likely to complete education or training compared with those in the control group. The authors found no statistically significant impacts of the program on employment or earnings from either the UI data or survey data.
- The quality of the causal evidence is high for employment and earnings outcomes based on UI data because those outcomes were based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects for those outcomes are attributable to the WorkAdvance program at the Madison Strategies Group site and not to other factors. For education and training outcomes from the follow-up survey, the quality of the causal evidence is moderate because those outcomes were based on a randomized controlled trial in which many people did not complete the follow-up survey, but the authors did account for existing differences between the treatment and control groups. This means we are somewhat confident that the estimated effects for those outcomes are attributable to the WorkAdvance program at the Madison Strategies Group site, but other factors might also have contributed.
Intervention Examined
The WorkAdvance sectoral training program
Features of the Intervention
WorkAdvance was a sectoral training program that coupled a career readiness and occupational training program with job placement supports and post-employment job retention services. The training at the Madison Strategies Group site in Tulsa, Oklahoma, occurred at private or technical schools and community colleges. The WorkAdvance program included five separate six-hour sessions on career readiness training, up to 32 weeks of occupational training toward an industry certificate or license, and career coaching. The training targeted the transportation and manufacturing sectors.
To be eligible for the program, people had to be at least 18 years old, be legally allowed to work in the United States, have a family income of less than 200 percent of the federal poverty line, and either be unemployed or employed at a job making less than $15 per hour. In addition, people had to have math and reading scores at the 8th-grade level or higher, pass a mechanical aptitude and behavioral assessment, pass a manual dexterity test, and have a driver’s license.
Features of the Study
The study used a randomized controlled trial design, in which 699 people were assigned to either the treatment group (which was offered the WorkAdvance program) or a control group (which could not join the WorkAdvance program but could access other services available in the community). The sample was primarily male (84 percent) with an average age of 35 years old. In all, 39 percent of people in the sample were white and 28 percent were African American. Forty percent of the sample had previously been convicted of a crime.
The authors collected data from two sources: (1) a follow-up survey completed two years after random assignment by 560 members of the sample and (2) quarterly UI wage and employment data for nine quarters after random assignment from 697 members of the sample. The authors used a statistical model to compare the outcomes of treatment and control group members.
Findings
Employment
- The authors did not find any statistically significant impacts of the WorkAdvance program on employment using either the survey data or UI data.
Earnings and wages
- The authors did not find any statistically significant impacts of the WorkAdvance program on earnings using either the survey data or UI data.
Education and skills gains
- Using survey data from the second year after random assignment, the authors found that those assigned to the treatment group were significantly more likely to have obtained a degree or credential (28 percentage points), to have completed a skills training program (27 percentage points), and to have obtained a credential in the targeted sectors of transportation or manufacturing (25 percent) compared with those in the control group.
Considerations for Interpreting the Findings
The outcomes using follow-up survey data came from a randomized controlled trial in which there was no follow-up data available for a large number of people in the treatment and control groups. The authors did account for potential existing differences between the groups, however, so these outcomes receive a moderate evidence rating.
Fifty-one percent of control group members received some form of employment service from community-based or governmental organizations, and 30 percent received skills training. If this training was similar in intensity and content to the WorkAdvance training, then there might not have been a clear difference between the treatment and control groups, making it more challenging to detect impacts of the WorkAdvance program.
The study took place during the Great Recession, so there could have been fewer employment opportunities available to the sample.
Causal Evidence Rating
The quality of the causal evidence is high for employment and earnings outcomes based on UI data because those outcomes were based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects for those outcomes are attributable to the WorkAdvance program at the Madison Strategies Group site and not to other factors. For education and training outcomes from the follow-up survey, the quality of the causal evidence is moderate because those outcomes were based on a randomized controlled trial in which many people did not complete the follow-up survey, but the authors did account for existing differences between the treatment and control groups. This means we are somewhat confident that the estimated effects for those outcomes are attributable to the WorkAdvance program at the Madison Strategies Group site, but other factors might also have contributed.