Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the impact of the Back on Track framework on obtaining a high school credential and being disconnected from both work and school (that is, neither working nor enrolled in school). The authors investigated similar research questions for three other contrasts, the profiles of which can be found here:
- Comparison across 3 site
- South King/Seattle site
- Hartford site
- Using propensity score matching, the study compared outcomes for students who participated in the Opportunity Works Philadelphia program to those of youth from other “business-as-usual” programs. The study tested for differences in outcome means between the treatment group and matched comparison group. The primary data sources were a baseline survey and a follow-up survey conducted 12 to 17 months later.
- The study suggested there was no significant difference in Opportunity Works Philadelphia participants’ chance of earning a high school credential or of being disconnected from both work and school compared to the matched comparison group participants.
- The causal evidence rating for this study is low. This means we would not be confident that any estimated effects would be attributable to the Back on Track framework; other factors are likely to have contributed. However, the study did not report statistically significant effects.
Intervention Examined
Back on Track
Features of the Intervention
The Back on Track framework aims to improve outcomes for “opportunity youth,” young people ages 16 to 24 who are not in school or meaningfully employed. Back on Track provides a framework that Opportunity Works sites could adapt to focus their activities and service delivery on helping young people find education and employment success. The Back on Track framework has three program phases. The Philadelphia Opportunity Works site focused on the second phase, postsecondary/career bridging, which helps youth who have obtained a high school credential or are very close to doing so, prepare for college courses and career-oriented professional training.
Philadelphia participants enrolled in a College Success program at one of the city’s high school reengagement centers. For 12 weeks, participants attended one-hour college readiness courses four days a week, with the courses alternating focus daily between academic content and noncognitive skills. At the same time, students could receive literacy support, high school equivalency instruction, case management, and more. After those 12 weeks, staff helped participants enroll at the Community College of Philadelphia, covering the cost of a “first college experience.”
Features of the Study
Using propensity score matching, the study compared outcomes for students who participated in the Opportunity Works program to those of youth from other “business-as-usual” programs that aimed to help underemployed, out-of-school youth of the same ages. The intervention group for this analysis is 89 Philadelphia participants who responded to the follow-up survey. The matched comparison group includes individuals from any of the three sites involved in the broader study (Philadelphia, Hartford, and Seattle/South King County). The sample size for the matched comparison group is unclear.
About half (47%) of the Philadelphia sample was male, with 33% being young men of color. Over half (53%) of the Philadelphia participants identified as non-Hispanic Black, 28% as Hispanic/Latinx, 3% as non-Hispanic White, and 17% as multiracial or another race. Almost all (96%) had less than a high school credential.
Findings
Education and skills gains
- For the Philadelphia site, the study suggested there was no significant difference in the share of treatment and comparison group members who earned a high school credential.
Employment
- For the Philadelphia site, the study suggested no significant differences in the share of treatment and comparison group members who were disconnected from both school and work.
Considerations for Interpreting the Findings
The authors did not demonstrate that the Philadelphia intervention group was equivalent to its matched comparison group before the intervention. Pre-existing differences between the groups, and not the Back on Track framework, could be one factor explaining the observed difference in outcomes between the groups.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not demonstrate that the groups being compared were similar before the intervention. This means we would not be confident that any estimated effects would be attributable to the Back on Track framework; other factors are likely to have contributed. However, the study did not report any statistically significant effects.