Skip to main content

PACE cross-program implementation and impact study findings (Gardiner & Juras, 2019)

Review Guidelines

Absence of conflict of interest

This study was conducted by staff from Abt Associates, which administers CLEAR. The review of this study was conducted by ICF Incorporated, which also administers CLEAR and is trained in applying the CLEAR causal evidence guidelines.

Citation

Gardiner, K., & Juras, R. (2019). PACE Cross-Program Implementation and Impact Study Findings (Report No. 2019-32). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services. [HCA]

Highlights

  • The study's objective was to examine the impact of Health Careers for All (HCA) on education outcomes. The authors investigated similar research questions for other programs, the profiles of which can be found here.
  • The study used a randomized controlled trial to examine the impact of the HCA program on education outcomes. Using participant surveys, the authors conducted a statistical model to compare the outcomes of the treatment and control group members.
  • The study did not find a significant relationship between HCA participation and credential receipt.
  • This study receives a low evidence rating. This means we would not be confident that any estimated effects would be attributable to Health Careers for All (HCA); other factors would likely have contributed. However, the study did not find statistically significant effects.

Intervention Examined

Health Careers for All (HCA)

Features of the Intervention

In 2010, the Workforce Development Council of Seattle-King County was awarded a 5-year Health Profession Opportunity Grant (HPOG) to launch the Health Careers for All (HCA) program. HCA offered participants navigation and case management services to help them select a training program and identify any participation barriers. Participants could access three levels of occupational training ranging from foundational (e.g., basic skills) through advanced (e.g., Licensed Practical Nurse credential). Participants received financial supports including vouchers and paid community college classes, as well as funding for supportive services (e.g., transportation). Participants were also provided with employment services after they completed their training. The program served participants with low occupational skills who were interested in working in the healthcare industry. Although low-income participants were not specifically targeted, they were recruited by the program.

Features of the Study

This study was part of the multi-program Pathways for Advancing Careers and Education (PACE) evaluation. The PACE evaluation was intended to address the gap in research on the effectiveness of career pathways programs, or programs providing postsecondary training or education in growing employment sectors. This profile focuses on the HCA program.

The study used a randomized controlled trial to examine the impact of the HCA program. Of the 654 participants who enrolled in the study between September 2012 and December 2014, 328 were randomly assigned to the HCA program (treatment group) and 326 were assigned to the control group. Control group members could not receive HCA program services but could choose to participate in other employment services within the community. The majority of the study sample was female (85 percent), with over half (51 percent) identifying as Black/African American, 13 percent Hispanic/Latino, 29 percent White, and 15 percent other race. Thirty percent of the sample had a high school degree or equivalency credential without further education/training. The data for the credential receipt outcome were drawn from the PACE 18-month survey. The authors used a statistical model to compare the outcomes of the treatment group with those of the control group.

Findings

Education and skills gains

  • The study found no significant relationship between HCA program participation and earning one or more credential(s).

Considerations for Interpreting the Findings

Although the study design was a randomized controlled trial, the study had high attrition and was treated as a nonexperimental design for this review. The authors did not control for the required differences between the groups at baseline as outlined in the protocol. These preexisting differences between the groups—and not HCA—could explain any observed differences in outcomes. The authors also note that control group members had a high rate of enrollment in (outside) education and training, which could have contributed to the null finding.

Causal Evidence Rating

The quality of causal evidence presented in this report is low, because it was a randomized controlled trial with high attrition and the authors did not account for preexisting differences between the groups before program participation. This means we would not be confident that any estimated effects would be attributable to HCA; other factors would likely have contributed. However, the study did not find statistically significant effects.

Additional Sources

Glosser, A., Judkins, D., & Morrison, C. (2017). Workforce Development Council of Seattle-King County Health Careers for All Program: Implementation and Early Impact Report (Report No. 2017-106). Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Reviewed by CLEAR

June 2022

Topic Area

Topic Area