Absence of conflict of interest.
Citation
Cosgrove & Associates. (2016). Third-party evaluation of implementation & impact of the Southwest Missouri Public Safety and Emergency Medical Initiative TAACCCT grant. St. Louis, MO: Cosgrove & Associates.
Highlights
- The study’s objective was to examine the impact of Crowder College’s enhanced Public Safety and Emergency Medical Initiative (PSP) program on education and employment outcomes.
- The study used a nonexperimental design to compare the outcomes of students who were in Crowder College’s enhanced PSP program to a comparison group of students who were not enrolled in the program.
- The study found that participation in the PSP program was significantly associated with higher program completion and employment rates than the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Crowder College’s enhanced PSP program; other factors are likely to have contributed.
Intervention Examined
Crowder College Public Safety and Emergency Medical Initiative (PSP) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Crowder College in Neosho, Missouri used their TAACCCT grant to expand the capacity of their Criminal Justice, Fire Science, and Emergency Medical Services courses within their Public Safety and Emergency Medical Initiative (PSP), and developed new instructional and student support strategies. They sought to expand and build programs to provide needed skills and credentials within the industry, improve achievement rates and/or time to completion, facilitate completion for low-skilled workers, implement work-based teaching strategies, and accommodate student work schedules. The program targeted TAACCCT-eligible individuals, those without previous college, academically low-skilled adults, and under or unemployed adults in the region in the in-demand areas of criminal justice, fire science, and emergency medical services.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who participated in PSP to students who did not participate. The authors assessed 272 treatment group participants who enrolled in TAACCCT-enhanced Criminal Justice, Fire Science, and Emergency Medical Services courses from 2013-2015 and compared them to a retrospective comparison group sample of 1,232 students who, during the same time period, did not take enhanced courses. Both cohorts of students were new to the college. Outcomes included completion rates and employment rates upon program completion. Using data from Crowder College's data systems, supplemental data collection tools developed by the PSP grant team and Cosgrove & Associates, and state unemployed insurance (UI) databases, the authors used statistical models to examine differences in the outcomes between the treatment and comparison groups.
Findings
Education and skills gain
- The study found that participation in the PSP program was significantly associated with higher program completion (13.92 times more likely to complete their program) than the comparison group.
Employment
- The study found that participation in the PSP program was significantly associated with higher employment (17.52 times more likely to be employed) than participation in the comparison group.
Considerations for Interpreting the Findings
The authors do not provide detail on which programs comparison group students are enrolled in; student self-selection into public safety courses versus other college courses may present a confound. In addition, the authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the economy) and not the intervention. Furthermore, the authors do not report baseline equivalence or controls for race/ethnicity, student’s pre-intervention degree of financial disadvantage, and student’s pre-intervention education/training or employment. These potential preexisting differences between the groups—and not PSP program—could explain the observed differences in the outcome. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Crowder College’s enhanced PSP program; other factors are likely to have contributed.