Absence of conflict of interest.
Citation
Bragg, D., Cosgrove, J., Cosgrove, M. & Blume, G. (2018). Final evaluation of the ACED grant at Salt Lake Community College. Salt Lake City, UT: Bragg & Associates, Inc.
Highlights
- The study’s objective was to evaluate the impact of a competency-based education (CBE) program offered by Salt Lake Community College on educational, employment, and earnings outcomes.
- The study used a nonexperimental design to compare outcomes among students enrolled in the CBE-based program and students who were not enrolled in CBE.
- The study found CBE program participation was associated with a statistically significant increase in post-program wages.
- The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Salt Lake Community College’s CBE Program; other factors are likely to have contributed.
Intervention Examined
Salt Lake Community College's CBE Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories. Utah’s Salt Lake Community College (SLCC) received TAACCCT funding to convert programs of study offered by its School of Applied Technology and Technology Studies (SATTS) to competency-based education (CBE) to increase student retention and increase or accelerate program completion.
SLCC’s CBE program is self-paced and its programs of study use a hybrid delivery approach that includes online learning and face-to-face instruction. Moreover, the program measures learning by the demonstration and completion of competencies rather than time to completion (i.e. credits or clock hours). The program categorized 24 programs of study into six career clusters that align to industry sectors important to the Salt Lake City region. The Test for Adult Basic Education (TABE) was used to place students into a program of study for CBE. Students enrolled in the CBE curriculum received student services where administrators tracked their progression on competencies and gave them academic and career advice. Transcripts for the CBE program delineated competencies mastered and not course names so employers could more easily understand prospective employees’ capabilities and skillset. Another feature of the CBE program is that depending on the program of study, students can obtain three levels of awards after program completion: certificate of achievement, certificate of proficiency, and certification of completion.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who were in the CBE program with students who were not. The authors matched CBE program participants to a retrospective sample of students enrolled in similar programs of study that were not enhanced by the CBE initiative using propensity scores developed from baseline demographic information. The final analytical sample was comprised of 1,448 study participants with 724 in the treatment and 724 in the comparison group. Data sources included intake forms, databases managed by various campus units at Salk Lake Community College, and the Utah Department of Workforce Services. The authors used statistical models with controls for demographic and programmatic characteristics to examine differences in outcomes between the groups. The outcomes of interest included program completion and post-program employment and wages.
Findings
Education and skills gain
- The study found no significant relationship between CBE program enrollment and program completion.
Earnings and wages
- The study found that CBE program enrollment was significantly associated with an average quarterly wage increase of $674.70 post-program.
- The study also found that CBE program enrollment was significantly associated with a higher quarterly wage for any quarter two years after program completion.
Employment
- The study found no significant relationship between CBE program enrollment and employment in the first quarter following program completion or any quarter two years after program completion.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to CBE program participants. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not Salt Lake Community College’s CBE program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Salt Lake Community College’s CBE Program; other factors are likely to have contributed.