Absence of conflict of interest.
Citation
Pacific Research and Evaluation. (2016). Final evaluation report Trade Adjustment Assistance Community College and Career Training (TAACCCT) Grant: Round 2 CyberSecurity Career Pathways Program. Portland, OR: Pacific Research & Evaluation, LLC.
Highlights
- The study’s objective was to examine the impact of the CyberSecurity Career Pathways Program (CSCPP) on education, earnings, and employment outcomes.
- The authors used a nonexperimental design to compare the outcomes of students who were in the CSCPP to a historical comparison group.
- When compared to a historical cohort, the study found a significant relationship between CSCPP participation and high rates of program completion and retention, and wage increases.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation or include sufficient control variables. This means we are not confident that the estimated effects are attributable to CSCPP; other factors are likely to have contributed.
Intervention Examined
The CyberSecurity Career Pathways Program (CSCPP)
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The CyberSecurity Career Pathways Program (CSCPP) was a TAACCCT-funded program at Utah Valley University in Orem, Utah, created to fill local industry and training gaps in the area. Efforts of CSCPP focused on developing and modifying courses and programs in information systems and technology from entry-level to post-baccalaureate certificates, and Associate (AS, AAS) and Bachelor (BS) degree programs. This included AS in Information Technology, AAS in Information Technology, BS in Information Technology, and BS in Information Systems. The CSCPP also enhanced student support services (e.g., advising, mentoring, tutoring, funding for industry certification exams, career guidance), and strengthened partnerships with local industry, including formation of an advisory board.
Features of the Study
The nonexperimental study compared the outcomes of students who were in CSCPP to a historical comparison group. The treatment group included 288 students enrolled in the four CSCPP credential programs during the first year of the grant (2012-13 academic year). The comparison group included 291 students enrolled in one of the same four credentials in the 2008-09 academic year, prior to grant implementation. The authors examined program completion, program retention, credentials earned, wage increases, and employment attained and retained. Using institutional research data from Utah Valley University and data from the Utah Department of Workforce Services, the authors conducted chi-square analyses to examine differences between the groups, but did not control for differences in student characteristics.
Findings
Education and skills gain
- The study found a significant relationship between CSCPP participation and program completion, with higher program completion rates in the treatment group (42%) than the comparison group (30%).
- The study also found that CSCPP participation was significantly associated with retention rates, with higher retention rates in the treatment group (28%) than the comparison group (16%).
- The study found no statistically significant relationship between participation in CSCPP and credentials earned.
Earnings and wages
- The study found a significant relationship between CSCPP participation and wage increases for incumbent workers, with higher proportions of the treatment group receiving wage increases (79%) than the comparison group (70%).
Employment
- The study found no significant relationships between participation in CSCPP and entering or retaining employment.
Considerations for Interpreting the Findings
The authors did not account for preexisting differences between the groups before program participation or include sufficient control variables. These preexisting differences between the groups—and not CSCPP—could explain the observed differences in outcomes. The authors also used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the economy) and not the program. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation or include sufficient control variables. This means we are not confident that the estimated effects are attributable to CSCPP; other factors are likely to have contributed.