Absence of conflict of interest.
Citation
NC State Industry Expansion Solutions. (2017). Final evaluation report Round 3 TAACCCT grant: Mission critical operations. Raleigh, NC: NC State Industry Expansion Solutions.
Highlights
- The study’s objective was to evaluate the effects of the National Consortium for Mission Critical Operations (MCO) program on educational outcomes. This summary contains the findings from Cleveland Community College.
- The authors used a nonexperimental design to compare the outcomes of students enrolled in the MCO program to a matched comparison group of students enrolled in the same courses of study prior to the implementation of the MCO program.
- The study found that participation in the MCO program was significantly associated with a higher retention rate, but lower rates of program progress.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MCO program; other factors are likely to have contributed.
Intervention Examined
The Mission Critical Operations (MCO) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Mission Critical Operations (MCO) program was funded by the TAACCCT grant to build institutional capacity to increase credentials in a range of industries focusing on electrical/electronic, industrial, and information technology in response to the disappearance of traditional manufacturing and skilled service jobs. The MCO program had three main objectives: 1) to better prepare TAA eligible and other adults for high-skill, high wage employment or re-employment by increasing the attainment of certificates, diplomas, and other industry related credentials; 2) to improve learning outcomes and retention rates for TAA workers and other adults by redesigning curriculum with innovative and effective methods to address specific industry needs; and 3) demonstrate improved employment outcomes, for TAA workers in particular. A consortium of colleges was formed to implement and monitor the MCO program. The National Consortium for Mission Critical Operations consisted of the following five colleges: Cleveland Community College, Nash Community College, Wake Technical Community College, Moultrie Technical College, and the University of North Carolina at Charlotte. Each college in the consortium implemented various intervention activities, including articulation, collaboration, course design, credential development, instruction/support, scorecard development, telepresence, and tracking/evaluation.
Features of the Study
The nonexperimental study took place at Cleveland Community College in Shelby, North Carolina. The authors compared the education outcomes of students who enrolled in the MCO program to students enrolled in the same courses of study prior to the implementation of the MCO program. The treatment group consisted of 1,259 students who participated in the MCO program after Fall 2014. The comparison group included 2,008 students enrolled between Fall 2010 and Summer 2013, in the same courses that were later impacted by the MCO funds. The authors matched MCO participants to similar nonparticipants using propensity scores developed from baseline demographic information. Using data from Cleveland Community College’s Student Information System, the authors conducted statistical models and analyses to examine differences in education outcomes between the groups. Education outcomes included retention rates, program progress (measured as earning grades of A, B, or C in courses), and program completion.
Findings
Education and skills gain
- The study found a significant relationship between MCO program participation and retention rates, with MCO participants twice as likely to be retained than students in the comparison group.
- However, the study found a significant association between MCO program participation and lower rates of program progress than students in the comparison group.
- The study did not find a significant association between MCO program participation and program completion.
Considerations for Interpreting the Findings
The authors used propensity score matching to create a comparison group; however, they did not account for preexisting differences between the groups in financial disadvantage or baseline education outcomes. These preexisting differences between the groups—and not the MCO program—could explain the observed differences in outcomes. In addition, the authors used a cohort from previous enrollment years as the comparison group. Because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the MCO program; other factors are likely to have contributed.