Absence of conflict of interest.
Citation
Dockery, J., Bottomley, M., Murray, C., Tichnell, T., Stover, S., Schroeder, N., Hance, D., Fruchey, K., & Franco, S. (2018). Northwest State Community College of Ohio Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) Initiative (TAACCCT Grant TC-26481-14-60-A-39). Dayton, OH: Wright State University, Applied Policy Research Institute.
Highlights
- The study’s objective was to examine the impact of the Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) initiative on education outcomes.
- The authors used a nonexperimental design to compare education outcomes of students enrolled in the IAM iSTAR program to a historic cohort of students enrolled in the same programs of study prior to the implementation of the IAM iSTAR program.
- The study found that IAM iSTAR program participation was significantly associated with higher numbers of credit hours attempted and completed per semester.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the IAM iSTAR initiative; other factors are likely to have contributed.
Intervention Examined
Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) Initiative
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Northwest State Community College (NSCC) in Ohio used TAACCCT funding to implement the Industrial Automation Manufacturing innovative Strategic Training Achieving Results (IAM iSTAR) program. The grant funds were awarded to the college to assist in the conversion of traditional industrial technology courses to an intensive modularized hybrid format. The industrial technologies division at NSCC trains industrial automation maintenance technicians in the advanced manufacturing industry. The IAM iSTAR program included several components and strategies which included: developing new and modified competency-based curriculum, transforming the academic process to accelerate student learning and completion (e.g., virtual trainers), creating innovative instructional media and technology to accelerate learning (e.g., online classes and open labs), and redesigning student support services to increase student success (e.g., career coaching).
Features of the Study
The study was a nonexperimental design to compare the education outcomes of students enrolled in the IAM iSTAR program to a historic cohort of students attending NSCC. The treatment group consisted of 589 NSCC students who were enrolled in the IAM iSTAR program. The comparison group consisted of 233 students enrolled in the same programs of study between fall 2014 and spring 2016. Data sources included the NSCC Banner System and self-reported data for the treatment group and the Higher Education Information (HEI) records for the comparison group. The authors conducted repeated measures ANOVAs to examine differences in education outcomes (credit hours attempted and credit hours completed) between the treatment and comparison groups.
Findings
Education and skills gain
- The study found a significant relationship between the IAM iSTAR program and credit hours attempted, with treatment students attempting more credit hours each semester than students in the comparison group.
- The study also found a significant relationship between the IAM iSTAR program and credit hours completed, with treatment students completing more credit hours than students in the comparison group.
Considerations for Interpreting the Findings
The authors used a historical cohort of NSCC students as the comparison group. However, these students were not matched to treatment students, and control variables were not included in the analyses to account for preexisting differences between the groups. These preexisting differences between the groups—and not the IAM iSTAR program—could explain the observed differences in outcomes. Additionally, because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the community college) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the IAM iSTAR program; other factors are likely to have contributed.