Absence of conflict of interest
Citation
Pacific Research & Evaluation, LLC. (2017). Final evaluation report Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant: Round 3 Kapiʻolani Community College. Portland, OR: Pacific Research & Evaluation, LLC.
Highlights
- The study’s objective was to assess the impact of the grant-funded Implementing an Island-wide Strategy for Industry Cluster Formation and Community Revitalization in Hawai'i (C3T) project on education outcomes.
- The authors used a nonexperimental design to compare the outcomes of students enrolled in C3T programs and courses with students enrolled in comparable programs offered at the same community college.
- When compared to nonparticipating students, the study found that C3T participation was significantly associated with higher program completion and retention rates for students enrolled in the hospitality program.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the C3T project; other factors are likely to have contributed.
Intervention Examined
The Implementing an Island-wide Strategy for Industry Cluster Formation and Community Revitalization in Hawai'i (C3T) Project
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
Kapi’olani Community College (KCC) in Hawai’i leveraged a TAACCCT grant to fund the Implementing an Island-wide Strategy for Industry Cluster Formation and Community Revitalization in Hawai'i (C3T) Project. This project created or modified six courses and programs offered at KCC by providing workforce development and training for the local culinary, hospitality, and health industries. These C3T courses and programs that were developed or enhanced offered additional certifications, pathways for further degree completion, or training that addressed local industry demand. Instructional and curricular modifications were made with the help of local industry partners and the project provided academic and career guidance to program participants in the form of job fairs, field trips, informal advising, and work-based learning opportunities. Moreover, these programs and courses implemented new delivery methods by leveraging technology that fostered distance learning and used new equipment to enhance the students’ learning experience.
Features of the Study
The study used a nonexperimental design to compare the outcomes of students who participated in C3T to those who did not. Students enrolled in the following C3T-enhanced programs or courses comprised the treatment cohorts: Hospitality Operations Management (HOST APC) program (one cohort - Fall 2016), HOST 101 course with gamification (two cohorts - Fall 2016 and Spring 2016), or School Health Aid (SHA) Level 1 training program (three cohorts - Summer 2014, Fall 2014, and Spring 2015). Each program or course in the treatment cohort were compared to a respective comparison cohort based on program characteristics (including learning objectives, credential attainment, and certification outcomes). Students who participated in comparable programs or courses that ran in tandem with those in the treatment cohorts were part of the comparison cohorts. Study participants included 294 students in the treatment group and 234 in the comparison group. The authors conducted statistical analyses to examine differences between the treatment and comparison groups in program completion, earning of credentials, and retention rate.
Findings
Education and skills gain
- The study found that there was a significant relationship between C3T participation and program completion for the HOST 101 gamified treatment cohorts. Specifically, program completion for these treatment cohorts was 18 and 21 percentage points higher than their respective comparison cohorts.
- The study found no significant relationship between C3T participation and program completion for all three of the SHA treatment cohorts or the HOST APC treatment cohort.
- The study found that there was a significant relationship between C3T participation and retention rates for the Fall 2016 HOST 101 gamified treatment cohort, with the retention rate being 30 percentage points higher for treatment students than comparison students. However, the study found no significant relationship between C3T participation and retention rates for the Spring 2016 HOST 101 gamified cohort.
- The study found no significant relationship between C3T participation and earning credentials.
Considerations for Interpreting the Findings
The authors did not account for other factors that could have affected the difference between the treatment and comparison cohorts prior to the intervention. No demographic information for the comparison cohorts is provided in the report. Thus, preexisting differences between the respective group pairings—and not the C3T-enhanced programs or courses—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the C3T project; other factors are likely to have contributed.