Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the implementation of the Illinois Network for Advanced Manufacturing (INAM) across 21 two-year colleges in Illinois.
- The study authors conducted an implementation evaluation of how INAM built its institutional capacity, developed partnerships, and maintained fidelity to its original design over the course of the grant. Data sources included student surveys pre- and post-INAM, site visits, key stakeholder interviews and focus groups (including employers, project directors, faculty and staff, and students), a project director survey, and expert reviews of course designs.
- The study found that INAM colleges were generally able to strengthen their advanced manufacturing programs through the implementation period, though some more than others. INAM followed its original plan with fidelity for the most part but dropped plans for a green manufacturing program. Further, internships, prior learning credits, and the educational planning process were implemented to a lesser extent than anticipated.
- The implementation study was comprehensive in its design, data collection, and analyses. Data support implementation findings, although evaluators note biased recruitment of study participants.
- The embedded impact study was reviewed by CLEAR in May 2020.
Intervention Examined
The Illinois Network for Advanced Manufacturing (INAM)
Features of the Intervention
- Type of organization: Two-year colleges
- Location/setting: Multi-site in Illinois
- Population served and scale: Adults, dislocated worker, veteran or military, unemployed; 2,655 participants
- Industry focus: Manufacturing
- Intervention activities: Career pathways; Student support services; Work-based learning
- Organizational partnerships: Employers
- Cost: Not included
- Fidelity: Not included
The Illinois Network for Advanced Manufacturing (INAM) was developed using the U.S. Department of Labor’s (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program funds. It consisted of 21 colleges throughout the state who came together to promote and provide training in the field of advanced manufacturing, leading to industry-recognized certificates or associate degrees that prepared participants for employment in high wage, high skill occupations that meet employer needs. The intervention targeted TAA-eligible workers, veterans, incumbent workers, and the unemployed, but accepted all students who applied. Each INAM college offered between one and four of the five INAM programs (Certified Production Technician, Industrial Maintenance, Mechatronics, Precision Machining Computer Numerical Control (CNC), and Welding/Metal Working). The intervention included supports to facilitate student completion (including a process for helping students create and follow an education plan). Students could also receive credit based on their prior learning experience. Upon completion of their INAM program, students were supported through placement in internships and on-the-job training opportunities, stacked credentials, articulation agreements with four-year colleges, and job placement. This intervention was implemented from summer 2013-fall 2015 and served 2,655 unique participants.
According to the logic model, critical inputs included the INAM colleges and grant funding. Additional inputs included partnerships with local industry, existing college curricula in advanced manufacturing, and student participants. Outputs included the number of new degrees and certificates, employer feedback sessions, articulation agreements, internships secured, and tools and equipment purchased. Short-term, intermediate, and long-term outcomes were listed for the region, the INAM institutions, and students. They included reduced unemployment, increased productive capacity, reduced skills gaps, improved curricula and knowledge sharing, improved student retention, earned credentials and certificates, increased wages, and employment.
Features of the Study
The study design included interviews, focus groups, surveys, and site visits. Key stakeholders, including project directors, faculty, students, and employers, were included in the study sample. A total of 27 project directors, 14 education planners, 63 faculty, and 163 students participated in focus groups and interviews. The authors also spoke to 25 employers. The participating employers were all in the field of advanced manufacturing and had some involvement with INAM. The evaluators analyzed focus group and interview data using standard qualitative analysis procedures that included transcribing each audio recording, developing a coding scheme, and reviewing the coding scheme on an ongoing basis and adjusting when necessary. They also reviewed the coded data and summarized responses to each question in the protocol, including quotes when relevant. Additionally, the evaluators hired outside content experts in each of the five INAM areas to review syllabi/curricula for each INAM program and college and provided detailed critiques. The content experts reviewed both the curricula (in terms of their usefulness as a national model) and the INAM learning objectives.
Findings
Intervention activities/services
- The study found that INAM colleges were able to build their institutional capacity by restructuring curricula, increasing advanced manufacturing program offerings, acquiring new equipment, developing relationships with faculty at other colleges and local businesses, and changing course sequences (e.g., stacked and latticed earned credentials, coordination with national credentialing opportunities, new articulation agreements with four-year colleges) as a result of grant implementation.
- The study found that many INAM colleges leveraged relationships with local employers and businesses and were able to learn from and cooperate with larger, more experienced INAM programs.
- The study found that INAM colleges varied greatly in the extent to which they implemented two key support strategies for students: (1) development of an education plan, and (2) a system to award credit for prior learning experience.
- The study found that students and faculty shared a range of opinions about the success of the educational planning process, and many needed to be reminded of it or were unaware of it altogether.
- The study found that the actual use of prior learning credits and prior learning assessments to help students accelerate program completion was lower than expected.
Implementation challenges and solutions
- The study found that certain partnerships with employment centers never manifested due to funding constraints at the employment centers. Further, INAM had difficulty recruiting target populations, such as veterans and dislocated workers. Some study participants attributed this difficulty to insufficient funds for advertising. Nevertheless, INAM exceeded its enrollment targets through several innovative recruitment strategies (creation of special events such as open houses, tours, community fairs, informational meetings; flyers, websites, TV, radio).
- The study found that the creation and use of internships was underutilized, often for reasons outside of INAM’s control (such as the condition of the economy and employers’ concerns about liability issues).
- The study found that students expressed a need for greater access to instructors/aides, particularly during lab time, when they spend a lot of class time waiting for an instructor for assistance.
- The study found that project directors and faculty members experienced challenges of response burden related to paperwork, reporting, maintaining the student database, and responding to requests for data from the INAM central office.
Considerations for Interpreting the Findings
The implementation study was comprehensive in its design, data collection, and analyses. The findings did not explicitly align with the research questions, but they were supported by the data. The authors did note limitations around recruitment of study participants, acknowledging potential bias due to, in some cases, participants being selected by project directors or based on convenience.