Skip to main content
UNITED STATES DEPARTMENT OF LABOR

TAACCCT3 INTERFACE project: Final evaluation report. ( Smith et al. 2017)

  • Findings

    See findings section of this profile.

    Evidence Rating

Review Guidelines

Implementation Guidelines

Absence of conflict of interest.

Citation

Smith, L., Gregg, A., Roth, L., Stoeklen, P., Krueger, B., Lawton, G., & Knaeble, B. (2017). TAACCCT3 INTERFACE project: Final evaluation report. Menomonie, WI: University of Wisconsin-Stout, Applied Research Center, Wisconsin’s Polytechnic University.

Highlights

  • The study’s objective was to examine the implementation of the Intentional Networks Transforming Effective and Rigorous Facilitation of Assessment, Collaboration, and Education (INTERFACE) program, a project created by a Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant, which provided career pathways, enhanced instruction, and students supports through Wisconsin’s 16 technical community colleges.
  • The study design was mixed-method, using qualitative and quantitative data collected through phone meetings with project leadership, college site visits, consortium meetings, performance reporting, and annual progress surveys.
  • The study found that the program exceeded its projections in five areas: student enrollment, retention of students in their program of study, participants completing credit hours, participants earning a credential, and the number of employed participants at enrollment that received a wage increase. The study found the program was less successful meeting its further education and employment goals.
  • The implementation study of the 16 technical colleges included in the INTERFACE consortium was comprehensive its design with extensive data collection including site visits, interviews, surveys, and document review; however, one limitation was the lack of information regarding analytic techniques.
  • The embedded impact study was reviewed by CLEAR in August 2020.

Features of the Intervention

  • Type of organization: Technical colleges
  • Location/setting: Multi-site in Wisconsin
  • Population served and scale: Adults; Veterans or military; 4,962 unique participants
  • Industry focus: Information
  • Intervention activities: Career pathways; Student support services; Technology; Work-based learning
  • Organizational partnerships: Employers; Workforce Development Boards; State agencies
  • Cost: Not include

The INTERFACE program was funded by a Round 3 Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant from the U.S. Department of Labor. The program was designed to provide basic computer literacy and IT career pathway instruction to TAA-eligible workers, incumbent workers, veterans, and other students in Wisconsin’s 16 technical community colleges. The Wisconsin Technical College System (WTCS) collaborated with employers, workforce development systems, and state agencies. The program allowed colleges to create new credentials and curriculum, purchase equipment, hire staff, and provide new or expanded student supports in the areas of student outreach, academic advising, career-readiness support, academic, and non-academic student supports. The logic model includes inputs, activities, outputs, and outcomes at the societal, system, school, and student levels.

The INTERFACE program centered on 12 activities (or categories of related activities); these activities were: create a framework for aligning regional/state initiatives with a representative from each region, create a standardized computer literacy course, professional development, create and use mobile applications, create stacked/latticed credentials, work with four-year colleges on transferability, train students, have IT Expos, increase internships or on-the-job training opportunities, enhance and promote an IT career pathway website, create a continuous improvement system, and create a participant tracking system.

Features of the Study

The study design used both qualitative and quantitative data collection methods. Data were collected through phone meetings with project leadership, seven consortium meetings, performance reporting, annual progress surveys, and several additional surveys. Additionally, approximately 120 individuals (e.g., evaluation, workforce, and campus liaisons; college deans, faculty, and staff; student coaches and success specialists; and others) across the 16 sites were interviewed during site visits that occurred in 2014, 2015, and 2016. The authors reported on the 12 activities (or categories of related activities) listed above.

To assess fidelity to the program model, evaluators used a "progress rating scale" to measure program activities—rating activities as a super strength, strength, opportunity, or outstanding opportunity corresponding to, in order, exceeding goals, meeting goals, significant work accomplished although the goals were not met, and goals not met as planned. The authors assessed fidelity by assigning a rating to 12 activities (or categories of related activities) to denote if a program activity exceeded a goal, met a goal, if significant work was accomplished although the goal was not met, and if a goal was not met as planned.

Findings

Program Activities/Service Components

  • The study found that the program exceeded its projections in five areas: student enrollment, retention of students in their program of study, participants completing credit hours, participants earning a credential, and the number of employed participants at enrollment that received a wage increase. The program almost met its goal regarding the number of participants completing a TAACCCT-funded program (99%). The program did not meet its goal in three areas; 1% of participants enrolled in further education after completing their program, 3% were employed after completing their program, and 1% of participants retained employment after program completion.
  • The study found that adopting a standardized competency for a course on computer literacy was successful with the creation and use of a Basic IT Skills course (BITS) at all colleges.
  • The study found that the goal for all colleges to implement 1-3 recommendations in their program pathway was met at some of the colleges.
  • The study found that the goal to build, store, pilot, and use mobile apps was exceeded by 20% (target 375 and actual 454).
  • The study found that activities related to developing stacked and latticed credentials included developing 45 technical diplomas, 25 certificates, and 10 associate degrees; in all, 148 courses were developed or updated.

Fidelity

  • The study found that four goals were exceeded, five goals were met, and three were not met but significant progress was made. None were scored as not meeting the goal.

Implementation challenges and solutions

  • The study found that the program did not meet its goal in three areas; participants enrolled in further education after completing their program (1% of goal), participants employed after completing their program (3% of goal), and participants that retained employment after program completion (1% of goal).

Considerations for Interpreting the Findings

The strengths of the study design were the inclusion of all 16 technical colleges included in the INTERFACE consortium, multiple site visits, upwards of 150 interviews, several surveys, document reviews, meeting observations, and close collaboration (including technical support) provided by evaluators to try to ensure accurate data. Another key strength was the systematic linking of activities to research questions. A limitation of the study was that evaluators did not provide a description of their analytical methods, nor did they report on the validity or reliability of data collection instruments. Additionally, they noted that their fidelity measure was subjective. Fidelity was assessed by the study authors and not by the CLEAR team.

Reviewed by CLEAR

August 2-21

Topic Area