Citation
Walsh, S., Goldsmith, D., Abe, Y., & Cann, A. (2000). Evaluation of the Center for Employment Training replication sites: Interim report. Oakland, CA: Berkeley Policy Associates.
Highlights
- This report presents an implementation analysis of 12 Center for Employment Training (CET) replication sites.
- The Center for Employment Training (CET) model targets economically disadvantaged youth who are not in school or employed. It delivers training in a work-like environment, requires full-time participation, and involves local employers in program design and delivery.
- The analysis assessed the fidelity of implementation to the CET model using qualitative data collected during site visits to 12 sites and preliminary administrative data from 8 sites.
- The study found that all the replication sites were generally successful in structuring services to mirror the workplace. Sites operated by CET (8 of 12) had overall higher fidelity to the model than non-CET-operated sites. Additionally, the study suggested that organizational capacity and stability are critical to replication; the 7 replication sites led by less-experienced organizations were prone to upheavals, and 4 closed during the course of the evaluation.
Intervention Examined
The Center for Employment Training (CET) Replication
Features of the Intervention
The CET program targets disadvantaged, out-of-school youth ages 18 to 21. CET was started in San Jose, California, in 1967, and two randomized controlled trials conducted in the late 1980s confirmed that CET-SanJose produced positive results for participating youth. By 2000, the program had expanded to 35 sites in 14 states. The U.S. Department of Labor (DOL) initiated a replication study to determine whether the CET-San Jose model could be implemented successfully in other settings. The CET replication sites agreed to participate in the replication study and the national CET office in San Jose provided each site with technical assistance.
The CET model has four core features. First, it provides occupational and basic skills training in a work-like setting, and trainees advance at their own pace. Second, it requires a full-time commitment from students, to accustom them to a work schedule. Third, it maintains close connections to employers, who help design and implement the training programs. Fourth, it has an extensive orientation process, during which the rigor of the program is emphasized; less motivated individuals may drop out at that point.
Features of the Study
The evaluation of the CET replication included both an implementation and impact study. The interim report profiled here focused on whether the replication sites had faithfully implemented the CET-San Jose model and how they adapted it to local circumstances. Of the 12 participating sites, 6 were in Eastern and Mid-Western sites that participated in the study through a DOL grant application process: Orlando, FL; Chicago, IL; Reidsville, NC; Camden, NJ; Newark, New Jersey; and New York, NY. Six were Western sites randomly selected from within CET’s existing network of programs: El Centro, Oxnard, Riverside, San Franciso, and Santa Maria in California and Reno in Nevada.
Data sources for the implementation study include (1) semistructured interviews with site directors, training instructors, support counselors, job developers, and participating youth during two rounds of site visits; (2) observations of training activities; (3) written materials that describe program operations and background; and, (4) preliminary participation data from CET’s management information system for the CET sites covering closed cases through June 30, 1999.
Findings
The study found that CET replication sites differed greatly from one another and from the original CET-San Jose site on contextual elements such as unemployment rates, racial and ethnic composition of the local population, and population density. Five of the 12 sites had highly stable organizational capacity based on funding and staffing. These organizations had been in place for at least 20 years at the time of the study and had developed close ties to funders and employers within their local communities to weather challenges. Replication sites led by less-stable organizations were prone to upheavals, and 4 of the 7 closed during the course of the evaluation.
Replication sites were generally faithful to the CET model in structuring services to mirror a work-like environment and the design of services. Although 10 of the sites adopted curricula that required intensive participation—a key aspect of the program model—eight of the 12 sites were rated as having low fidelity to this aspect of the program model, largely due to high program dropout rates. Only half of the sites successfully involved industry representatives. In terms of overall fidelity to the CET model, the study found that CET-operated sites (8 of 12) had higher fidelity than sites not operated by CET, and sites that had been operating CET longer tended to be more faithful to the model.
Based on the preliminary administrative data, of 365 youth randomly assigned to CET in 8 of the 12 sites before June 30, 1998, 68 percent enrolled in training. On average, youth enrolled in training received 630 hours of training over 26.6 weeks (or 186 days). For all program group members (including those who did not enroll in training), the average hours of training were in the range of 428 to 510 hours. The amount of training received by program youth compared favorably with other large-scale youth training demonstration programs and with mainstream Job Training Partnership Act programs; however, program youth received less training than that experienced in the original CET-San Jose site.
Based on the qualitative analysis, many sites, particularly the less mature sites, had difficulty finding and maintaining full-time staff positions for job developers, making the job placement assistance element of the model one of the more difficult to replicate successfully. And, although most sites implemented programs consistent with the major elements of CET, some programs could not be sustained. The authors suggested that future attempts to replicate the CET model should consider organizational capacity and stability as critical factors affecting program sustainability and consider providing more extensive financial support.
Considerations for Interpreting the Findings
The reporting of qualitative information by respondents and the definitions used by researchers were somewhat subjective. It is possible that different definitions of the measures could produce different results. The study appropriately acknowledged that variation in the selection of the CET sites produced a sample split between early implementation sites and steady-state, mature programs that inhibited the ability to make generalizations across the study sites as a whole.