Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the implementation of the Urban Alliance High School Internship Program, a program that offers skills training, internships, mentoring, coaching, and alumni services to help high school seniors in low-income areas pursue college or careers.
- The study authors conducted an implementation evaluation using data from a variety of sources including program staff, program participants, observations, and program documents including audited financial records and training materials.
- The study found that communication between job mentors and Urban Alliance coordinators was one key to success. Additionally, while it was difficult to recruit job sites, employers stayed involved for several years and appreciated the interns placed with them. Key challenges were large caseloads for program coordinators, not enough time to help youth plan for life after high school, program attrition, and organizational growth. Key achievements were the earnings, confidence, and hard and soft skills that youth received through their internship and other program activities.
- The key strength of the study was the wide variety of quantitative and qualitative data collected at the individual, household, school, neighborhood, and state levels. Additionally, researchers collected data from a variety of individuals with different connections to and perspectives on the program. One limitation was the lack of information regarding specific analytic techniques.
- The companion impact study was reviewed by CLEAR in June 2024 and can be found here: Pathways after high school: Evaluation of the Urban Alliance High School Internship Program (Theodos et al., 2017).
Intervention Examined
Urban Alliance High School Internship Program
Features of the Intervention
- Type of organization: Nonprofit
- Location: Multi-site
- Population served and scale: Youth, 544 program participants
- Industry focus: Not included
- Intervention activities: Training; Internships; Coaching; Mentoring
- Organizational partnerships: Employers; Schools
- Cost: Cost per participant $4,925 to $ 8,886
- Fidelity: Not included
The Urban Alliance High School Internship Program was started in Washington, DC in 1996, and expanded to Baltimore, MD in 2008, Chicago, IL in 2012, and Northern Virginia in 2013. At the time of the study, the organization had 42 employees and served over 1,500 students. The program offers a) training to prepare participants for their internships and workshops during their internships, b) paid internships, c) coaching and mentoring, and d) alumni services to encourage post-secondary education and provide job connections. Program coordinators manage caseloads of 30-35 interns, offering support through workshops, scheduled individual meetings, and additional support through email, phone, and face-to-face interactions. The internship program is funded by the internship sites and private foundations. It includes job sites that vary from small to large organizations, including nonprofit and government agencies. Most job duties are office or clerical in nature, overseen by a mentor who is an employee at the internship site.
The Urban Alliance High School Internship Program helps high school seniors in distressed communities. It targets average students who can leave school early to join internships from 2:00 to 5:00 PM. The program primarily targets students with a GPA of 2.5 or higher, but it is open to others as well. In Washington, DC, students learn about Urban Alliance internship through assemblies or in-class presentations. In Baltimore, school counselors or teachers refer students who also earn credit for their participation.
Urban Alliance’s logic model encompasses activities and outputs centered around four key program components: pre-work trainings, paid internships, post-high school coaching, and alumni services. Each component has specific metrics, such as 90% of youth completing 40 hours of training, 75% of interns attending 90% of workshop trainings, and 75% of interns developing a post-high school plan. Short-term outcomes include skillset growth, post-high school planning, and postsecondary achievements, each with corresponding metrics. Intermediate outcomes aim for 80% of alumni to be connected (employed or in school) one year after program completion, 70% to be connected two years post-program, and 70% to persist in college for at least one year. Long-term outcomes target 70% of students in four-year colleges to graduate within six years, 70% of students in two-year colleges to graduate within three years, and 80% of program alumni to earn a livable wage without relying on public benefits.
Features of the Study
The study included the sites in Washington, DC, and Baltimore, MD, with participants from 38 public or charter schools across these locations. After the study began, the program expanded to Chicago and Northern Virginia. Data on individuals came from program applications, the Urban Alliance case management system, biweekly wages paid to interns, and school administration records. Community data were obtained from the American Community Survey. Nearly 50 individuals (students, staff, job mentors, and school counselors) participated in interviews or focus groups. The evaluation team also collected audited financial records, observed 12 training or workshop sessions, and reviewed training materials and alumni newsletters. Interviews and focus groups were recorded with the participants' consent and then transcribed. The evaluation team analyzed the transcriptions using coding software.
Findings
Intervention Activities/Services
- The study found that youth in DC received about 24 hours of training, while those in Baltimore received about 32 hours. The training covered both hard and soft skills, with interactive sessions and real-world examples preferred.
- DC used large auditoriums, while Baltimore used smaller classrooms due to fewer internships. Both study sites had similar Friday workshops. Job site matches considered availability, interest, distance, personality, and mentor support needs.
- Despite some youth finding the training boring or repetitive, they believed it would be helpful for future jobs. Youth valued earnings (averaging $3,796 in 2013 dollars), access to job sites, mentor support, and skills development in communication and confidence. Employers valued the program for increasing diversity, providing supervisory experience, community engagement, and improving their reputation.
- Program coordinators communicated frequently with job site mentors through weekly emails, monthly calls, and on-site visits. They also had weekly communication with their caseload and worked with school staff for additional support.
Implementation Challenges and Solutions
- The study found that the Urban Alliance sets a 30–35-person caseload, but many coordinators prefer a smaller caseload of 20-25. Organizational growth made consistent program operation challenging. Coordinators noted difficulties in focusing on post-high school plans and addressing serious personal or home life issues for youth.
- Program youth faced challenges such as transportation costs, commute times, competing after-school activities, and the need for early release from school. Other issues included lack of motivation, personal or family problems, and relocation.
- The Urban Alliance invested in new executive-level positions, implemented Salesforce for case management, and developed new initiatives and internal evaluation capacity.
Cost/ROI
- The study found that the cost per participant was $4,925 for all youth who completed some pre-internship training. For those who completed the entire program, the cost per student was $8,866.
Considerations for Interpreting the Findings
A major strength of the study was the range of quantitative and qualitative data gathered at various levels, including individual, household, school, neighborhood, and state. Furthermore, researchers obtained insights from a diverse group of individuals with varying connections to and perspectives on the program. However, a notable limitation was the absence of detailed information on the specific analytic techniques used.