Absence of conflict of interest.
Citation
Highlights
- The study's objective was to examine the impact of the Southern California Promise Program (SCPP) on credit attainment and degree completion.
- The study used a nonexperimental design to compare the outcomes of the SCPP participants to a matched comparison group of non-participants. Using administrative data, the author conducted statistical tests to compare differences in outcomes between the groups.
- The study found that the SCPP was significantly related to more credits earned and higher rates of degree completion.
- This study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to the Southern California Promise Program; other factors are likely to have contributed.
Intervention Examined
Southern California Promise Program
Features of the Intervention
In 2015, President Obama launched the College Promise program to increase investment in community colleges. The state of California's community college districts used this funding to increase student services and financial aid. The Southern California Community College District implemented the Southern California Promise Program (SCPP) in the Fall of 2016 for first-time college students who had completed the Free Application for Federal Student Aid (FAFSA) or Dream Act application. The primary goal of the SCPP was to help students complete their degree or certificate or transfer to a four-year university in two years. The SCPP included a comprehensive onboarding process for students before college, featuring a one to three day on-campus orientation. This orientation informed students about available programs and student services, offered campus tours, and explained program requirements. At orientation, students were paired with peer mentors and academic counselors, met their Promise Coordinators, and signed contracts to commit to the program.
The SCPP paired new students with peer mentors to help them during their first year. This process connected experienced students with newcomers, offering a source for guidance and support. Participants were expected to meet with their mentors monthly. The program created cohorts of students to cultivate community and belonging. The SCPP also provided supportive funding for books and other enrollment fees. Finally, program participants were eligible to receive academic support during their time at the community colleges.
To participate in the SCPP, students had to be recent high school graduates, first-time college students, and a resident of the state of California. Program participants were required to enroll full time (a minimum of 12 credits) and attain a GPA of 2.0 or higher.
Features of the Study
The study used a nonexperimental design with a matched comparison group to evaluate the impact of the SCPP on education outcomes. The author matched students admitted into the SCPP with non-participating students on three variables: first-time college status, full-time enrollment status, and residency status. The study sample included 9,909 students across the Southern California Community College District. There were 186 participants in the SCPP in Cohort 1 (students who entered in Fall 2016); 543 participants in Cohort 2 (students who entered in Fall 2017); and 1,712 participants in Cohort 3 (students who entered in Fall 2018). Across the three cohorts, the SCPP group had a higher proportion of females and Filipino and Latinx students than the comparison group. In the SCPP cohorts, 39% of students were first generation college students and 76% received financial aid. The study used administrative data from the community colleges on student outcomes and program engagement. The author used statistical tests to estimate the impact of the SCPP on credit attainment (all cohorts) and degree completion (Cohorts 1 and 2 only).
Findings
Education and skills gains
- The study found that SCPP participation was significantly associated with more credits earned for students in Cohorts 1 and 3. No significant relationship was found for Cohort 2.
- The study found that SCPP participation was significantly associated with more credits earned for racially and ethnically diverse students in Cohorts 1 and 3. No significant relationship was found for Cohort 2.
- The study also found that SCPP participation was significantly associated with higher rates of degree completion for students in Cohort 1, including racially and ethnically diverse students. No significant relationships were found for Cohort 2.
Considerations for Interpreting the Findings
The author matched the treatment and comparison groups on first-time college status, full-time enrollment status, and residency status. The author conducted separate analyses by socioeconomic status and race/ethnicity and stated that the groups were similar. However, the author did not match on gender, age, or a preintervention outcome measure (such as GPA) or control for them in the outcome analyses as required by CLEAR. These preexisting differences between the groups—and not the Southern California Promise Program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Southern California Promise Program; other factors are likely to have contributed.