Absence of conflict of interest.
Citation
Webber (2018). Starting to succeed: The impact of CUNY Start on academic momentum. Retrieved from http://www1.cuny.edu/sites/cunystart/wp-content/uploads/sites/51/2018/08/gateway_brief_final.pdf
Highlights
- The study’s objective was to examine the impact of the CUNY Start program on community college students’ gateway course completion.
- The study used a nonexperimental design to compare the outcomes of students who participated in the CUNY Start program to a matched comparison group who did not participate in the program. Using program and institutional research data, the author conducted statistical models to examine differences between the groups.
- The study found that students participating in CUNY Start were significantly more likely to complete gateway courses than nonparticipating students.
- The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to CUNY Start, but other factors might also have contributed.
Intervention Examined
The CUNY Start Program
Features of the Intervention
In 2009, the City University of New York (CUNY) began implementing CUNY Start, a low-cost developmental education program for incoming community college students who are in need of remedial math, reading, and writing courses (as determined by their performance on the CUNY Assessment Tests). The purpose of the program is to reduce or eliminate remedial needs by delaying enrollment in a degree program for one semester. During that time, CUNY Start accelerates remedial coursework by providing intensive instruction in the areas in which developmental education is required. Students in the CUNY Start program also receive other services, including advising, tutoring, and a weekly seminar that teaches skills they need to succeed in college.
Features of the Study
The nonexperimental study was conducted within the CUNY public university system. The author matched CUNY Start participants to similar nonparticipants using propensity scores developed from baseline demographic information. Study participants included 11,198 first-time freshmen requiring remedial education, with 5,599 students in both the treatment and comparison groups. Using data from the CUNY Start Database, the Institutional Research Database, and the CUNY Application System, the authors conducted statistical models to examine differences in gateway (entry-level) course completion rates (measured as course passing rates) between the groups.
Findings
Education and skills gain
- The study found that gateway course completion rates were significantly higher in CUNY Start students compared to nonparticipating students. The difference in gateway math course completion was 9.1 percentage points in the first year and 10.6 percentage points two years after enrollment. The difference in gateway English course completion was 7.6 percentage points at both the one-year and two-year time periods. For both courses, the difference in course completion was 7.0 percentage points in the first year and 9.8 percentage points two years after enrollment.
Considerations for Interpreting the Findings
The author noted large dissimilarities between the two groups across observable variables at baseline, therefore using a nonexperimental design with propensity score matching. Although propensity score matching can result in a reduction of bias, this methodology is not as strong as a randomized controlled trial. It is uncertain whether the analysis could fully account for large preexisting differences between the groups. These preexisting differences between the groups—and not CUNY Start—could explain the observed differences in outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this study is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to CUNY Start, but other factors might also have contributed.