Citation
Scrivener, S., Weiss, M., & Sommo, C. (2012). What can a multifaceted program do for community college students? Early results from an evaluation of Accelerated Study in Associate Programs (ASAP) for developmental education students. New York: MDRC.
Highlights
- The study’s objective was to examine the impact of the Accelerated Study in Associate Programs (ASAP) on full-time enrollment, credits attempted, and credits earned.
- The study was a randomized controlled trial. Data sources included a baseline information form (BIF), which collected demographic and background characteristics and students’ records from the City University of New York (CUNY), including students’ transcripts and CUNY assessment test data.
- The study found that ASAP had significant impacts on full-time enrollment, credits attempted, and credits earned during the first, second, and third semesters following random assignment.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to ASAP, and not to other factors.
Intervention Examined
The Accelerated Study in Associate Program (ASAP)
Features of the Intervention
CUNY, in conjunction with the Center for Economic Opportunity, developed ASAP in 2007. The program aimed to support community college students seeking associate’s degrees by providing enhanced services. The program was implemented in all six CUNY community colleges and included the following components: (1) a requirement to attend college full time; (2) block-scheduled classes with other ASAP students for the first year of the program; (3) participation in an ASAP seminar that covers goal-setting, study skills, and academic planning for at least two semesters of the program; (4) free use of textbooks during the semester; (5) comprehensive advising from an ASAP adviser with a small caseload of 60 to 80 students; (6) tutoring; (7) career services; (8) a tuition waiver that covered any gap between a student’s financial aid and tuition and fees; and (9) free monthly MetroCards. ASAP encouraged participants to complete their developmental education early and graduate within three years. Therefore, ASAP included only students in majors whose requirements could reasonably be completed in three years. To be eligible for the program and evaluation beginning in 2010, students had to meet the following criteria: (1) low-income New York City resident; (2) in need of developmental courses based on CUNY Assessment Tests; (3) new student or continuing student who completed at most one full-time, 12-credit semester; (4) be willing to attend college full-time; and (5) in an ASAP-eligible major.
Features of the Study
The study was a randomized controlled trial, with 896 eligible students from three participating CUNY community colleges. Data for the analysis came from the BIF, which provided demographic and background characteristics, and CUNY students’ records, which provided transcript information from all colleges in the CUNY system as well as scores from CUNY assessment tests. Upon completing the BIF and consenting to participate in the study, eligible students were randomly assigned to the treatment group, which was offered the opportunity to participate in ASAP, or the control group, which could not participate in ASAP but could access standard community college services. Due to differences between the colleges’ academic calendars, the authors separated the participating colleges into two cohorts. The authors conducted random assignment during spring 2010 for Kingsborough Community College and LaGuardia Community College (cohort 1), and in fall 2010 for Borough of Manhattan Community College (cohort 2). This report compared outcomes for the treatment and control groups during the first and second semesters after random assignment for both cohorts. In addition, the authors reported outcomes for the third semester after random assignment for cohort 1. The authors estimated regression models comparing the outcomes of treatment and control group members, controlling for college and cohort.
Study Sites
- Kingsborough Community College, New York, NY
- Borough of Manhattan Community College, New York, NY
- LaGuardia Community College, New York, NY
Findings
- The study found that, at the end of the first program semester, ASAP group members in both cohorts were 10.6 percentage points more likely to be enrolled in school full-time and 8.1 percentage points less likely to be enrolled part-time compared with the control group. Both of these findings were statistically significant.
- Treatment group members attempted 2.2 more credits by the end of the first program semester than control group members. ASAP participants were also 14.7 percentage points more likely to have completed their developmental requirements at the end of the first semester relative to the control group. Both of these findings were statistically significant.
- After two semesters, ASAP group members were 20.6 percentage points more likely to be enrolled as full-time students, and attempted 1.7 more college-level credits, compared with the control group. Both findings were statistically significant.
- In the third program semester, the cohort 1 treatment group had significant increases in full-time enrollment, number of college-level credits attempted, and number of semesters enrolled, relative to the cohort 1 control group.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on outcomes related to progress toward degree completion. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to ASAP, and not to other factors.