Citation
Scrivener, S., Bloom, D., LeBlanc, A., Paxson, C., Rouse, C.E., & Sommo, C. (2008). A good start: Two-year effects of a freshman learning community program at Kingsborough Community College. New York: MDRC.
Highlights
- The study’s objective was to examine the impact of Opening Doors learning communities at Kingsborough Community College in New York on progress toward completing a degree and enrollment in a four-year college in the program semester and three follow-up semesters. Students in the Opening Doors learning communities took three linked courses during one semester and were provided tutoring and case management services.
- The study was a randomized controlled trial. Eligible students were randomly assigned to either the treatment group, which was offered the opportunity to participate in a learning community, or the control group, which was not allowed to enroll in a learning community. The primary data sources were a baseline survey on the background characteristics of students, student transcripts, and degree attainment information from the National Student Clearinghouse.
- The study found that the treatment group earned significantly more credits than the control group, cumulatively across the program semester and the three follow-up semesters,. However, during that time, there were no significant cumulative differences between the two groups on registration for any courses, the number of semesters registered, or enrollment in either a two- or four-year college.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects would be attributable to the learning communities, and not to other factors.
Intervention Examined
Opening Doors Learning Communities at Kingsborough Community College
Features of the Intervention
At Kingsborough Community College in Brooklyn, New York, the Opening Doors learning communities program involved enrolling small cohorts of students in three linked courses during one semester. These learning communities involved an English course, a college-level course required for a specific major, and a one-credit freshman orientation class. The freshman orientation class was a seminar designed to reinforce the learning objectives of the other two courses and provide students practical information about careers in their selected majors. The instructor of this course also served as a case manager for the students in the learning community, meeting with faculty members to ensure the success of learning community students. In addition, each learning community member had a tutor who provided students with one-on-one tutoring; learning community students were also provided with $225 textbook vouchers. This study focused on four cohorts of students who participated in learning communities from fall 2003 to spring 2005.
To participate in the Opening Doors learning communities, students had to be first-time incoming freshman ages 17 to 34 who planned to attend college full-time during the day, and be enrolled in either a developmental or college-level English course. Although not a specific requirement for eligibility throughout program implementation, the program targeted low- and moderate-income students.
Features of the Study
This study was a randomized controlled trial. Randomization occurred at the student level, separately for each cohort. Eligible students who consented to participate in the study filled out a baseline data form. Then, they were randomized to either the treatment group, which was offered the opportunity to enroll in a learning community, or the control group, which could not enroll in a learning community, but could access existing college services. There were 1,534 students in the study with 769 in the treatment group and 765 in the control group.
The authors used data from the baseline data form, student transcripts, and the National Student Clearinghouse for outcomes related to progress toward degree completion. Impacts were reported as the difference of means between the treatment and control groups, adjusted for cohort and the number of English tests passed at baseline. This study reported outcomes for four semesters: the program semester and three subsequent follow-up semesters.
Findings
- The study found that the treatment group earned 2.4 more credits than the control group cumulatively across the program semester and the three follow-up semesters, which was a significant difference. However, in each individual follow-up semester, there were no significant differences between the groups on credits earned.
- Across the program semester and three follow-up semesters, there were no significant cumulative differences between the two groups on registration for any classes, the number of semesters registered, enrollment in any college, or enrollment in any four-year college.
Considerations for Interpreting the Findings
Kingsborough Community College provided extensive services to all students, so those in the control group might have had access to more services than a typical community college student at other schools. This could have attenuated the differences in service receipt between the treatment and control group students. Moreover, the authors noted that midway through the study period, Kingsborough opened a new academic advising center, which was aimed to improve the quality of academic advising across the college. These factors that are specific to Kingsborough might have decreased the ability to detect large program impacts over time.
The study authors performed multiple statistical tests on related outcomes in the progress toward degree completion domain, which makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated. Nevertheless, most differences are not considered statistically significant by CLEAR standards.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects would be attributable to the Opening Doors learning communities, and not to other factors.