Citation
Richburg-Hayes, L., Brock, T., LeBlanc, A., Paxson, C., Rouse, C.E., & Barrow, L. (2009). Rewarding persistence: Effects of a performance-based scholarship program for low-income parents. New York: MDRC.
Highlights
- The study’s objective was to examine the impact of the Opening Doors program at Delgado Community College and Louisiana Technical College-West Jefferson in New Orleans, Louisiana, on progress toward completing a college degree. The Opening Doors program offered students with dependent children a performance-based $1,000 scholarship in each of two semesters. The study examined both the shorter- and longer-term impacts of the Opening Doors program using different cohorts of students.
- The study was a randomized controlled trial. Eligible students were randomly assigned to either the treatment group, which was offered the opportunity to receive the scholarship, or the control group, which was not eligible for the scholarship. The primary data sources were a baseline survey on the background characteristics of students, student transcripts, and degree attainment information from the National Student Clearinghouse.
- The study found that for all four cohorts of students, the treatment group was significantly more likely to have registered for any course and have enrolled full-time in college, and to have attempted and earned significantly more credits in both the first and second program semesters, compared with the control group. The study found that for the first two cohorts of students, the treatment group registered for classes at a higher rate, enrolled in more semesters of college, and earned more credits than the control group cumulatively through the fourth semester after random assignment.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Opening Doors program, and not to other factors.
Intervention Examined
The Opening Doors Program at Delgado Community College and Louisiana Technical College-West Jefferson
Features of the Intervention
At Delgado Community College and Louisiana Technical College-West Jefferson in New Orleans, Louisiana, the Opening Doors program involved a performance-based scholarship of $1,000 that participants were eligible to receive in each of two semesters. The scholarship was payable directly to the students in three installments each semester: $250 for enrolling in at least six credit hours; $250 after completing midterms, having passing grades in all classes, and staying enrolled in at least six credit hours; and $500 for completing six or more credit hours with a C average or better. Program counselors monitored students’ performance at each payment benchmark and physically handed checks to qualified students at the beginning, middle, and end of each semester. Students could earn the scholarship in two semesters, which could be back-to-back or nonconsecutive semesters. The scholarship was offered in addition to Pell Grant and other financial aid that the students might have been receiving, and did not substitute for those sources. The Opening Doors program enrolled students in the spring 2004, summer 2004, fall 2004, and spring 2005 terms.
To be eligible to participate in the Opening Doors program, students had to be ages 18 to 34, be parents with at least one dependent child younger than 19 years old, have a household income less than 200 percent of the federal poverty level, hold a high school diploma or a general educational development certificate or have passed all college entrance exams, and not already have earned a college degree or certificate.
Features of the Study
This study was a randomized controlled trial. Randomization occurred at the student level, separately for each cohort. Eligible students who consented to participate in the study filled out a baseline data form. Then, the students were randomized either to the treatment group, which was offered the opportunity to participate in the Opening Doors program, or the control group, which could not participate in the program, but could access existing college services.
The study reported shorter-term outcomes from the two program semesters for the full sample of all four student cohorts: spring 2004, summer 2004, fall 2004, and spring 2005. In total, there were 1,019 students in the four cohorts, with 505 in the treatment group and 514 in the control group. For the first two cohorts of students, which were enrolled in spring 2004 and summer 2004, the study also reported longer-term outcomes for four semesters: the two program semesters and two subsequent follow-up semesters. There were 537 students in the first two cohorts, with 264 in the treatment group and 273 in the control group.
The authors used data from the baseline data form, students’ transcripts, and the National Student Clearinghouse for outcomes related to progress toward degree completion. Impacts were reported as the difference of means between the treatment and control groups, adjusted for cohort and school campus.
Findings
- For the first two cohorts, the study found that, cumulatively, across the two program semesters and two follow-up semesters, the treatment group was 6.5 percentage points more likely to be registered for any courses than the control group The treatment group enrolled in 0.4 more semesters of college and earned 3.5 more credits than the control group cumulatively across the four semesters. All of these differences were statistically significant.
- For all four cohorts, the study found that the treatment group was 5.3 percentage points more likely to register for any course in the first program semester and 15.0 percentage points more likely to do so in the second program semester. The study also found that the treatment group was 6.4 percentage points more likely to enroll in college in the first program semester and 15.3 percentage points more likely to enroll in college in the second program semester, compared with the control group. The treatment group attempted 0.6 more credits in the first program semester and 1.2 more credits in the second program semester, compared with the control group. The treatment group earned 1.2 more total credits in the first semester and 1.1 more total credits in the second program semester, compared with the control group. These differences were statistically significant.
Considerations for Interpreting the Findings
When Hurricane Katrina struck in August of 2005, both colleges closed temporarily and many students moved away, which shortened available follow-up. Although the authors did report some post-Katrina outcomes, this review does not focus on those findings given the difficulties in measuring program impacts in the aftermath of a natural disaster.
Program implementation differed in the summer semesters compared with the regular school year semesters at Delgado. In summer 2004, there were only two scholarship payment installments, rather than three; in summer 2005, students were offered a $500 scholarship if they completed one three-hour course, rather than the typical expectation of completing at least six credit hours.
The study authors estimated multiple impacts on outcomes in the progress toward degree completion and actual completion of a degree domains. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in this domain is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects would be attributable to the Opening Doors program, and not to other factors.