Absence of conflict of interest.
Citation
Highlights
- The study's objective was to examine the impact of New Mexico Legislative Lottery Scholarship (NMLLS) on education outcomes.
- This study was a difference-in-differences design exploring the impact of NMLLS on University of New Mexico (UNM) college completion and credit accumulation. The authors used administrative data to conduct statistical models to compare changes in outcomes between the treatment and comparison groups before and after NMLLS was introduced.
- The study found that NMLLS was not significantly associated with graduation rates or cumulative credits earned.
- This study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to New Mexico Legislative Lottery Scholarship (NMLLS); other factors are likely to have contributed.
Intervention Examined
New Mexico Legislative Lottery Scholarship (NMLLS)
Features of the Intervention
NMLLS is a merit-based state scholarship designed to encourage students to complete their degrees in a timely fashion, keep talented students in the state, and increase higher education access. Starting in 1997, NMLLS became available to any New Mexico resident who earned a high school diploma or equivalent in the state and started at a public New Mexico postsecondary institution in the fall or spring after they graduated high school. Unlike other states which provide scholarships based on achievement in high school, NMLLS is based on achievement of at least a 2.5 grade point average (GPA) in the first semester of a full course load at a two- or four-year public college. New Mexico also provides scholarships for the first semester to offset tuition until students become eligible for NMLLS. During the study period, NMLLS was available to eligible students for up to eight semesters so long as they remained enrolled in a full-time course load and maintained a 2.5 GPA.
Features of the Study
This study used a difference-in-differences design to examine the impact of NMLLS on University of New Mexico (UNM) college completion and credit accumulation. Specifically, researchers compared the difference in rates of college completion and completed course credits before and after NMLLS was implemented for students who were New Mexico residents and eligible for the scholarship and those who were not New Mexico residents and thus not eligible for the scholarship. The study used administrative data from 1995 through 1999 and included 9,830 University of New Mexico students. The treatment group consisted of 8,802 students who were New Mexico residents and the comparison group consisted of 1,028 students who were domestic out of state students. The authors excluded international students from the analysis. The data included sociodemographic information, high school academic history, and college academic performance by semester. Data were measured before and after NMLLS was implemented. The authors used statistical models to compare the changes in outcomes between the groups.
Findings
Educational and skills gains
- The study found no significant relationships between the NMLLS and graduation rates at 4, 4.5, 5 or 6 years since first enrollment.
- The study found no significant relationships between the NMLLS and cumulative credits by the end of the 1st, 2nd, 3rd, 4th, 5th, and 6th years.
Considerations for Interpreting the Findings
The authors used a concurrent cohort of out of state students as a comparison group. The state varying characteristics of the comparison group creates a confound. Additionally, although the authors controlled for gender, race, and a pre-intervention measure of academic achievement, they did not control for age as required by the protocol. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of evidence presented in this report is low, because of the presence of a confound and the lack of sufficient control variables. This means we are not confident that the estimated effects are attributable to the New Mexico Legislative Lottery Scholarship (NMLLS); other factors are likely to have contributed. However, the study did not find any statistically significant effects.