Absence of conflict of interest.
Citation
Yamada, H., & Bryk, A. S. (2016). Assessing the First Two Years’ Effectiveness of Statway®. Community College Review, 44(3), 179–204. doi: 10.1177/0091552116643162
Highlights
- The study’s objective was to evaluate the impact of Statway® on community college students’ earning of math course credits.
- The study used a nonexperimental design to compare the outcomes of students who were in the Statway® program to a matched comparison group. Using institutional research data from the community colleges, the authors conducted statistical models to examine differences between the groups.
- When compared to non-participating students, the study found that Statway® program participation was associated with a significantly higher likelihood of earning college math credit.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Statway®; other factors are likely to have contributed.
Intervention Examined
Statway®
Features of the Intervention
Statway® is a one-year intensive community college course designed to help developmental math students obtain college-level credit in statistics. The program provides an alternative to the traditional developmental mathematics sequence at community colleges that reduces structural barriers to college math achievement while meeting the requirements for introductory college-level math. It combines college-level statistics with skills and concepts from more rudimentary mathematics courses and supplements it with faculty support and instruction. The program is grounded in research-based principles that underscore connecting mathematical concepts to real-world problems, employing repetitive practice to increase students’ understanding of core concepts, fostering active student engagement, and reducing language and literacy barriers. Moreover, Statway® provides professional development support for faculty to ensure high-quality teaching is maintained and facilitates continuous improvement in teaching and learning.
Features of the Study
The authors used a nonexperimental design to compare the outcomes of students who were in the Statway® program with students who had taken traditional developmental math. The study included students from two Statway® cohorts. The Year 1 Cohort (Fall 2011) included students from 17 community colleges across 5 states (California, Connecticut, Florida, Texas, and Washington). The Year 2 Cohort (Fall 2012) included students from 15 community colleges in the same states. The authors received student demographic, prior course enrollment, and performance data from each community college. The authors used a hierarchical linear model framework when creating propensity scores to match students on 44 characteristics. Statway® students were matched with non-Statway® students who had similar demographic characteristics and similar past academic performance prior to the start of the study. The Year 1 sample included 928 students in the treatment group and 4,549 in the comparison group. The Year 2 sample included 771 students in the treatment group and 3,583 in the comparison group. The author conducted statistical models to examine group differences in math credit accumulation for the first cohort only.
Study Sites
- California
- Connecticut
- Florida
- Texas
- Washington
Findings
Education and skills gain
- The study found a significant relationship between program participation and earning of math credits, with Statway® students accumulating 5.57 college credits one year after program completion versus 4.08 credits for comparison students.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating students to compare to Statway® students. However, the authors did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not the Statway® program—could explain the observed differences in outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to Statway®; other factors are likely to have contributed.