Citation
Hubelbank, J., Demetry, C., Nicholson, S., Blaisdell, S., Quinn, P., Rosenthal, E., & Sontgerath, S. (2007). Long-term effects of a middle school engineering outreach program for girls: A controlled study. In Proceedings, American Society for Engineering Education Annual Conference & Exhibition.
Highlights
- The study’s objective was to assess the long-term impact of Camp Reach, a summer engineering enrichment program for middle school girls, on enrollment in STEM courses in high school and college.
- Admission to Camp Reach was determined through a lottery of girls who applied to the program and met eligibility requirements. Six to seven years later, the authors administered a survey to 88 girls who had been admitted to the program and 41 girls who had not.
- The study found that Camp Reach attendees were significantly more likely to have taken computer science and other science or engineering courses in high school than those who did not attend.
- The quality of causal evidence presented in this report is low because the authors adjusted the research sample after random assignment and did not include adequate controls to ensure that the resulting groups were similar on all relevant attributes. This means we are not confident that the estimated effects are attributable to Camp Reach; other factors are likely to have contributed.
Intervention Examined
Camp Reach
Features of the Intervention
Camp Reach was a two-week summer engineering program operated by Worcester Polytechnic Institute (WPI) for rising 7th-grade girls. The program aimed to encourage girls’ interest in and pursuit of engineering and other STEM fields through design workshops and mentoring. It emphasized collaborative problem-solving through a group public service engineering project highlighting engineering as a helping profession. The camp promoted continued engagement through alumnae newsletters and reunions.
Features of the Study
Camp Reach made 30 places available to interested applicants each year; those applicants meeting minimum eligibility criteria participated in a lottery through which camp organizers selected the summer’s participants, who formed the treatment group. The group of selected participants was then adjusted to reflect the demographic profile of Worcester, Massachusetts, where the program was held. Those who were not invited to participate formed the control group.
The authors contacted 178 girls who had applied to Camp Reach from 1997 to 2000 six to seven years after they had applied (that is, the authors contacted them in 2004 through 2006) to complete a survey. Of these, 129 responded. The authors compared the responses of the treatment and control group members to survey questions about their knowledge of engineering, the science and engineering courses they took in high school, and plans to major in science or engineering in college.
Findings
- The study found that Camp Reach attendees were significantly more likely to have taken computer science and other science or engineering courses in high school than those who did not attend.
Considerations for Interpreting the Findings
The authors acknowledged modifying the randomly selected attendee groups after random selection to replicate Worcester’s demographic profile. This adjustment means that the assignment was no longer random and, therefore, this study cannot receive a high causal evidence rating. The adjustment of the research sample after random assignment could have induced differences between the treatment and control groups on characteristics that also influenced the outcomes of interest. Because the techniques the authors used to estimate the program’s effect did not adequately control for these potential differences, the authors could not distinguish the effects of Camp Reach from the effects of differences in relevant demographic characteristics.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors adjusted the research sample after random assignment and did not include adequate controls to ensure that the resulting groups were similar on all relevant attributes. This means we are not confident that the estimated effects are attributable to Camp Reach; other factors are likely to have contributed.