Citation
Rivera, M., Davis, M., Feldman, A., & Rachkowski, C. (2013). An outcome evaluation of an adult education and postsecondary alignment program: The Accelerate New Mexico experience. Problems and Perspectives in Management, 11(4), 105-120.
Highlights
- The study’s objective was to examine the impact of the Accelerate New Mexico Math Camp on outcomes for community college students, including women and minority students traditionally underrepresented in STEM.
- The study used course-administered assessments to compare the math competency and attitudes about math of students before and after participation in the Accelerate summer program.
- The authors found that students (both male and female) who participated in the Accelerate program had higher math competency and less anxiety about math after completing the program, compared with their competency and anxiety measured before enrollment in the program.
- The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the Accelerate New Mexico Math Camp program; other factors are likely to have contributed.
Intervention Examined
The Accelerate New Mexico Math Camp
Features of the Intervention
The Accelerate New Mexico Math Camp operated at six public northern New Mexico community colleges and was one component of a four-fold program to support traditionally underrepresented students’ pursuit of careers in STEM fields. It was an intensive eight-week summer program that covered the content of several remedial math classes in a single semester and included a robotics component. Successful completion of the Accelerate New Mexico Math Camp enabled students to move on to college-level math and science courses that are often required for majors in STEM fields. Students received a stipend to attend the program. The report evaluated only the Accelerate New Mexico Math Camp, not the entire Accelerate program. Accelerate also provided students with support from technical career advisors and coordinated professional readiness events and helped students secure internships and jobs in STEM-related fields.
Features of the Study
Students volunteered to participate in the Accelerate New Mexico Math Camp, and the study included all 55 program participants at the six sites. Females accounted for 60 percent of the students, 66 percent were Hispanic, and 27 percent were white. The average age was 27.
The authors estimated regression and other statistical models comparing the outcomes of students before and after they participated in the Accelerate New Mexico Math Camp. The models controlled for participants’ demographics (age, gender, and ethnicity); participants’ experiences (data from students on their opinions of teacher effectiveness, tutor effectiveness, the robotics curriculum, and the format and pace of the curriculum); and length of time each student spent using the course’s self-paced software program.
Findings
- Students in the study significantly improved their math competency from 22 percent of students who were competent at the beginning of the program to 64 percent at the end.
- Students’ attitudes toward math, as measured by self-reported math efficacy and anxiety, also significantly improved by 10 percent from the beginning to the end of the program.
- Both of these results were true for men and women, but there was no statistically significant difference in the magnitude of the outcomes by gender.
Considerations for Interpreting the Findings
The authors compared the outcomes of participants measured before and after they participated in the Accelerate New Mexico Math Camp; there was no comparison group that did not participate in the program and could represent what might have occurred had the people who chose to participate in the camp not enrolled in it. For study designs of this type,. CLEAR’s guidelines require that the authors observe outcomes for multiple periods before the intervention to rule out the possibility that participants had increasing or decreasing trends in the outcomes examined before enrollment in the program. That is, if students who had increasing math competency or attitudes tended to enroll in the program, we would anticipate further increases over time, even if they did not participate in the program. Without knowing the trends before program enrollment, we cannot rule this out.
Causal Evidence Rating
The quality of causal evidence presented in this report is low. This means we are not confident that the estimated effects are attributable to the Accelerate New Mexico Math Camp; other factors are likely to have contributed.