Citation
Weiss, M. J., Mayer, A., Cullinan, D., Ratledge, A., Sommo, C., & Diamond, J. (2014). A random assignment evaluation of learning communities at Kingsborough Community College: Seven years later. New York: MDRC.
Highlights
- The study aimed to examine the impact of the Opening Doors learning communities at Kingsborough Community College in New York on progress toward completing a degree, actual completion of a degree, employment, and earnings. It measured impacts seven years after random assignment. Students in the Opening Doors learning communities took three linked courses during one semester and received tutoring and case management services.
- The study was a randomized controlled trial. Eligible students were randomly assigned to either the treatment group, which was offered the opportunity to participate in one of the learning communities, or the control group, which was not allowed to enroll in a learning community. The primary data sources were a baseline survey on the background characteristics of students, student transcripts, degree attainment information from the National Student Clearinghouse, and employment and earnings information from Unemployment Insurance (UI) wage records in New York.
- The study found no significant impacts on the number of credits earned or attempted, completion of any college degree, or employment or earnings in the seventh year after random assignment.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects would be attributable to the learning communities, and not to other factors.
Intervention Examined
Opening Doors Learning Communities at Kingsborough Community College
Features of the Intervention
At Kingsborough Community College in Brooklyn, New York, the Opening Doors learning communities program involved enrolling small cohorts of students in three linked courses during one semester. These learning communities involved an English course, a college-level course required for a specific major, and a one-credit freshman orientation class. The freshman orientation class was a seminar designed to reinforce the learning objectives of the other two courses. Each learning community also had a case manager who offered academic guidance to students and a tutor who provided students with one-on-one tutoring. Learning community students were also provided with $150 textbook vouchers. This study focused on four cohorts of students who participated in learning communities from 2003 to 2005 and examined their educational, employment, and earnings outcomes seven years later.
To be eligible to participate in the Opening Doors learning communities, students had to be first-time incoming freshman ages 17 to 34 planning to attend college full-time during the day, and be enrolled in either a developmental or college-level English course. Although not a specific requirement for eligibility throughout program implementation, the program targeted low- and moderate-income students.
Features of the Study
This study was a randomized controlled trial. Randomization occurred at the student level, separately for each cohort (fall 2003, spring 2004, fall 2004, and spring 2005). Eligible students who consented to participate in the study filled out a baseline data form. Then, the students were randomly assigned to either the treatment group, which was offered the opportunity to enroll in a learning community, or the control group, which could not enroll in a learning community, but could access existing college services. There were 1,534 students in the study with 769 in the treatment group and 765 in the control group.
The authors used data from the baseline data form, student transcripts, and the National Student Clearinghouse for outcomes related to progress toward degree completion. The authors used data from UI wage records from the New York State Department of Labor to report outcomes on the earnings and employment of participants. The authors estimated program impacts for the pooled sample using a linear regression model that included students’ baseline characteristics and fixed effects for student cohorts. The main focus of this study was on students’ outcomes seven years after random assignment.
Findings
- The study found no significant impacts on the number of credits earned or attempted, or completion of any college degree in the seventh year after random assignment for all cohorts.
- The study found no significant impacts on employment or earnings in the seventh year after random assignment for all cohorts.
Considerations for Interpreting the Findings
Although all the basic features of the Opening Doors learning community were implemented at Kingsborough Community College, the authors noted a few concerns with fidelity to the program model. First, although the program planned for classes to have about 25 students, the learning community courses were often underenrolled, with class sizes varying from 6 to 25 students and averaging 17 students. Second, the level of integration between courses and collaboration among faculty teaching the linked courses varied by learning community, which might have dampened the overall impact of the integrative component of the Opening Doors learning community model at Kingsborough.
The authors noted that the analysis of earnings might be underpowered due to relatively small sample sizes, which means that the study would detect only very large earnings differences between the program and control groups.
This study examined the impact of a one-semester program seven years after random assignment. Over the course of the long follow-up period, it is possible the program impacts had faded.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects would be attributable to the learning communities, and not to other factors.