Citation
Bloom, D., & Sommo, C. (2005). Building learning communities: Early results from the Opening Doors Demonstration at Kingsborough Community College. New York: MDRC.
Highlights
- The study’s objective was to examine the impact of the Opening Doors learning community program at Kingsborough Community College in Brooklyn, New York, on students’ persistence, course completion, and credits earned during the 2003–2004 academic year.
- The study was a randomized controlled trial in which students were randomly assigned to the Opening Doors learning community treatment group or the control group. Data sources included students’ transcript data provided by Kingsborough and data from a baseline survey completed by students at the time of random assignment.
- This study found that the Opening Doors learning community program at Kingsborough had positive impacts on developmental course enrollment and passage rates, percentage and number of courses passed, equated credits earned in the program semester, and developmental course enrollment in the first postprogram semester. During the first academic year, Opening Doors students earned significantly more equated credits and made significantly more progress in required English courses than the control group.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Opening Doors learning communities, and not to other factors.
Intervention Examined
Opening Doors Learning Communities at Kingsborough Community College
Features of the Intervention
At Kingsborough Community College in Brooklyn, New York, the Opening Doors learning communities program involved enrolling small cohorts of students in three linked courses during one semester. These learning communities involved an English course, a college-level course required for a specific major, and a one-credit freshman orientation class. The freshman orientation class was a seminar designed to reinforce the learning objectives of the other two courses and provide students practical information about careers in their selected majors. The instructor of this course also served as a case manager for the students in the learning community, meeting with faculty members to ensure the success of learning community students. In addition, each learning community member had a tutor who provided students with one-on-one tutoring; learning community students were also provided with $225 textbook vouchers. This study focused on the first of four cohorts of students who participated in learning communities from fall 2003 to spring 2005.
To participate in the Opening Doors learning communities, students had to be first-time incoming freshman ages 17 to 34 who planned to attend college full-time during the day, and be enrolled in either a developmental or college-level English course. Although not a specific requirement for eligibility throughout program implementation, the program targeted low- and moderate-income students.
Features of the Study
This study was a randomized controlled trial. Randomization occurred at the student level. Eligible students who consented to participate in the study filled out a baseline data form. Then, they were randomized to either the treatment group, which was offered the opportunity to enroll in a learning community, or the control group, which could not enroll in a learning community, but could access existing college services. This study reported results for the first cohort of students who entered the study in fall 2003. There were 387 students in this cohort with 192 in the treatment group and 195 in the control group. (The full study sample consisted of 1,534 students with 769 in the treatment group and 765 in the control group.)
The authors used student transcript data and reported impacts on courses taken and passed and credits earned during the program semester and the first postprogram semester, and registration/enrollment status for the second postprogram semester. One of the outcomes was equated credits earned; these are class hours in developmental and compensatory courses for which actual credits are not allowed, but they are counted as regular credits for certain purposes, such as determining financial aid availability. Impacts were estimated using two-tailed t-tests applied to unadjusted differences between the treatment and control groups.
Findings
- In the program semester (fall 2003), students in the treatment group were 15 percent more likely to take one or more English courses overall and 11 percent more likely to take a developmental English course than control students. They were 20 percent more likely to pass one or more English courses overall and 16 percent more likely to pass a developmental English course. They were about 57 percent more likely to take and pass the student development course. They passed 8 percent more courses (0.5 courses) and earned 1.5 more equated credits. All these differences were statistically significant.
- In the first postprogram semester (spring 2004), students in the treatment group were about 10 percentage points more likely to take one or more English courses than control students. This difference was statistically significant.
- During the first academic year overall, students in the treatment group earned significantly more equated credits (4.9 versus 3.5 for the control group) and were more likely to have either completed or enrolled in one or both of the required, credit-bearing English courses as of spring 2004 (52 versus 42 percent for the control group).
Considerations for Interpreting the Findings
The study was a well-implemented randomized controlled trial. However, participants were randomly assigned in slightly different ways—those who enrolled early enough were told about the program individually during the early registration period, whereas those who enrolled later (the majority) were recruited into the study in groups during a registration fair in the week before the start of the semester. Otherwise, the eligibility requirement and randomization procedure did not change and the authors did not discuss any potential issues stemming from this change in procedure. The authors cautioned that the results for the first cohort of this multiyear evaluation, which is the focus of this study, should be interpreted with care because the program might have been implemented with greater fidelity and consistency in the later semesters.
The study authors estimated multiple related impacts on outcomes related to progress toward degree completion. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Opening Doors learning communities, and not to other factors.
Additional Sources
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2014). Developmental Students in Postsecondary Education intervention report: Linked learning communities. Retrieved from http://whatworks.ed.gov