Absence of conflict of interest.
Citation
Price, D., Childress, L., Sedlak, W., & Roach, R. (2017). Northeast Resiliency Consortium final evaluation report. Indianapolis, IN: DVP-PRAXIS LTD.
Highlights
- The study’s objective was to examine the impact of the Northeast Resiliency Consortium (NRC) on education, earnings, and employment outcomes. This summary focuses on the comprehensive support services strategy.
- The authors used a nonexperimental design to compare the outcomes of NRC students who received comprehensive support services to a matched comparison group.
- The study found a significant association between receipt of support services and increased program completion, credential attainment, credit accumulation, earnings for incumbent workers, and employment attainment and retention.
- The quality of causal evidence presented in this study is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the NRC program; other factors are likely to have contributed.
Intervention Examined
The Northeast Resiliency Consortium (NRC)
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Northeast Resiliency Consortium (NRC) was comprised of seven community colleges across four northeastern states. The consortium received grant funds in 2013, under TAACCCT, to close skill gaps through enhanced and expanded programing in healthcare, information technology, hospitality, and environmental science. As a result, NRC colleges offered 84 programs of study (44 continuing education and workforce development programs and 40 credit-bearing programs). NRC’s goals included: (1) accelerating skills, competencies, and credential acquisition; (2) incorporating advanced technology to increase access to classes and accelerate learning; (3) increasing employer engagement; and (4) providing comprehensive outreach, assessment, and student supports. The NRC program included two primary strategies. The first strategy focused on continuing education to credit-based educational pathways, including stacked and latticed credentials, advanced technology, and work-based learning opportunities. The second strategy included comprehensive support services (career, personal, and academic) framed to support resiliency.
Features of the Study
The study used a nonexperimental design to compare the outcomes of continuing education students who received comprehensive support services to students at the same colleges who did not. The authors matched NRC participants to similar nonparticipants using propensity scores developed from demographic information, education, employment status, and state. Across the seven colleges, study participants included 1,396 students in the treatment group and 1,396 students in the comparison group. The authors conducted statistical analyses to examine differences in outcomes between the groups using community college administrative data, a centralized database, and state workforce data. Outcomes included program completion rates, attainment of credentials, credit accumulation, employment attainment and retention, and earnings increases.
Study Sites
- Atlantic Cape Community College in Hamilton, New Jersey
- Passaic County Community College in Paterson, New Jersey
- Bunker Hill Community College in Boston, Massachusetts
- Capital Community College in Hartford, Connecticut
- Housatonic Community College Bridgeport, Connecticut
- Kingsborough Community College in Brooklyn, New York
- LaGuardia Community College in Queens, New York
Findings
Education and skills gain
- The study found a significant relationship between receipt of comprehensive support services and program completion, with 82% of treatment students completing their program compared to 44% of comparison students.
- The study also found a significant relationship between receipt of comprehensive support services and credential attainment, with higher proportions of treatment students earning a credential than comparison students (74% versus 37%).
- Receipt of comprehensive support services was significantly associated with higher rates of credit accumulation, with 41% of treatment students earning or banking credits for continuing education compared to 24% of comparison students.
- The study did not find a significant relationship between receipt of comprehensive support services and transition to a credit program.
Earnings and wages
- The study found a significant relationship between receipt of comprehensive support services and increased earnings for incumbent workers, with higher proportions of treatment students having an increase in earnings than comparison students (87% versus 76%).
Employment
- The study found a significant relationship between receipt of comprehensive support services and employment for students who were unemployed upon entry into NRC programs, with higher employment rates for treatment students than comparison students (41% versus 19%).
- Receipt of comprehensive support services was also significantly associated with higher rates of employment retention, with 67% of treatment students being employed three quarters after program exit compared to 50% of comparison students.
Considerations for Interpreting the Findings
The authors created a matched group of non-participating individuals to compare to students who were enrolled in an NRC program. However, the authors did not appropriately control for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention degree of financial disadvantage. These preexisting differences between the groups—and not the NRC program—could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar. This means we are not confident that the estimated effects are attributable to the NRC program; other factors are likely to have contributed.