Absence of conflict of interest.
Citation
Hughes, K., Belfield, C., Ran, F., & Jenkins, D. (2018). KCTCS Enhancing Programs for IT Certification (EPIC): Independent evaluation contracted by the Kentucky Community and Technical College System. New York, NY: Community College Research Center, Teacher's College, Columbia University.
Highlights
- The study’s objective was to assess the impact of the Enhancing Programs for IT Certification’s (EPIC) Learn on Demand (LoD) courses on community college students’ education outcomes. This summary focuses on the comparison between students who enrolled in EPIC LoD courses and students who enrolled in the traditional format courses.
- The authors used a nonexperimental design to compare the outcomes of students enrolled in EPIC LoD courses with those enrolled in traditional formats of courses where an EPIC LoD version was available.
- The study found that EPIC LoD course participation was significantly associated with increased course completion and certificate completion.
- The quality of causal evidence presented in this report is moderate for the course completion outcome because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the EPIC LoD courses, but other factors might also have contributed. For the certificate completion outcome, the quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the EPIC LoD courses; other factors are likely to have contributed.
Intervention Examined
The Enhancing Programs for IT Certification (EPIC) Project
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The Kentucky Community and Technical College System (KCTCS)—a consortium of six community colleges—was awarded a TAACCCT grant in October 2014 to support its Enhancing Programs for IT Certification (EPIC) project. The project’s objective was to expand these colleges’ online, competency-based curriculum called Learn on Demand (LoD) and to help dislocated workers, veterans, and other adults obtain credentials and degrees in high-demand careers such as computer information technology (CIT) and medical information technology (MIT). The TAACCCT grant allowed the EPIC project to develop five new LoD degree programs, 14 new stackable certificate programs in CIT, and two degree programs and six certificate programs in MIT. These different EPIC programs offered self-paced, modularized learning for students online, and had flexible start dates. They also provided academic support (e.g., 24/7 advising and student success coaching) and workplace readiness preparation. The EPIC project built upon the KCTCS’s strategy to improve recruitment and retention of adults in business administration and information technology programs.
Features of the Study
The authors used a nonexperimental design to compare the outcomes of students who enrolled in EPIC LoD courses with students who did not. The treatment group included students who enrolled in one or more EPIC LoD courses between spring 2015 and spring 2017. The comparison group consisted of students who enrolled in traditional courses that were offered in EPIC format during the same time period. Specifically, comparison students took courses in one of the following traditional delivery modes: (1) face-to-face classroom instruction, (2) online but non-competency-based instruction called Learn-by-term, or (3) a hybrid instruction that combines face-to-face and Learn-by-term. The authors limited comparison students to programs in computer and information sciences, allied health, STEM, and business/marketing. Using data provided by KCTCS, the authors conducted statistical models with controls to examine differences in course completion and certificate completion between the groups. The sample included 1,182 students in the treatment group and 165,455 students in the comparison group (21,664 in the hybrid format, 73,903 in face-to-face, and 69,888 in Learn-by-term).
Study Sites
- Big Sandy Community and Technical College in Prestonsburg, Kentucky
- Jefferson Community and Technical College in Louisville, Kentucky
- Hazard Community and Technical College in Hazard, Kentucky
- Somerset Community College in Somerset, Kentucky
- Southeast Kentucky Community and Technical College in Cumberland, Kentucky
- West Kentucky Community and Technical College in Paducah, Kentucky
Findings
Education and skills gain
- The study found a significant relationship between EPIC LoD participation and course completion, where students who enrolled in EPIC LoD courses were more likely to complete their courses than students enrolled in any of the traditional formats.
- The study also found a significant relationship between EPIC LoD participation and certificate completion, where EPIC LoD participants who took more than two EPIC courses were significantly more likely to complete a certificate compared to students enrolled in any of the traditional formats.
Considerations for Interpreting the Findings
The baseline composition of the treatment and comparison groups varied by Pell grant eligibility, specifically for the hybrid comparison group. However, the authors did not control for this variable in the analysis for the certificate completion outcome. This preexisting difference between the groups—and not the EPIC LoD courses—could explain the observed differences in outcomes. Also, treatment group participants self-selected into the EPIC LoD courses. Students who self-selected into the courses could differ in observable and unobservable ways, affecting the outcomes.
Causal Evidence Rating
The quality of causal evidence presented in this report is moderate for the course completion outcome because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the EPIC LoD courses, but other factors might also have contributed. For the certificate completion outcome, the quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the EPIC LoD courses; other factors are likely to have contributed.