Absence of conflict of interest.
Citation
Jensen, J., Horohov, J., & Wright, C. (2016). Online2Workforce (O2W) Elizabethtown Community & Technical College TAACCCT Round II Grant Final Evaluation Report. Lexington, KY: University of Kentucky College of Education Evaluation Center.
Highlights
- The study’s objective was to assess the impact of the Online2Workforce (O2W) program on education, earnings, and employment outcomes.
- The authors used a nonexperimental design to compare the outcomes of students who were in the O2W program to a matched comparison group.
- The study found that O2W program participants had significantly higher rates of credentials earned and awarded, and lower rates of employment after program completion than the comparison group.
- The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the Online2Workforce program, but other factors might also have contributed.
Intervention Examined
The Online-2-Workforce (O2W) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2012, Elizabethtown Community and Technical College (ECTC), one of 16 colleges in the Kentucky Community and Technical College System (KCTCS), was awarded a TAACCCT grant to implement the Online-2-Workforce (O2W) program. Students in O2W were required to be enrolled at one of the seven colleges in the KCTCS hosting a preexisting online program called Learn on Demand (LOD). O2W was a redesign of LOD that expanded upon the preexisting program and built on a pre-existing online coursework infrastructure. The goal of the redesign was to create a logically presented sequence of courses that allowed students to stack credentials for increased job preparedness. O2W reoriented the flow of Business Administration courses from the concurrent course model used in the LOD modularized courses to a linear sequence of modules. This was coupled with a specified course list, and was intended to reduce credits lost to irrelevant or dropped courses and increase students' success and retention. Other program innovations included competency-based assessment and advising (enhancement of the coaching structure). The new course design included personalized content, embedded assessment, interactive modalities, and immediate student feedback designed to increase student engagement and improve learning outcomes.
Features of the Study
The nonexperimental study was conducted at ECTC and compared the outcomes of students in the O2W program to students who were not in the program during the same time frame. The authors matched O2W participants to similar nonparticipants using propensity scores developed from demographic and academic information. There were 457 students in the treatment group and 1,117 students in the comparison group before matching. After matching, the analysis sample included 379 students in each group. Using college administrative data and data from the Kentucky Center for Workforce Statistics (KCEWS), the authors conducted statistical analyses to examine differences in outcomes between the groups.
Findings
Education and skills gains
- The study found that a significantly higher proportion of O2W students (37%) earned a credential relative to students in the comparison group (20%).
- The study also found that a significantly higher proportion of O2W students (23%) were awarded a credential relative to students in the comparison group (12%).
- The study found no significant difference in credits earned between the treatment and comparison groups.
Earnings and wages
- The study found no significant differences in earnings outcomes between the treatment and comparison groups.
Employment
- The study found that a significantly smaller proportion of O2W students (60%) were employed after program completion relative to their comparison peers (71%).
Considerations for Interpreting the Findings
The authors only had access to aggregate descriptive information for employment and earnings. Without individual-level data, they were unable to conduct tests for the differences between the O2W and comparison students. However, they note that, given the small difference in median quarterly wages between the two groups, it is unlikely they would have detected statistically significant differences regardless.
Causal Evidence Rating
The quality of causal evidence presented in this report is moderate because it was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to the O2W program, but other factors might also have contributed.