Absence of conflict of interest.
Citation
York, V. (2018). Johnson County Community College (JCCC) TAACCCT final evaluation report. Office of Educational Innovation and Evaluation. Manhattan, KS: Kansas State University.
Highlights
- The study’s objective was to evaluate the effects of Johnson County Community College’s (JCCC) Accelerated, Collaborative Technology Training Services (ACTTS) project on education outcomes.
- The author used a nonexperimental design to compare cohorts of students enrolled in ACTTS and comparison programs on five educational outcomes of interest.
- The study found that ACTTS students were significantly more likely than comparison group participants to complete their program of study, be retained in their program of study, complete credit hours, and earn credentials.
- The quality of causal evidence presented in this study is low because the authors did not ensure that the groups being compared were similar before the intervention or include sufficient controls. This means we are not confident that the estimated effects are attributable to ACTTS; other factors are likely to have contributed.
Intervention Examined
JCCC’s Accelerated, Collaborative Technology Training Services (ACTTS) project
Features of the Intervention
The U.S. Department of Labor's Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant program was a multi-year (2011 to 2018), $1.9 billion initiative to build human capital around workforce development, particularly in the following areas: manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, the Department of Labor awarded a total of 256 TAACCCT grants to approximately 800 educational institutions, most of which were community colleges., received TAACCCT funding to implement the Accelerated, Collaborative Technology Training Services (ACTTS) project. The project was created to expand JCCC’s capacity to provide information technology (IT) training to meet growing IT sector needs in the Kansas City Metropolitan Area. The program was designed to serve TAA-eligible workers, eligible veterans and spouses, and other adults. The goals of the ACTTS project were to: 1) create an IT curriculum to better meet students’ unique scheduling and learning needs; 2) engage local employers to assist in curriculum development, professional development, and job forecasting; and 3) provide enhanced student support services for ACTTS programs. The ACTTS programs included four career pathways (Computer Information Systems/Programming, Information Technology Network, Web Development and Digital Media, and Health Information Systems), whereby students could pursue eight certificates and six associate degrees. Students enrolled in an ACTTS program were provided with career coaches to create customized educational plans and professional development opportunities.
Features of the Study
The nonexperimental study was conducted at Johnson County Community College (JCCC) in Overland Park, Kansas. The author compared the outcomes of students who participated in the ACTTS program to students who did not. The treatment condition included 396 students who participated in one of the four career pathways in the ACTTS program and were enrolled between September 2015 and March 2018. The comparison group was composed of 685 students enrolled between August 2015 and January 2018 in one of two IT programs similar to the career pathways in the ACTTS program, but not included in the ACTTS project. Data sources included JCCC's banner system, the ACTTS student intake forms, the National Student Clearinghouse, and Kansas Department of Commerce's KANSASWorks data system. The author conducted chi-square analyses to examine differences between the groups on five educational outcomes (program completion, retention, completed credit hours, earned credentials, and pursued further education).
Findings
Education and skills gain
- The study found that ACTTS students were significantly more likely than comparison group students to complete their program of study (a difference of 12 percentage points).
- The study found that more than students in the ACTTS programs were significantly more likely than comparison group students to be retained in their program of study (a difference of 37 percentage points).
- The study found that ACTTS students were significantly more likely than comparison group students to complete more credit hours (a difference of 8 percentage points) and earn more credentials (a difference of 12 percentage points).
- However, the study found that ACTTS program participation was significantly associated with lower rates of further education, with fewer treatment students pursuing further education after completing their program than comparison students (a difference of 14 percentage points).
Considerations for Interpreting the Findings
The composition of the treatment and comparison groups varied significantly by gender, race, school enrollment status, and Pell grant status. However, the authors did not control for the differences between the groups. These preexisting differences between the groups—and not the ACTTS program—could explain the observed differences in outcomes. Additionally, the author used a comparison group of students enrolled in different IT programs than those supported by the ACTTS project. Although the comparison group students attended the same college, the program-varying differences between the participants can lead to a confound, which could also impact the results. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups were similar before the program or include sufficient controls. This means we are not confident that the estimated effects are attributable to the ACTTS program; other factors are likely to have contributed.