Absence of conflict of interest.
Citation
Skilton-Sylvester, P., Myran, S., Myran, G., Ross, S., & Williams, M. (2016). Final evaluation report: Implementation and outcomes of Credentials to Careers. Canadian Lakes, MI: Myran & Associates LLC.
Highlights
- The study’s objective was to examine the impact of the Credentials to Careers (C2C) program on earnings.
- The authors used a nonexperimental design to compare outcomes of C2C participants to a matched comparison group using data provided by the college.
- The study found that C2C program participation was significantly associated with a larger average percent increase in wages relative to the comparison group.
- The quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the C2C program; other factors are likely to have contributed.
Intervention Examined
The Credentials to Careers (C2C) Program
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
In 2013, a seven-college consortium, led by Northern Virginia Community College (NOVA), was awarded a TAACCCT grant to implement the Credentials to Careers (C2C) program. The C2C program was designed to provide training and credentials in the Science, Technology, Engineering and Mathematics (STEM) careers, with a focus on advanced manufacturing, information technology, and health care technology. The C2C program included evidence-based design, stacked and latticed credentials, online and technology-enabled learning, curriculum that can be transferred to other colleges, agreements with 4-year institutions, and strategic alignment to the employment sector. The program was designed to meet the needs of unemployed and displaced workers. Each community college focused on STEM career pathways based on the local/regional employment needs. NOVA focused on Computer/Information Technology (IT) career pathways leading to occupations as a software developer/analyst, project manager, database administrator, IT security, or systems engineer. NOVA's C2C program provided structured, wrap-around support services, included problem-based and authentic learning, and prepared students to be successful job seekers and employees by also developing “soft” skills and dispositions.
Features of the Study
The study took place at NOVA and used a nonexperimental design to compare the outcomes of students who participated in the C2C program to students who did not participate. The treatment group included students who enrolled and completed the C2C Computer/IT program. The comparison group included students who completed the Business Information Technology program. The authors matched C2C students to similar students using propensity scores developed from demographic variables. Study participants included 102 students in the treatment group and 141 students in the comparison group. Using data provided by the college, the authors conducted t-tests to examine differences in percentage wage increases between the groups. Treatment group wage data was from January 2013-June 2015; comparison group wage data was from January 2012- January 2014.
Findings
Earnings and wages
- The study found that C2C program participation was significantly associated with a larger average percent increase in wages relative to the comparison group (57.06 versus 27.59).
Considerations for Interpreting the Findings
This study has several issues that should be considered when interpreting the findings. First, despite using a matched design to ensure that students in the research groups were similar in terms of baseline demographic characteristics, the composition of the treatment and comparison groups varied by gender and Pell grant eligibility. However, the authors did not control for these variables in their analyses. These preexisting difference between the groups—and not the C2C program—could explain the observed differences in outcomes. Moreover, the authors state that they had difficulty in finding a comparison group and had to go back to data from 1992. Using students from previous enrollment years as the comparison group presents a confound because differences in outcomes could be due to time-varying factors (such as overall changes at the community college) and not the program. Also, the authors note that the wage data had issues with non-normality (skewed data) so they removed the extreme cases based on visual inspection. Finally, the study did not account for changes in state or regional labor markets, thus, a broader economic context may have impacted the estimated effects of the C2C program on earnings outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not use sufficient controls in their analysis. This means we are not confident that the estimated effects are attributable to the C2C program; other factors are likely to have contributed.