Absence of conflict of interest. This study was conducted by staff from ICF, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Takyi-Laryea, A., Passa, K., & Gall, A. (2017). University of the District of Columbia Community College TAACCCT round 3 final evaluation report. Fairfax, VA: ICF.
Highlights
- The study’s objective was to examine the impact of the grant-funded DC Construction Academy (DCCA) program on education outcomes.
- The study used a nonexperimental design to compare the education outcomes of DCCA program participants to a matched historical comparison group of students enrolled in the same construction programs prior to TAACCCT funding.
- The study found that DCCA program participants were significantly more likely to attain a credential than the comparison group.
- The quality of causal evidence presented in this study is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the DCCA program; other factors are likely to have contributed.
Intervention Examined
The DC Construction Academy (DCCA)
Features of the Intervention
The U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) program provided $1.9 billion in grants to community colleges to improve skills and support employment in high-demand industries, notably manufacturing, health care, information technology, energy, and transportation. Through four rounds of funding, DOL awarded 256 TAACCCT grants to approximately 800 educational institutions across the United States and its territories.
The University of the District of Columbia Community College (UDC-CC) received TAACCCT grant funds to enhance programs in the construction and hospitality sectors and reduce unemployment and underemployment in the District of Columbia by enhancing the skills of residents. These two industries were identified as high-growth, high-demand sectors. UDC-CC implemented the DC Construction Academy and the DC Hospitality Academy to address these labor-market needs. An academy approach was added to existing curricula for the hospitality and construction programs offered at UDC-CC. Features of the new program included expanded online programming, new curricula, latticed and stackable programming, learning assessments, student supports, and integrated teaching.
Features of the Study
The study took place on the campus of UDC-CC in Washington, District of Columbia and focused on the impact of the DC Construction Academy (DCCA). The authors used a nonexperimental design to compare the outcomes of students enrolled in the grant-funded DCCA program to a matched comparison group of students enrolled in construction programs prior to the TAACCCT funding. The treatment group consisted of 266 students that were enrolled in the DCCA grant-funded program from Summer 2015-Spring 2017 and received grant-funded services. The comparison group consisted of 266 students who were enrolled from Summer/Fall 2014-Spring 2015 in the same construction programs and did not receive grant-funded services. The authors matched DCCA program participants to similar nonparticipants using propensity scores developed from demographic information. Using administrative, and baseline and follow-up survey data, the study examined differences in completion rates and credential attainment between the treatment and historical matched comparison groups.
Findings
Education and skills gains
- The study found a significant relationship between the grant-funded DCCA program and credential attainment, with a higher percentage of students enrolled in the DCCA program (51%) earning a credential compared to the students in the comparison group (22%).
- The study did not find a significant relationship between the grant-funded DCCA program and completion rates.
Considerations for Interpreting the Findings
Although the authors created a matched comparison group using propensity score matching, they did not account for other factors that could have affected the difference between the treatment and comparison groups, such as pre-intervention measures of financial disadvantage or education as required in the protocol. These preexisting differences between the groups—and not the DCCA program—could explain the observed differences in outcomes. Additionally, because the outcome data on the two groups were collected from participants at different times, differences in outcomes could be due to time-varying factors (such as overall changes in the community college) and not the intervention. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this study is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the DCCA program; other factors are likely to have contributed.