Citation
Hollenbeck, K., Huang, W.-J., and WE Upjohn Institute for Employment Research (2008). Workforce program performance indicators for the Commonwealth of Virginia. Upjohn Institute technical report no. 08-024. Kalamazoo, MI: W. E. Upjohn Institute for Employment Research. [TANF/VIEW]
Highlights
- The study’s objective was to examine the impact of the Virginia Initiative for Employment Not Welfare (TANF/VIEW) program on employment and credential completion rates.
- The authors used administrative records to compare outcomes for low-income adults who took part in the TANF/VIEW program to a nonexperimental matched group of adults who did not take part in the program.
- The study found that, compared with those who did not take part in the program, the employment rate was significantly lower for those who took part in the TANF/VIEW program, but the TANF/VIEW group earned significantly more educational or training credentials.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the TANF/VIEW program; other factors are likely to have contributed.
Intervention Examined
The Virginia Initiative for Employment Not Welfare (TANF/VIEW) Program
Features of the Intervention
The TANF/VIEW program aimed to help adults who received TANF move into employment. The program provided services such as child care and transportation and could have included education and training support.
Features of the Study
Using data from program administrative records, Unemployment Insurance records, the Wage Record Interstate System, and the community college system, the authors compared those who exited the TANF/VIEW program from July 2004 to June 2005 with those who did not take part in the program and used the Employment Services program. The authors used a nonexperimental statistical approach called propensity-score matching to create an Employment Services group that was similar in terms of demographics, education, and labor market history. The authors compared the two groups on employment two and four quarters after program exit, and on the percentage who had earned an education or training credential within one year of program exit. This analysis included 6,373 participants of the TANF/VIEW in Virginia.
Findings
- Employment. The study found that TANF/VIEW participants were significantly less likely to be employed after program exit than the Employment Services group, with an employment rate that was 2.1 percentage points lower at the second quarter and 2.0 percentage points lower at the fourth quarter after program exit.
- Education and/or training attainment and completion. The study found that those who took part in the TANF/VIEW program were significantly more likely to complete an educational or training credentials than those who did not take part in the program (0.4 percentage points higher).
Considerations for Interpreting the Findings
Although the authors accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the authors’ decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the TANF/VIEW and Employment Services participants were on identical wage trajectories before receiving services from their respective programs and that the average length of participation in TANF/VIEW services was six months, whereas that for Employment Services was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would be looking at TANF/VIEW participants’ earnings about 12 months after they started receiving services and Employment Services participants’ earnings about 7 months after they started receiving services. If both programs were completely ineffective and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the TANF/VIEW participants earned more 6 months after their exit dates. However, this would not be attributable to receiving TANF/VIEW services; it would be caused by the different elapsed time across the groups (12 months for TANF/VIEW participants versus 7 months for Employment Services participants). Therefore, studies defining the groups based on exit date, rather than entry date, cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the TANF/VIEW program; other factors are likely to have contributed.
Additional Sources
Hollenbeck, K. (2011). “Short-term net impact estimates and rates of return.” In Douglas J. Besharov and Phoebe H. Cottingham (Eds.), The Workforce Investment Act: Implementation experiences and evaluation findings (pp. 371–295). Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.