Citation
Hollenbeck, K., & Huang, W-J. (2014). Net impact and benefit-cost estimates of the workforce development system in Washington State. Upjohn Institute technical report no. 13-029. Kalamazoo, MI: W. E. Upjohn Institute for Employment Research. http://dx.doi.org/10.17848/tr13-029 [Adult basic education programs on community and technical college campuses]
Highlights
- The study’s objective was to examine the impact of community and technical college campus Adult Basic Education (ABE) programs on the employment rate, earnings, and benefit receipt of adults in Washington State.
- The authors used a nonexperimental method to compare short-term (3 quarters after program exit) and long-term (9 to 12 quarters after program exit) employment, earnings, and unemployment insurance benefits between those who took part in the ABE program with those who registered for services at the Labor Exchange (Employment Services).
- The study found that, compared with those who registered for services at the Labor Exchange, the employment rate for ABE participants decreased and average quarterly earnings increased, in both the short and long terms. Compared with the Labor Exchange group, ABE participants received fewer benefits in the short and long terms.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups compared were similar before program participation. This means we are not confident that the estimated effects are attributable to the ABE program; other factors are likely to have contributed.
- This study also examined the effectiveness of other workforce development programs. Please click here to find CLEAR profiles of those studies.
Intervention Examined
The Adult Basic Education (ABE) Program
Features of the Intervention
The ABE program offered classes in adult literacy, family skills, workplace skills enhancement, English language, citizenship, basic education skills, high school equivalency preparation, and alternative high school diploma programs. The classes were offered at community and technical colleges and participants typically had limited English language skills.
Features of the Study
The authors used a nonexperimental statistical approach called propensity-score matching to create a comparison group of people who registered for services at the Labor Exchange and were similar to ABE participants in terms of demographic characteristics, including gender, age, educational attainment, race, and employment and program participation history. The authors then compared the two groups on employment, hourly wages, hours worked per quarter, quarterly earnings, and receipt and amount of benefits received before and after participation. The authors collected Unemployment insurance records for those who had exited the ABE or Labor Exchange program from July 2005 to June 2006 to estimate the long-term impacts of the program in quarters 9 through 12 after program exit, and those who exited from July 2007 to June 2008 to estimate the short-term impacts in the 3 quarters following program exit. There were 11,129 ABE participants who exited in 2005-2006 and 12,540 who exited in 2007–2008.
Findings
Employment
- The authors reported that, compared with those who registered for services at the Labor Exchange, the percentage of quarters employed for those who took part in the ABE program decreased significantly more, by 2.1 percentage points in the 3rd quarter and by 3.9 percentage points in quarters 9 to 12 after program exit. The average number of hours worked increased significantly more for those who participated in the ABE program, with an average increase of 9.4 more hours worked in the 3rd quarter than the comparison group.
Earnings
- The authors reported that average quarterly earnings significantly increased by $131 more in the 3rd quarter and $90 more in quarters 9 to 12 after program exit for those who took part in the ABE program compared with those who registered at the Labor Exchange. Average hourly wages significantly increased by $0.26 more in the 3rd quarter after program exit than the comparison group.
Public benefit receipt
- The authors found that, compared with those who registered with the Labor Exchange, the likelihood of receiving benefits decreased significantly more, by 2.5 percentage points in the 3rd quarter and 5.6 percentage points in quarters 9 to 12 after program exit for those who took part in the ABE program, with claimants receiving a significant decrease of $72 more in the 3rd quarter and a significant decrease of $57 more in quarters 9 to 12 after program exit.
Considerations for Interpreting the Findings
Although the authors accounted for many underlying characteristics of the groups being compared, which could also influence their outcomes, the authors’ decision to define the groups based on their date of program exit rather than program entry is problematic. For example, suppose that the ABE program and Labor Exchange participants were on identical wage trajectories before receiving services from their respective programs and that the average length of participation in the ABE program was six months, whereas that for Labor Exchange was one month. At the conclusion of participation, they exited the program.
If we compared the groups’ earnings 6 months after their recorded exit dates, we would observe ABE participants’ earnings about 12 months after they started receiving services and Labor Exchange participants’ earnings about 7 months after they started receiving services. If both programs were completely ineffective, and everyone stayed on their original upward-sloping wage trajectory, it would appear as though the ABE participants earned more 6 months after their exit dates. However, this would not be attributable to receiving ABE classes; it would be caused by the different elapsed time across the groups (12 months for ABE participants versus 7 months for Labor Exchange participants). Therefore, studies defining the groups based on exit date, rather than entry date, cannot receive a moderate causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because treatment is confounded with systematic differences in outcome measurement follow-up length for treatment and comparison group members. This means we are not confident that the estimated effects are attributable to the Adult Basic Education (ABE) Program; other factors are likely to have contributed.