Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Decker, P., Olsen, R., & Freeman, L. (2000). Assisting Unemployment Insurance claimants: The long-term impacts of the Job Search Assistance Demonstration. Princeton, NJ: Mathematica Policy Research.
Highlights
- The study’s objective was to examine the impact of the Job Search Assistance Demonstration in Washington, D.C., and Florida on Unemployment Insurance (UI) benefits receipt, earnings, and employment.
- In this demonstration, about 20,000 UI claimants were randomized into one of three intervention groups, all of which received some variation of job search assistance services, or into a control group, which could receive existing services in the community. For all participants, the study team collected administrative data, which provided information on UI receipt, earnings, and employment.
- The study found that structured job search assistance was generally successful at reducing UI benefits receipt and increasing earnings in Washington, D.C., but not in Florida. The findings for individualized job search assistance treatment groups were less robust.
- The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the job search assistance strategies tested, and not to other factors.
Intervention Examined
JSA services
Features of the Intervention
The Job Search Assistance Demonstration was implemented in one UI benefit office in Washington, D.C., and 10 randomly selected offices across Florida. For each eligible UI claimant, the researchers predicted the probability that the claimant would exhaust his or her benefits based on characteristics including previous occupation and industry, education level, years of tenure at the previous job, and the local unemployment rate. In Washington, those with a predicted exhaustion rate greater than or equal to 60 percent were eligible for the demonstration; in Florida, those with a predicted exhaustion rate greater than or equal to 40 percent were eligible. Eligible claimants were randomly assigned to one of three treatment groups or a control group.
The three treatment groups were (1) structured job search assistance (SJSA), (2) individualized job search assistance (IJSA), and (3) IJSA plus training (IJSA+). In Washington, about 2,000 eligible claimants were randomly assigned to each of the treatment groups and the control group. In Florida, roughly 3,000 eligible claimants were assigned to each group.
All three treatment groups shared some components, including a required orientation in the sixth or seventh week of unemployment and a one-on-one assessment with UI staff. Participants in all three groups could access the job search resource center and other services provided through Job Service, but they were not required to do so. In addition, the SJSA group was required to attend a 15-hour job search workshop. Those in the IJSA group created an individualized service plan with office staff; the services specified in the plan then became mandatory. Those in the IJSA+ group, like the IJSA group, created an individualized service plan with office staff; this group also received additional information on training opportunities. Those in the IJSA+ group interested in training were referred to the Economic Dislocation and Worker Adjustment Act staff for voluntary training opportunities.
Researchers collected administrative UI benefits receipt data from initial intake into the study (from June 1995 to July 1996 in Washington, D.C., and from March 1995 to March 1996 in Florida) through two years after. Data on employment and earnings came from state UI wage records for 10 follow-up quarters in Washington, D.C., and 12 follow-up quarters in Florida. To measure impacts on UI receipt, earnings, and employment, the authors compared the outcomes of members of each of the three treatment groups with those of the control group.
Findings
Public benefits receipt
- The study found that the SJSA treatment group in Washington, D.C., received 1.13 fewer weeks of UI benefits, $183 less in total UI benefits, and had a 5 percentage point lower rate of UI benefit exhaustion in Year 1. However, these effects did not persist into Year 2. This group also had higher earnings over the 10-quarter follow-up period of $2,005, and many quarter-by-quarter earnings comparisons were also favorable for the SJSA group.
- The SJSA treatment group in Florida received 0.41 fewer weeks of UI benefits in Year 1, but this was the only statistically significant impact on UI benefits receipt.
- The IJSA treatment in Washington, D.C., had limited impacts on UI benefits receipt: a 0.46 reduction in weeks of benefits received in Year 1. Results was similar for the IJSA group in Florida: 0.6 fewer weeks of UI benefits received, $100 less in total UI benefits received, and a 2 percentage point lower benefit exhaustion rate in Year 1 than the control group, but no impacts in Year 2 on benefit receipt. In addition, there were no statistically significant earnings or employment impacts.
- The rate of UI benefit exhaustion in Year 1 for the IJSA+ group in Washington, D.C., was 4 percentage points lower than the control group, but there were no other statistically significant impacts on benefit receipt outcomes. This group also had higher earnings of $150 to $180 in quarters 2, 3, and 5, but this did not translate into higher earnings over the full follow-up period.
- The IJSA+ group in Florida had 0.52 fewer weeks of UI benefits received and a 3 percentage point lower benefit exhaustion rate in Year 1 than the control group. However, there were no statistically significant impacts on UI benefits outcomes in Year 2, no earnings impacts, and an impact on employment in only one of the 12 quarters examined.
Employment
- The study found no impacts on earnings or employment in Florida or Washington, DC
Earnings and wages
- The treatment group had higher earnings of $160 to $190 in quarters 4 through 6, but these did not translate to greater total earnings over the entire follow-up period.
Considerations for Interpreting the Findings
The study authors estimated multiple related impacts on UI benefits receipt, earnings, and employment outcomes. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated. In addition, the p-values reported in the study are based on one-tailed tests, which also likely overstates the number of statistically significant findings based on a more appropriate two-tailed test.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it is based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the job search assistance strategies tested, and not to other factors.