Citation
Black, D., Smith, J., Berger, M., & Noel, B. (2003). Is the threat of reemployment services more effective than the services themselves? Evidence from random assignments in the UI system. American Economic Review, 93(4), 1313-1327.
Highlights
- The study examined the impact of the Kentucky Worker Profiling and Reemployment Services (WPRS) on Unemployment Insurance (UI) benefit receipt and earnings.
- In this study, about 2,000 UI claimants in Kentucky were randomly assigned to the treatment group, which was required to participate in reemployment services, or to the control group, which was not required to participate but could receive services voluntarily. The authors collected administrative data on UI benefit receipt and earnings for all study participants.
- The study found that those in the WPRS treatment group had a statistically significant reduction in UI benefit receipt of 2.2 weeks during the six-quarter follow-up period, compared to the control group. There were no statistically significant impacts of the program on the fraction of recipients exhausting benefits or total UI benefits received.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial (RCT). This means we are confident that the estimated effects are attributable to the WPRS and not to other factors.
Intervention Examined
Profiling
Features of the Intervention
At the time of this study, recent UI claimants in Kentucky who did not have a definite recall date and who were not subject to a union hiring hall agreement were assigned profiling scores reflecting their predicted duration of UI benefits. Those with high profiling scores were selected for WPRS and received a letter informing them that they were required to attend reemployment services. Within 10 business days of receiving the letter, claimants had to attend an orientation, where they completed a questionnaire. Staff used the information from the questionnaire to refer each claimant to specific services. Staff referred about 77 percent of claimants to less-intensive job search and job preparation activities (about four to six hours of programming), and 14 percent to more intensive services. Staff could assign claimants to more than one job search activity.
Features of the Study
The study included data from people who filed initial UI claims from October 1994 through June 1996. In most cases, the local UI office had enough resources to provide mandatory WPRS to all claimants with high profiling scores., but in some cases there were not enough resources to serve all the claimants with high profiling scores. In those cases, claimants were randomly assigned to receive or not receive the mandatory services. In all, 1,236 claimants were randomly assigned to the treatment group, which was mandated to receive WPRS, and 745 were randomly assigned to the control group, which did not have to participate in WPRS. For each participant in the study, the authors collected UI administrative records from Kentucky for six quarters after random assignment. The authors compared the outcomes of treatment and control members using a regression model that controlled for the time of random assignment, the claimants’ local UI office, and the profiling score. The authors also estimated a regression model that weighted estimates based on random assignment ratios and profiling score groups.
Findings
Public benefits receipt
- The study found that those in the WPRS treatment group had a statistically significant reduction in UI benefit receipt of 2.2 weeks during the six-quarter follow-up period, compared to the control group. The second estimation method also found a statistically significant reduction.
- The first regression model found no statistically significant impacts of WPRS on the percentage of recipients exhausting benefits or on the total UI benefits received. The second model, however, found a 3 percentage point reduction in the percentage of recipients exhausting benefits.
Earnings and wages
- The post-UI claim earnings of treatment group members appeared to be higher among the WPRS group than the control group: $1,054 in the first model, which was not statistically significant, and $1,600 in the second model, which was statistically significant.
Considerations for Interpreting the Findings
Of those referred to services in the treatment group, only 61 percent attended and completed one of the prescribed services. Thirty-two percent of those referred to services in the treatment group did not receive any services. Supplemental analyses showed that the impacts on reduced UI benefit receipt occurred primarily in the first two weeks, corresponding to the period when the letters were mailed to claimants informing them of the requirement to participate in job search activities. Likewise, the earnings impacts were achieved in the first two quarters. These early (pre-service receipt) impacts, together with low participation rates in actual services, suggest that the participation mandate, rather than the reemployment services themselves, was primarily responsible for the impacts.
Because this study took place when the economy was particularly strong, not many local offices were oversubscribed. At offices that were not oversubscribed, all claimants received the treatment services but were not included in this study. The local offices that were oversubscribed and included in the study may not be representative of the full group of offices across the state.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented RCT. This means we are confident that the estimated effects are attributable to the WPRS and not to other factors.
Additional Sources
Black, D., Smith, J., Berger, M., & Noel, B. (2002). Is the threat of reemployment services more effective than the services themselves? Experimental Evidence from the UI System. Cambridge, MA: National Bureau of Economic Research.
Galdo, J. C. (2006). Three essays in programme evaluation. (Doctoral dissertation). Retrieved from ProQuest. (Accession No. 304941896).