Absence of conflict of interest.
Citation
Klerman, J. A., Saunders, C., Dastrup, E., Epstein, Z., Walton, D., Adam, T., & Barnow, B. S. (2019). Evaluation of impacts of the Reemployment and Eligibility Assessment (REA) Program: Final report. Cambridge, MA: Abt Associates. [Comparison between existing REA services group and control group]
Highlights
- The study’s objective was to examine the impact of the Reemployment and Eligibility Assessment (REA) program in four states on people’s public benefit receipt, employment, and earnings. The authors investigated similar research questions for three other contrasts, the profiles of which are available using the study search.
- The authors used a randomized controlled trial to compare the public benefit, earnings, and employment outcomes among unemployment insurance (UI) claimants randomly assigned to each state’s existing REA program or a control group. The authors drew on state administrative records from the UI benefit and case management systems for 28 weeks after random assignment and National Directory of New Hires data for two years after random assignment.
- The study found that, across the states, the existing REA treatment group received significantly fewer weeks and dollars of UI benefits than the control group did, and the existing REA treatment group had higher employment, longer job tenure, and higher earnings than the control group in the first or second year after random assignment.
- The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors.
Intervention Examined
The Reemployment and Eligibility (REA) Program
Features of the Intervention
Since REA’s inception in 2005, 40 states have implemented the initiative to encourage rapid reemployment of UI claimants through a combination of in-person eligibility reviews and employment-focused case management. The set of REA programs examined in this study—in Indiana, New York, Washington, and Wisconsin—provided two types of services to UI claimants: (1) an initial, mandatory REA meeting shortly after claimants start to receive UI benefits, which included review of ongoing UI eligibility, and (2) reemployment assistance (including a one-on-one meeting in person at an American Job Center, labor market information, development of a reemployment plan, orientation to services available at American Job Centers, and registration to the state job bank) as well as referrals to reemployment services such as interview or resume-writing workshops. Some of the programs examined in this study also provided a third component: additional REA meetings, in person or by phone, that were typically shorter than the initial REA meeting and included another eligibility review, updates to the reemployment plan, and additional labor market information.
Features of the Study
The authors used a randomized controlled trial to estimate the impacts of the REA program. Within each of the four states, UI claimants were randomly assigned to one of four conditions (though the specific components of some conditions varied somewhat between states): control, partial REA treatment group, single REA treatment group, and multiple REA treatment group. Those in the single REA treatment group were invited to a mandatory, initial REA meeting that included reemployment assistance and referral to at least one reemployment service. The multiple REA treatment group added to the single REA condition an invitation to one or two additional meetings. The control group was not required to attend an REA meeting and did not receive any referrals to reemployment services. Each state had an existing REA condition before random assignment; the existing programs in New York, Washington, and Wisconsin were multiple REA, and the existing program in Indiana was single REA. This contrast compares the existing REA treatment group against the control group.
In total across all four states, 214,142 claimants were randomly assigned from March 2015 to April 2016. For the pooled sample across all four states, the analytic sample included 141,227 existing REA treatment group members and 59,295 control group members. The analytic sample included 32,130 people from Indiana, 131,154 people from New York, 21,122 people from Washington, and 16,116 people from Wisconsin.
The authors used statistical models to compare the outcomes of claimants assigned to the two conditions, accounting for individual background characteristics, including demographic characteristics and earnings and UI benefit receipt history, and state and county characteristics, including urban status and unemployment rates. They reported findings across all states together and for each state separately. The study used UI benefit and case management systems data to assess UI benefit receipt and National Directory of New Hires data for employment, UI benefit receipt, and earnings outcomes.
Findings
Public benefit receipt
- The study found that the existing REA treatment group received 1.3 fewer weeks of UI benefits than the control group 28 weeks after random assignment, which was a significant difference. There was also a significant difference between the existing REA group and the control group in each state’s analyses for this outcome.
- The study also found that the existing REA treatment group received 0.1 fewer quarters of UI benefits than the control group in the first year after random assignment and 0.02 fewer quarters in the second year after random assignment; both findings were statistically significant. In the first year after random assignment, the existing REA treatment group had fewer quarters of UI benefits received in each state’s analysis, but only Indiana and Washington’s existing REA treatment groups had fewer quarters of UI benefits received in the second year after random assignment.
- The existing REA treatment group received $347 fewer in UI benefits than the control group 28 weeks after random assignment; this was a significant difference. This outcome was also significant in each state’s analysis.
Employment
- The study found that the existing REA treatment group was employed for 0.06 more quarters than the control group in the first year after random assignment and 0.02 more quarters in the second year after random assignment. Both findings were statistically significant. In the first year after random assignment, the existing REA group was employed significantly more than the control group in all states but Wisconsin; in the second year after random assignment, the existing REA treatment group was employed significantly more only in New York.
- The existing REA treatment group had significantly higher job tenure (0.05 months) than the control group in the two years after random assignment across the four states. This finding was also significant in the New York and Indiana state results.
Earnings
- The existing REA treatment group earned $465 more in the first year after random assignment than the control group, a significant difference. The treatment group also earned significantly more in Indiana and New York compared with the control group. The pooled results across the four states and the four states’ individual results were not significant for earnings in the second year after random assignment.
Considerations for Interpreting the Findings
The authors found that less than 60 percent of claimants attended the initial REA meeting on time, and about 27 percent never attended. In addition, the authors found that the proportion of people in the multiple REA treatment group scheduled for second and third REA meetings varied across the states. In all, 72 percent of people in the multiple REA group in New York were scheduled for second meetings compared with just 27 percent in Wisconsin. And 48 percent of people in the multiple REA group New York were scheduled for third meetings compared with just 4 percent in Wisconsin. Taken together, these implementation findings suggest that there might be less contrast between the services received by the existing REA and control conditions than expected.
The study authors estimated multiple related impacts on outcomes related to public benefit receipt, earnings, and employment. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the REA program and not to other factors .