Absence of conflict of interest: This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Dickinson, K., Kreutzer, S., West, R., & Decker, P. (1999). Evaluation of Worker Profiling and Reemployment Services systems: Final report. Research and Evaluation Report Series 99-D. Washington, DC: U.S. Department of Labor, Employment, and Training Administration.
Highlights
- The study assessed the effectiveness of Worker Profiling and Reemployment Services (WPRS), a system for predicting which Unemployment Insurance (UI) recipients are at greatest risk of exhausting benefits and offering them early intervention services.
- The study used a nonexperimental design to compare the UI benefit receipt, employment, and earnings outcomes of WPRS participants in six states to the outcomes of similar UI claimants who did not receive WPRS. The main data source was state UI administrative records for nearly 457,000 UI claimants.
- The study found that WPRS reduced UI benefit weeks and amounts in three of the states during the benefit year. However, there were few statistically significant impacts on employment rates or earnings in the four quarters after the initial claim, and some of the impacts were in an unexpected direction.
- The quality of causal evidence presented in this report is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to WPRS, but other factors might also have contributed.
Intervention Examined
Profiling
Features of the Intervention
In 1993, Congress amended the Social Security Act to require states to establish a system to identify which UI claimants are most likely to exhaust all eligible benefits, in order to offer them services earlier in their unemployment spells and track their progress toward reemployment. In this study, UI claimants were identified and selected for referral to WPRS through a two-step process. First, UI claimants with a definite recall date or an exclusive union hiring hall agreement were screened out. Among those remaining, each state employed a statistical method to identify claimants at high risk of exhausting their UI benefits. The exact method differed across states, but the probability of exhausting benefits typically was predicted as a function of claimant characteristics, such as education and previous job tenure, characteristics of the local economy, and, in some cases, measures of claimants’ previous earnings.
WPRS services were intended to be individualized; in most states, however, they included a 30- to 60-minute orientation that explained WPRS participation requirements. This orientation could be followed by a required group workshop (typically lasting four or fewer hours) that provided labor market information, training in job search methods, referrals to job openings, and help preparing job application materials. Some states also required profiled claimants to meet one-on-one with an employment counselor to assess each claimant’s interests and aptitudes and, in some cases, to develop an individualized service plan.
Features of the Study
This nonexperimental study assessed the effectiveness of WPRS in six states: Connecticut, Illinois, Kentucky, Maine, New Jersey, and South Carolina. The treatment group was made up of claimants who were identified as having a high probability of benefit exhaustion and were referred to WPRS services. The comparison group was made up of claimants who were initially screened for WPRS but not referred to WPRS because of capacity constraints or local area discretion. The authors estimated the impact of WPRS services as the regression-adjusted differences in outcomes of 155,047 treatment group members and 301,920 comparison group members. Impact models were run separately for each state and typically controlled for age, gender, race/ethnicity, education, job tenure, industry, and occupation of previous job; earnings in pre-UI receipt base period; state unemployment office; calendar quarter of initial claim; and maximum benefit amount.
Findings
Public benefits receipt
- The study found that WPRS significantly reduced the average weeks of UI benefits received in Connecticut, Illinois, Maine, and New Jersey by 0.25 to 0.98 weeks. This translated into statistically significant reductions in the average total amount of UI benefits received in Illinois, Maine, and New Jersey from $64 to $139.
- In addition, WPRS significantly reduced the proportion of claimants who exhausted all available benefits in Connecticut, Maine, and New Jersey by 1.4 to 4.3 percentage points. However, WPRS increased the proportion who exhausted benefits in Kentucky by a statistically significant 4.1 percentage points.
Employment
- WPRS seemed to have few impacts on employment. The exceptions were New Jersey, where WPRS reduced the probability of employment in each of the four quarters following the initial claim, and South Carolina, where WPRS reduced employment in the fourth quarter after the initial claim, in each case by a small but statistically significant percentage.
Earnings and wages
- In addition, there were few impacts on earnings. Statistically significant, positive impacts on earnings were found for Maine in the first quarter and New Jersey in the second quarter after the initial claim. However, WPRS reduced earnings for South Carolina participants in the fourth quarter after the initial claim.
Considerations for Interpreting the Findings
The authors noted that WPRS-referred claimants were from 1 to 45 percentage points more likely than nonreferred claimants to receive at least one service beyond orientation, with a particularly large impact (as high as 42 percentage points) on the percentage attending a job workshop or job club. Nevertheless, the authors noted that the inconsistency of the labor market and UI benefit receipt impacts, as well as the variation in labor market impacts across states and during follow-up periods, provides little evidence of a strong, consistent WPRS effect.
Causal Evidence Rating
The quality of causal evidence presented in this report is moderate, because it is a well-implemented nonexperimental study. This means we are somewhat confident that the estimated effects are attributable to WPRS, but other factors might also have contributed.
Additional Sources
Dickinson, K., Decker, P., & Kreutzer, S. (2002). Evaluation of WPRS systems. In Randall W. Eberts, Christopher J. O’Leary, and Stephen A. Wandner (Eds.), Targeting Employment Services (pp. 61-90). Kalamazoo, MI: W.E. Upjohn Institute.