Absence of conflict of interest. This study was conducted by staff from Mathematica Policy Research, which administers CLEAR. Therefore, this study was reviewed by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Needels, K., Corson, W., & Van Noy, M. (2002). Evaluation of the Significant Improvement Demonstration Grants for the provision of reemployment services for UI claimants. Princeton, NJ: Mathematica Policy Research.
Highlights
- The study examined the impacts of the Significant Improvement Demonstration Grants—awarded to 11 states to improve reemployment services for Unemployment Insurance (UI) claimants—on weeks of UI receipt and UI benefit exhaustion.
- For each state, the authors compared outcomes before and at the end of the grant period using monthly data that states reported to the UI Service.
- The study found no statistically significant relationships between the Significant Improvement Demonstration Grants and UI benefit receipt outcomes.
- The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Significant Improvement Demonstration Grants; other factors are likely to have contributed.
Intervention Examined
Significant Improvement Demonstration Grants
Features of the Intervention
In 1999, the U.S. Department of Labor, Employment and Training Administration, awarded two-year grants to 11 states to improve their Worker Profiling and Reemployment Service systems and increase the effectiveness of reemployment services available to their UI claimants. Grantee states took a variety of approaches to meet these goals, including changing profiling mechanisms by expanding selection pools and enhancing selection methods; improving reemployment services such as workshops, assessments, and self-access services and increasing service capacity; and improving coordination among programs and agencies.
Features of the Study
The authors compared the average outcomes of UI claimants in grantee states during six-month periods in the two years before the grant and during the last six months of the grant period. They analyzed monthly data that states reported to the UI Service and reported findings separately for each state.
Study Sites
- Alaska
- California
- Louisiana
- Maryland
- Minnesota
- New York
- North Carolina
- South Carolina
- Washington
- West Virginia
- Wisconsin
Findings
Public Benefits Receipt
- The study found no statistically significant relationships between the Significant Improvement Demonstration Grants and UI benefit receipt outcomes.
Considerations for Interpreting the Findings
The authors compared the outcomes of UI claimants in grantee states measured before and at the end of the period in which their states engaged in the intervention. For these types of designs, the authors must observe outcomes for multiple periods before the intervention to rule out the possibility that participants had increasing or decreasing trends in the outcomes examined before states began implementing grant activities. For example, if states experiencing gradual increases in the number of weeks that UI claimants received UI benefits tended to apply for and receive the grants, we would anticipate further increases over time, even if they did not receive the grant. Without knowing the trends before grant award, we cannot rule this out. Therefore, the study receives a low causal evidence rating.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the Significant Improvement Demonstration Grants; other factors are likely to have contributed.