Absence of conflict of interest. This study was conducted by staff from MEF Associates, which administers CLEAR. Therefore, the review of this study was conducted by independent consultants trained in applying the CLEAR causal evidence guidelines.
Citation
Highlights
-
The study's objective was to examine the impact of two approaches to subsidized employment for Temporary Assistance for Needy Families (TANF) recipients: Paid Work Experience (PWE) in the nonprofit or public sector, and On-the-Job Training (OJT) in for-profit companies. Many outcomes were evaluated in the domains of employment, earnings and wages, education and training, public benefit receipt, and health and safety. This profile examines comparisons between the PWE group and a control group. The authors investigated similar research questions for two other comparisons, the profiles of which can be found here:
-
The study used a randomized controlled trial design. The impact analysis uses administrative data and survey data. The authors compared the outcomes of those receiving PWE to those in the control group.
-
The study found that PWE participants were more likely to be employed, earned more, had smaller TANF payments, and were more likely to have left TANF than control group participants in the first year of follow-up, but none of these effects were significant for the last year of follow-up. However, PWE participants were more significantly likely to work more than 34 hours per week than control group members at the time of the 30-month survey, more likely to have received income from working in the month prior, less likely to have received public assistance or welfare in the month prior, and more likely to have health insurance coverage through their employer.
-
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial.
Intervention Examined
Paid Work Experience (PWE)
Features of the Intervention
Following the 2007-2009 economic recession, subsidized employment interventions gained substantial attention. In 2010, the U.S. Departments of Health and Human Services and Labor made substantial investments in studying 8 subsidized employment interventions aimed at disadvantaged populations, including both Paid Work Experience (PWE) and On-the-Job Training (OJT) approaches for TANF recipients. This study focuses on programs in Los Angeles County, which has a large TANF program and a diverse population.
This review focuses on the PWE intervention. PWE fully subsidizes the wages of individuals placed at employers in the nonprofit or public sector for up to six months. In Los Angeles County, the South Bay Workforce Investment Board serves as an intermediary between the TANF participant and the employer and pays participants the minimum wage. The goal of this model is to provide working experiences that will help the participants find an unsubsidized job. The model also includes 16 hours of paid job search near the end of placements, including case management services to assist participants in their search for an unsubsidized job.
Features of the Study
The study used a randomized controlled trial design. The researchers used a lottery-like process to randomly assign 2,622 individuals to one of three groups: the PWE program group (n=874), an OJT program group (n=877), or the control group (n=871). All participants were TANF recipients in Los Angeles County who participated in the county’s welfare-to-work program, but were unable to find unsubsidized jobs during a four-week job club. To be included, participants had to be receiving TANF benefits and have five or more months left of TANF eligibility, must not have participated in Transitional Subsidized Employment in the preceding 12 months, and must have been able to work the required hours and have no major identified barriers that would prevent them from working.
Of the full sample, most (85.5%) of study participants were female, and more than 80% were non-White (55% Hispanic/Latino; 32% Black, non-Hispanic; 3% Asian, non-Hispanic). 39% of participants lacked a high school diploma or an equivalent credential. On average, participants were 31.7 years old.
Participants assigned to the PWE group were referred to a Worksource Center to be placed into the corresponding type of subsidized job. Like those in the control group, they could continue to receive TANF benefits, as well as other TANF welfare-to-work services and community services. Of those randomized to the PWE group, 79 percent were placed in subsidized employment. On average, PWE placements were made in 24 days and lasted 149 days. Individuals in the control group were not placed into the subsidized job, but they could still receive TANF benefits, other TANF welfare-to-work services and community services.
The impact analysis uses administrative data and survey data. Administrative data included quarterly wage data from the National Directory of New Hires and TANF and food stamp payment records from the Los Angeles Department of Public Social Services. Survey data were collected from the full study sample at approximately 30 months after random assignment, including information about employment, education, training, and personal and economic well-being. The authors used a statistical model to compare the outcomes of those in PWE to those in the control group, accounting for initial characteristics.
Findings
Employment
-
PWE participants were more likely to have been employed during the first year of follow-up than control group participants (34.1 percentage points according to administrative data, 23.0 percentage points according to survey data), but no more likely to be employed during the last year of follow-up. However, among participants who were not employed in the year prior to enrollment, participants remained more likely to be employed in the last year of follow-up.
-
PWE participants were no more likely to be currently employed at the time of the 12-month or 30-month survey than control group members.
-
However, PWE participants were employed for 1.3 more quarters, on average, than control group participants during the first year of follow-up and 0.2 more quarters in the last year of follow-up, and were 8.2 percentage points more likely to work more than 34 hours per week at the time of the 30-month survey.
Earnings and wages
-
Participants in the PWE program earned $2,716 more than those in the control group during the first year following enrollment, and $3,914 more in the total 30-month follow-up period.
-
PWE participants had a total income $2,189 more in the first year of follow-up than control group participants, but their total income was no greater in the last year of follow-up.
-
PWE participants were 5.6 percentage points more likely to have received income from working in the month prior to the 30-month survey, but were no more likely to be earning greater than $12.00 or greater than $15.00 per hour at that time.
Education and skills gains
-
Compared to those in the control group, participants receiving PWE were no more likely to be participating in postsecondary education at the time of the 30-month survey.
-
PWE participants were similar to the control group in terms of highest level of school completed.
Health and safety
-
PWE participants were 4.9 percentage points more likely to have health insurance coverage through their employer than control group participants, but no more likely to have health insurance in general.
-
There were no significant differences between PWE participants and control participants in terms of the percent reporting they were in ""good, very good, or excellent health"", the percent experiencing serious psychological distress in the past month, scores on the social support scale, or ratings of overall happiness.
Public benefits receipt
-
PWE participants received $464 less in TANF payments in the first year of follow-up compared to control group members, and $820 less during the total follow-up period, but received similar amounts from TANF during the last year of follow-up.
-
Similarly, PWE participants were 4.8 percentage points more likely to have left TANF in the year following enrollment, but effects were no longer significant for the 30-month follow-up period.
-
PWE participants were also 5.4 percentage points less likely to have received public assistance or welfare in the month prior to the 30-month survey as compared to control group members. However, there were no significant differences in terms of income received from food stamps or unemployment insurance, or the percent receiving the following: Supplemental Security Income, child support, childcare subsidies or vouchers, benefits from Section 8 or other housing assistance, food stamps, or Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) benefits.
Training
-
Participants in the PWE program were no more likely than control group participants to be participating in vocational training or to have earned a professional license or certification at the time of the 30-month survey.
Considerations for Interpreting the Findings
This profile describes multiple related impacts on outcomes related to: Employment; Earnings and wages; Education and skills gains; Training; Public benefit receipt; and Health and safety. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The findings presented do not include statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
The authors also point out at that placement rates for the PWE program differed across participating Worksource Centers. However, the authors were unable to study effects of the interventions at the individual center-level. For that reason, readers should recognize that the impacts of the PWE program may vary significantly depending on the operational context in which it is delivered.
Authors also note that the composite income measure was unlikely to have captured all income received by individuals and did not include other household members' income. For those reasons, it likely underestimated the true total incomes of participants.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the Paid Work Experience program and not to other factors.