Absence of conflict of interest. This study was conducted by staff from MEF Associates, which administers CLEAR. Therefore, the review of this study was conducted by independent consultants trained in applying the CLEAR causal evidence guidelines.
Citation
Highlights
-
The study's objective was to examine the impact of two approaches to subsidized employment for Temporary Assistance for Needy Families (TANF) recipients: Paid Work Experience (PWE) in the nonprofit or public sector, and On-the-Job Training (OJT) in for-profit companies. Many outcomes were evaluated in the domains of employment, earnings and wages, education and training, public benefit receipt, and health and safety. This profile examines comparisons between the PWE group and the OJT group. The authors investigated similar research questions for two other comparisons, the profiles of which can be found here:
-
The study used a randomized controlled trial design. The impact analysis uses administrative data and survey data. The authors compared the outcomes of those receiving PWE to those receiving OJT.
-
The study examined impacts over 30 months and found that PWE participants were more likely to be employed, earned more, and had smaller TANF payments than OJT participants in the first year of follow-up, but none of these effects were significant for the last year of follow-up. However, PWE participants did receive less income from food stamps than OJT participants in the last year of follow-up.
-
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial.
Intervention Examined
Comparison between Paid Work Experience (PWE) and On-the-Job Training (OJT)
Features of the Intervention
Following the 2007-2009 economic recession, subsidized employment interventions gained substantial attention. In 2010, the U.S. Departments of Health and Human Services and Labor made substantial investments in studying 8 subsidized employment interventions aimed at disadvantaged populations, including both Paid Work Experience (PWE) and On-the-Job Training (OJT) approaches for TANF recipients. This study focuses on programs in Los Angeles County, which has a large TANF program and a diverse population.
This review focuses on both the PWE and OJT interventions.
PWE fully subsidizes the wages of individuals placed at employers in the nonprofit or public sector for up to six months. The South Bay Workforce Investment Board serves as an intermediary between the TANF participant and the employer and pays participants the minimum wage. The goal of this model is to provide work experiences that will help the participants find an unsubsidized job. The model also includes 16 hours of paid job search near the end of placements and case management services to assist participants in their search for an unsubsidized job.
OJT offers wage subsidies to for-profit employers who agree to place employees onto their payrolls after an initial two-month trial period, during which participants are paid by the South Bay Workforce Investment Board. If participants are shifted onto the employer's payroll, the employers receive a partial wage subsidy for each participant for up to an additional four months. This approach aims to simulate a "real world" work environment and hopes to move participants into long-term unsubsidized work with participating employers.
Features of the Study
The study used a randomized controlled trial design. The researchers used a lottery-like process to randomly assign 2,622 individuals to one of three groups: the PWE program group (n=874), an OJT program group (n=877), or the control group (n=871). All participants were TANF recipients in Los Angeles County who participated in the county’s welfare-to-work program, but were unable to find unsubsidized jobs during a four-week job club. To be included, participants had to be receiving TANF benefits and have five or more months left of TANF eligibility, must not have participated in Transitional Subsidized Employment in the preceding 12 months, and must have been able to work the required hours and have no major identified barriers that would prevent them from working.
Of the full sample, most (85.5%) of study participants were female, and more than 80% were non-White (55% Hispanic/Latino; 32% Black, non-Hispanic; 3% Asian, non-Hispanic). 39% of participants lacked a high school diploma or an equivalent credential. On average, participants were 31.7 years old.
Participants assigned to the PWE or OJT groups were referred to a Worksource Center to be placed into the corresponding type of subsidized job. Like those in the control group, they could still continue to receive TANF benefits, as well as other TANF welfare-to-work services and community services. Of those randomized to the PWE group, 79 percent were placed in subsidized employment, compared to 42 percent of those randomized to the OJT group. On average, PWE placements were made in 24 days and lasted 149 days, while OJT placements were made in 33 days and lasted about 84 days.
The impact analysis uses administrative data and survey data. Administrative data included quarterly wage data from the National Directory of New Hires and TANF and food stamp payment records from the Los Angeles Department of Public Social Services. Survey data were collected from the full study sample at approximately 30 months after random assignment; the survey included items about employment, education, training, and personal and economic well-being. The authors used a statistical model to compare the outcomes of those in OJT to those in the control group, accounting for initial characteristics.
Findings
Employment
-
PWE participants were more likely to have been employed during the first year of follow-up than OJT group participants (15.7 percentage points according to administrative data, 8.5 percentage points according to survey data), but no more likely to be employed during the last year of follow-up.
-
PWE participants were no more likely to be currently employed than OJT group members at the time of the 12-month survey or the 30-month survey.
-
PWE participants were employed for 0.8 more quarters, on average, than OJT group participants during the first year of follow-up, but there were no such differences in the last year of follow-up.
Earnings and wages
-
Participants in the PWE program earned $1,447 more than those in the OJT group during the first year following enrollment, and $2,371 more over the total follow-up period, but did not earn significantly more in the last year of follow-up.
-
PWE participants had a total income $1,105 more in the first year of follow-up than OJT group participants, but their total income was no greater in the last year of follow-up.
-
PWE participants were no more likely than OJT participants to have received income from working in the month prior to the 30-month survey, or to be earning greater than $12.00 or greater than $15.00 per hour at that time.
-
Among only those participants who worked in the year prior to enrollment, PWE participants did not earn any more in the first or last year of follow-up than OJT participants. There were no differences in combined income during either the first or the last year of follow-up either.
Education and skills gains
-
-Compared to those in the OJT group, participants recieving PWE were no more likely to be participating in postsecondary education at the time of the 30-month survey.
-
-PWE participants were similar to OJT participants in terms of highest level of school completed.
Health and safety
-
PWE participants were no more likely to have health insurance (in general or through an employer) than OJT participants.
-
There were no significant differences between PWE participants and OJT participants in terms of the percent reporting they were in ""good, very good, or excellent health"", the percent reporting experiencing serious psychological distress in the past month scores on the social support scale, or ratings of overall happiness.
Public benefits receipt
-
PWE participants received $283 less in TANF payments in the first year of follow-up compared to OJT participants, but received similar amounts from TANF during the last year of follow-up and for the total follow-up period.
-
There were no differences between the percentage of PWE and OJT participants that left TANF at the 12-month and 30-month follow-up periods.
-
PWE participants received $211 less in terms of income from food stamps in the last year of follow-up when compared to OJT participants.
-
There were no significant differences between the PWE and OJT groups in terms of the percent receiving the following: Supplemental Security Income, public assistance or welfare (, child support, child care subsidies or vouchers, benefits from Section 8 or other housing assistance, food stamps, or Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) benefits.
Training
-
Participants in the PWE program were no more likely than OJT participants to be participating in vocational training or to have earned a professional license or certification at the time of the 30-month survey.
Considerations for Interpreting the Findings
This profile describes multiple related impacts on outcomes related to: Employment; Earnings and wages; Education and skills gains; Training; Public benefit receipt; and Health and safety. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The findings presented do not include statistical adjustments to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
The authors also point out at that placement rates for the PWE and OJT programs differed across participating Worksource Centers. However, the authors were unable to study effects of the interventions at the individual center-level. For that reason, readers should recognize that the impacts of the PWE and OJT programs may vary significantly depending on the operational context in which it is delivered.
Authors also note that the composite income measure was unlikely to have captured all income received by individuals and did not include other household members' income. For those reasons, it likely underestimated the true total incomes of participants.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the differences between the two models and not to other factors.