Citation
Jackson, R., Dixon, R., McCoy, A., Pistorino, C., Zador, P., Thomas, C., … Bruno, L. (2007). Youth Opportunity Grant Initiative: Impact and synthesis report. Houston, TX: Decision Information Resources, Inc.
Highlights
- This report’s objective was to examine the effectiveness of the Youth Opportunity (YO) Grant Initiative, which sought to improve job prospects and education outcomes for youth in high-poverty areas by providing them with a mix of 15 youth development activities. Companion reports describe the initiative’s implementation.
- The initiative’s effectiveness was evaluated using two separate comparison group designs.
- In general, the study found positive and statistically significant relationships between the YO initiative and educational enrollment outcomes. However, there were few statistically significant effects on employment-related outcomes and educational attainment.
- The quality of the causal evidence presented in this study is low. This means we are not confident that the estimated impacts were caused by the YO Grant Initiative, and not other factors.
Intervention Examined
Youth Opportunity Grant Initiative
Features of the Intervention
The YO Grant Initiative operated from 2000 until 2005. Thirty-six communities (24 urban, 6 rural, 6 Native American) provided comprehensive services to more than 92,000 economically disadvantaged youth, ages 14 to 21. Services were available regardless of income, but the program’s goal was to target youth programming in areas with high levels of joblessness and poverty, especially among out-of-school youth, and to serve as a foundation for community-wide efforts to mobilize resources for supporting youth to enter the economic mainstream.
The 36 grantees, selected by a competitive application process, provided services to both in-school and out-of-school youth in the community. The program’s core activities included job readiness training, internships or subsidized employment, short-term occupational skills training, short-term unsubsidized jobs, and GED preparation. Most youth participated in at least one core service as well as other services designed to support them, such as sports and recreation. On average, participants spent 563 hours participating in program activities at the study sites.
Features of the Study
The study used two different nonexperimental designs to examine the effectiveness of YO. The first design was a census tract-level analysis using data from the American Community Survey (ACS) to form a comparison group. The second design was an individual-level analysis using data drawn from the Current Population Survey (CPS).
For the analysis using the ACS data, the authors first grouped 29 non-Native American YO sites and comparison census tracts into bins based on their similarity in terms of their population, percentage of home ownership, median contract rent, percentage of vacant housing units, percentage of whites in the population, percentage of rural population, and labor force participation rate. The authors then estimated YO’s impact by comparing the changes in employment and education outcomes from baseline (2001) to follow-up (2003–2004) for the YO and comparison communities within each bin. These were combined across bins to calculate an overall estimate.
For the analysis using the CPS data, the YO group consisted of all youth ages 16 to 21 who lived in the target areas of 23 of the 24 urban YO sites, because they were all eligible to participate in YO. The comparison group consisted of similarly aged youth in high-poverty (defined as poverty above 20 percent according to the 1990 Census) central-city neighborhoods. The authors estimated YO’s impact by comparing the changes in employment and education outcomes between April 2000 and March 2004 for youth in the YO and comparison communities.
Study Sites
- Albany, GA*
- Baltimore, MD
- Birmingham, AL
- Boston, MA
- Brawley/Calipatria, CA*
- Brockton, MA
- Buffalo, NY
- Chicot/Desha County, AR*
- Cleveland, OH
- Denver, CO
- Detroit, MI
- Hartford, CT
- Houston, TX
- Kansas City, MO
- Los Angeles, CA
- Louisville, KY
- Maui & Molokai, HI*
- Memphis, TN
- Milwaukee, WI
- Monroe, LA*
- Philadelphia, PA
- Portland, OR
- Robeson County, NC*
- San Antonio, TX
- San Diego, CA
- San Francisco, CA
- Seattle, WA
- Tampa, FL
- Tucson, AZ
* indicates rural sites
Findings
- The study examined 11 employment-related outcomes across the two designs. Of these, the study found a statistically significant positive relationship between living in a YO target area and the labor force participation rate and percentage of those employed working full time in the CPS analysis. There were no statistically significant differences between the groups on the other 9 employment-related outcomes examined across the two studies.
- The study examined 5 educational attainment outcomes in the ACS analysis. Of these, the study found a statistically significant positive relationship between participation in the YO program and having completed 11th grade.
- The authors examined 8 educational enrollment outcomes across the two designs. The authors found that participation in YO was associated with a statistically significant reduction in the percentage of youth not in school and an increase in the percentage of youth in secondary school (ACS analysis). They also found that living in a YO target area was associated with a statistically significant reduction in high school dropout spells (CPS analysis).
- The authors also examined program impacts by gender, race/ethnicity, and nativity. Of note, the study found statistically significant increases in the labor force participation rates and mean hourly wages of women associated with living in a YO target area.
Considerations for Interpreting the Findings
Although the ACS analysis attempted to match YO communities to other similar communities, differences in labor force participation rates remained and were not controlled for in the analysis. This could have been mitigated if there were similar trends in those rates before the study period, but this was not demonstrated. Also, although participants spent an average of more than 500 hours participating in YO programs, the total time spent varied widely among grantees and among individuals within grantees.
Similarly, the high-poverty communities selected to be part of the CPS analysis had very different racial compositions from the YO target areas. In addition, for this analysis there was no indication that YO target areas and comparison areas had similar levels or trends in employment and education-related outcomes at the start of the study period. These factors indicate that the comparison group does not provide a valid representation of what would have happened to the YO group in the absence of YO. Finally, the effects of YO may be diluted because youth who did not participate in YO but lived in YO treatment areas were included in the treatment group.
Causal Evidence Rating
The quality of the causal evidence presented in the study is low. This means we are not confident that the estimated impacts were caused by the YO Grant Initiative, and not other factors. To provide more convincing causal evidence that meets CLEAR criteria, the study could have included additional controls for the level and trend in labor force participation rates in the ACS analysis.