Citation
Bloom, D., Gardenhire-Crooks, A., & Mandsager, C. (2009). Reengaging high school dropouts: Early results of the National Guard Youth ChalleNGe program evaluation. New York: MDRC.
Highlights
- This report presents results from a 9-month follow-up study of the National Guard Youth ChalleNGe program, a disciplinary and educational intervention for unemployed youth ages 16 to 18 who are not in school. Related reports examine outcomes at 21 months and 3 years post-intervention.
- The program’s effectiveness was evaluated using a randomized controlled trial conducted in 10 states. This report examined the effects of the program on participants’ educational attainment, earnings, employment, and other outcomes measured through a survey conducted about 9 months after participants entered the study.
- The study found statistically significant, positive impacts of the ChalleNGe program on employment, education, and training outcomes measured after 9 months.
- The quality of causal evidence presented in this study is high because it is a well-implemented randomized controlled trial. This means we are confident that the effects estimated in this study are attributable to the National Guard Youth ChalleNGe program, and not to other factors.
Intervention Examined
The National Guard Youth ChalleNGe Program
Features of the Intervention
The National Guard Youth ChalleNGe program began in the early 1990s with a mission to target at-risk youth and equip them with the skills and training to have successful adult lives. To be eligible, youth must be 16 to 18 years of age, have dropped out of or been expelled from school, be unemployed, not be drug users, and not be heavily involved in the criminal justice system.
The 17-month program consists of a two-week Pre-Challenge phase, a 20-week Residential phase, and a one-year Post-Residential phase. Participants live in barracks-style housing (sometimes on a military base) in a very disciplined environment during the first two phases. They wear their hair short, are referred to as cadets, and wear military uniforms. In the Pre-Challenge phase, participants are oriented to the program’s rules and begin physical training. During the Residential phase, they participate in a number of different activities addressing eight core pillars: leadership/followership, responsible citizenship, service to community, life-coping skills, physical fitness, health and hygiene, job skills, and academic excellence. Most of their time is spent in an educational component that is usually geared toward receiving a General Educational Development (GED) certificate. During the Post-Residential phase of the program, after participants are placed in employment, education, or military service, they receive structured mentoring.
Features of the Study
The study was a randomized controlled trial conducted at 10 program sites. In 2005 and 2006, about 3,000 eligible applicants were randomly assigned to either receive an offer to participate in the program (the treatment group) or to not receive an offer for the program in the current or future application cycles (the control group). For the 9-month survey, the authors attempted to survey a subsample of those originally randomly assigned (755 treatment and 446 control youth) and ultimately surveyed 648 treatment and 370 control youth. To estimate the effectiveness of the ChalleNGe program, the authors compared the outcomes of the treatment group with outcomes of the control group.
This study examined the effects of the ChalleNGe program on education, employment, and training outcomes, relative to the effects of other employment and training programs used by control group youth, measured about 9 months after random assignment. The study also examined outcomes on military enlistment, delinquency and criminal activity, health, sexual activity, and substance use.
Study Sites
- California
- Florida
- Georgia
- Illinois
- Michigan
- Mississippi
- New Mexico
- North Carolina
- Texas
- Wisconsin
Findings
- The study found statistically significant, positive impacts of the ChalleNGe program on employment, education, and training outcomes measured after 9 months.
- Nine months after randomization, 31 percent of youth in the treatment group were working full time compared to 21 percent of those in the control group. Fifty-one percent of youth in the treatment group were working any hours compared to 42 percent of those in the control group. These differences were statistically significant.
- Youth in the treatment group were significantly more likely than youth in the control group to have earned a high school diploma or GED certificate (46 percent versus 10 percent).
- Control group members were significantly more likely than youth in the treatment group to be in high school (36 percent compared to 16 percent) and significantly less likely to be enrolled in college courses (3 percent compared to 11 percent).
- Thirty-three percent of youth in the treatment group had ever received job training compared to 23 percent of youth in the control group, a statistically significant difference.
Considerations for Interpreting the Findings
Because this study only included youth who chose to apply for the program, these results cannot be assumed to apply to any youth who could be forced to attend the program. Youth who applied to the program may differ from other youth in a meaningful way. For example, youth who applied to ChalleNGe may have higher motivation than the “average” youth.
Not all youth who were randomly assigned to receive an offer to participate in the program did ultimately participate. However, youth were included in the treatment group regardless of whether they successfully completed the program. This could have reduced the estimated impact of the program. Additionally, sites were not randomly selected for participation in the study. Therefore, casual conclusions can only be drawn for the subset of sites that participated in this study and not the ChalleNGe program nationally.
The authors estimated multiple related impacts on outcomes related to education, employment, and earnings. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. The authors did not perform statistical adjustment to account for the multiple tests, so the number of statistically significant findings in these domains is likely to be overstated.
Causal Evidence Rating
The quality of causal evidence presented in this report is high because it was based on a well-implemented randomized controlled trial. This means we are confident that the estimated effects are attributable to the National Guard Youth ChalleNGe program, and not to other factors.