Citation
Millenky, M., Bloom, D., Muller-Ravett, S., & Broadus, J. (2011). Staying on course: Three-year results of the National Guard Youth ChalleNGe Evaluation. New York: MDRC.
Highlights
- This report presents results from a three-year follow-up study of the National Guard Youth ChalleNGe program, a disciplinary and educational intervention for unemployed youth ages 16 to 18 who are not in school. Earlier reports examined outcomes at 9 and 21 months.
- The program’s effectiveness was evaluated using a randomized controlled trial conducted in 10 states. This report examined the effects of the program on participants’ educational attainment, earnings, employment, and other outcomes measured through a survey conducted about three years after participants entered the study.
- The study found statistically significant, positive impacts of the ChalleNGe program on employment, earnings, education, and training outcomes measured after three years.
- The quality of causal evidence presented in this study is high because it is a well-conducted randomized controlled trial. This means we have confidence that the effects estimated in this study are attributable to the National Guard Youth ChalleNGe program.
Intervention Examined
The National Guard Youth ChalleNGe Program
Features of the Intervention
The National Guard Youth ChalleNGe program began in the early 1990s with a mission to target at-risk youth and equip them with the skills and training to have successful adult lives. To be eligible, youth must be 16 to 18 years of age, have dropped out of or been expelled from school, be unemployed, not be drug users, and not be heavily involved in the criminal justice system.
The 17-month program consists of a two-week Pre-Challenge phase, a 20-week Residential phase, and a one-year Post-Residential phase. Participants live in barracks-style housing (sometimes on a military base) in a very disciplined environment during the first two phases. They wear their hair short, are referred to as cadets, and wear military uniforms. In the Pre-Challenge phase, participants are oriented to the program’s rules and begin physical training. During the Residential phase, they participate in a number of different activities addressing eight core pillars: leadership/followership, responsible citizenship, service to community, life-coping skills, physical fitness, health and hygiene, job skills, and academic excellence. Most of their time is spent in an educational component that is usually geared toward receiving a General Educational Development (GED). During the Post-Residential phase of the program, after participants are placed in employment, education, or military service, they receive structured mentoring.
Features of the Study
The study was a randomized controlled trial conducted at 10 program sites in California, Florida, Georgia, Illinois, Michigan, Mississippi, New Mexico, North Carolina, Texas, and Wisconsin. In 2005 and 2006, about 3,000 eligible applicants were randomly assigned to either receive an offer to participate in the program (the treatment group) or to not receive an offer for the program in the current or future application cycles (the control group). For the three-year survey, the authors attempted to survey a subsample of those originally randomly assigned (916 treatment and 592 control youth). To estimate the effectiveness of the ChalleNGe program, the authors compared the outcomes of the treatment group with outcomes of the control group.
This study examined the effects of the ChalleNGe program on education, employment, and training outcomes, relative to the effects of other employment and training programs used by control group youth, measured about three years after random assignment. The study also examined outcomes on military enlistment, delinquency and criminal activity, health, sexual activity, and substance use.
Findings
- The authors reported statistically significant, positive impacts of the ChalleNGe program on earning a high school diploma and/or a GED certificate, earning any college credit, enrolling in college courses at the time of the follow-up survey, and receiving vocational training.
- One notably large impact was on the percentage of youth who had earned a high school diploma and/or GED: 72 percent of youth admitted to the ChalleNGe program, compared with 56 percent of control youth.
- The authors also reported statistically significant, positive impacts of the ChalleNGe program on five of the ten employment or earnings outcomes examined. One notable impact was that youth admitted to the ChalleNGe program had average earnings in the past 12 months of about $13,500, compared with an average of $11,250 among the control group.
- Other positive and statistically significant impacts of ChalleNGe were reported on the number of months employed in the previous 12 months, the percentage employed at the time of survey administration, the current average weekly earnings, the percentage currently working full-time, the percentage with an hourly wage under $6, and the percentage with an hourly wage of $10 or more.
Considerations for Interpreting the Findings
Not all youth who were randomly assigned to receive an offer to participate in the program did ultimately participate. However, youth are included in the treatment group regardless of whether they successfully completed the program. This could reduce the estimated impact of the program.
The authors estimated multiple related impacts on education, employment, and earnings. Performing multiple statistical tests on related outcomes makes it more likely that some impacts will be found statistically significant purely by chance and not because they reflect program effectiveness. CLEAR performed statistical adjustments to account for the multiple tests and the results remained statistically significant.
Causal Evidence Rating
The quality of causal evidence presented in this study is high because it is based on a well-conducted randomized controlled trial. This means we have confidence that the effects estimated in this study are attributable to the National Guard Youth ChalleNGe program, and not to other factors.
Additional Sources
Millenky, M., Schwartz, S., & Rhodes, J. (2013). Supporting the transition to adulthood among high school dropouts: An impact study of the National Guard Youth Challenge program. Prevention Science. doi: 10.1007/s11121-013-0388-4
Millenky, M., Bloom, D., Muller-Ravett, S., & Broadus, J. (2011). Staying on course: Three-year results of the National Guard Youth ChalleNGe evaluation. Executive summary. New York, NY: MDRC.