This study was conducted by staff from ICF, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Highlights
- The study’s objective was to examine the impact of NAF on high school graduation rates.
- The study used a nonexperimental design to compare the outcomes of students who participated in NAF to a matched group of students who did not. Using student level data and NAF academy level data, the authors conducted statistical tests to compare the outcomes between the groups.
- The study found a significant relationship between participating in NAF and a higher likelihood of graduating from high school.
- This study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to NAF; other factors are likely to have contributed.
Intervention Examined
National Academy Foundation (NAF)
Features of the Intervention
NAF was a national network of education, business, and community leaders that was created in 1982 to help high school students become college, career, and future ready. NAF consisted of five career themed learning academies that provide a career themed learning curriculum that is integrated within the core academic curriculum and instructional work-based learning. The five career themes included finance, information technology, engineering, health sciences, and hospitality and tourism. Academies included personalized learning environments, career and college focused education, and provided supports such as tutoring, mentoring, skills workshops, and credit recovery. NAF academies were based within traditional high schools that were most often in high need communities. NAF targeted students who were at risk of not graduating high school in four years.
Features of the Study
The study used a matched comparison group design to examine the impact of NAF on high school graduation rates. The treatment group were freshman students during the 2011-2012 school year who received NAF education services, while the comparison group were freshman students who received traditional school education during the same school year. Study authors matched treatment students to comparison students on similar school district and observable student characteristics. The matched sample included 7,367 treatment students and 14,679 comparison students. Over half of the sample were male (58%) and Hispanic (51%), and most were eligible for free and reduced lunch (72%). The primary data sources were student level data and NAF academy level data that was provided for each academic year during the length of the study. The authors used statistical tests to compare differences in outcomes between students in the treatment and comparison groups.
Study Sites
There were 10 school districts participating in the study.
- Porterville, CA
- Pasadena, CA
- Hartford, CT
- Broward, FL
- Miami-Dade, FL
- Charlotte-Mecklenburg, NC
- New York City, NY
- Rochester, NY
- Dallas, TX
- Waco, TX
Findings
Education and skill gains
- The study found that significantly more treatment students (79%) graduated from high school than comparison students (76%).
Considerations for Interpreting the Findings
The study authors were limited in their access to baseline data for students during their eighth-grade year, so the students were matched based on their academic performance at the end of ninth grade. The authors also matched on gender, race/ethnicity, free and reduced lunch status (an indicator of financial disadvantage), and English language learner status. However, they did not match the students on age or control for it in their analyses as required by CLEAR. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs.
Causal Evidence Rating
The quality of causal evidence presented in this report is low because the author did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to NAF; other factors are likely to have contributed.