McClanahan, W., Sipe, C., & Smith, T. (2004). Enriching summer work: An evaluation of the Summer Career Exploration Program. Philadelphia, PA: Public/Private Ventures.
- The evaluation’s objective was to examine the effects of the Summer Career Exploration Program (SCEP) on short-term earnings, employment, and other outcomes. SCEP offers youth job-related counseling, places them in career-related jobs, and supports them through mentoring. The report also describes program implementation and ways of improving SCEP.
- The program’s effectiveness was evaluated using a randomized control ed trial.
- The study found statistically significant, positive impacts of SCEP on participants’ employment and earnings over the summer that the evaluation occurred; however, these were not sustained over the one-year follow-up period.
- The quality of the causal evidence presented in this study is high for most outcomes and moderate for the remaining outcomes examined. For the outcomes with high causal evidence, this means we are confident that the estimated impacts are attributable to SCEP, and not other factors. For those with moderate causal evidence, other factors besides SCEP might have contributed to the estimated effects.
Summer Career Exploration Program
Features of the Intervention
SCEP was created in 1983 to help low-income youth develop the skills necessary to have meaningful careers after high school graduation. Twenty-eight agencies sponsored SCEP at the time of this evaluation, in 1999: 25 in Philadelphia, Pennsylvania; one in Delaware County, Pennsylvania; and two in Camden, New Jersey. To be eligible for participation, applicants must have: (1) had a family income less than or equal to 150 percent of the federal poverty level; (2) been enrolled in school and completed the 10th grade, or graduated the previous year; and (3) been a first-time applicant to the program.
Participants received preemployment training and were then placed in private-sector jobs identified by SCEP, representing a variety of industries. During the summer, they worked 25 hours a week for at least six weeks, and received the federal minimum wage at the time, $5.15 per hour. They could participate for up to three summers. Participants were mentored by college students of similar background, gender, and race. These mentors served as role models and provided academic, career, and personal advice. Mentors were required to conduct biweekly workplace visits to help facilitate positive work experiences for participants.
Features of the Study
The design was a randomized controlled trial of SCEP conducted at 28 sponsoring agencies during the summer of 1999. To estimate the effectiveness of SCEP, the authors used data on about 1,500 youth to compare the outcomes of those randomly selected to receive an offer to participate in SCEP (the treatment group) with outcomes of those not permitted to participate in SCEP until the following summer (the control group). The authors examined the effects of SCEP on employment and educational outcomes using data collected from two surveys conducted in (1) fall 1999, immediately following summer employment; and (2) summer 2000, approximately one year after random assignment. The study also examined outcomes related to likeliness to attend college and attitudes toward work.
- The analysis found two positive, statistically significant impacts on employment outcomes of SCEP youth: (1) a larger percentage of youth offered admission to SCEP were employed during summer 1999 than control group youth (92 percent versus 62 percent); and (2) youth in SCEP earned more during summer 1999 than did control group youth ($695 versus $447).
- However, among youth who were employed during the summer, SCEP youth earned slightly less per hour than control group youth ($5.30 versus $5.69); this difference was statistically significant.
- During the following school year, SCEP youth were much less likely to work for their summer employer than control group youth (32 percent versus 47 percent). However, a larger percentage of SCEP youth had enrolled in a college preparatory curriculum (12 percent versus 8 percent).
- No statistically significant differences between SCEP and control youth were found for the mean hours worked (among those youth who were employed) over the summer, the percentage of participants who worked during the 1999–2000 school year, or the hourly wage or total earnings over the school year.
Considerations for Interpreting the Findings
The study was a well-implemented randomized controlled trial for the outcomes measured at the end of the summer and for measures of student enrollment in college preparatory curricula during the following school year. However, most of the outcomes measured at the one-year follow-up suffered from high attrition. The authors included sufficient control variables, including baseline measures of the outcomes and other relevant youth characteristics, to give us confidence that these estimated effects are attributable at least in part to SCEP, although other factors might also have played a role. However, despite this approach, the results presented in the report are based on unadjusted means. Thus, the magnitudes should be interpreted with some caution.
Causal Evidence Rating
The quality of causal evidence presented in this study is high for the outcomes measured at the end of summer and for enrollment in a college preparatory curriculum because it is based on a well-conducted, randomized controlled trial. This means we are confident that the effects estimated on these outcomes are attributable to SCEP. The quality of causal evidence for the remaining outcomes is moderate, which means that other factors besides SCEP might have contributed to the estimated effects.