There is no conflict of interest.
Citation
Siwach, G., Smith, D. M., Milliano, M. D., Lin, D., Lee, D. H., & Yin, M. (2021). Maine Transition Work-Based Learning evaluation: Final evaluation report. American Institutes for Research.
Highlights
- The study’s objective was to examine the impact of the Jobs for Maine’s Graduates (JMG) model on employment and earnings. The authors investigated similar research questions for another contrast, the profile can be found here.
- Progressive Employment model: Maine Transition Work-Based Learning evaluation: Final evaluation report (Siwach et al., 2021) | CLEAR
- The study used a nonexperimental design to compare the outcomes of students who received different combinations of JMG and vocational rehabilitation services to a matched comparison group of non-recipients. Using administrative data and statistical models, the authors compared outcomes of the treatment and comparison groups.
- The study found that students who received vocational services and no JMG services and those who received JMG and vocational rehabilitation services had a significantly higher rate of employment and higher earnings over three years compared to students who did not receive JMG and vocational rehabilitation services.
- This study receives a moderate evidence rating. This means we are somewhat confident that the estimated effects are attributable to Jobs for Maine’s Graduates (JMG), but other factors may also have contributed.
Features of the Intervention
Jobs for Maine’s Graduates (JMG) was created in 1993 by the Maine Division of Vocational Rehabilitation and is one of the two work-based learning models within the Maine Transition Work-Based Learning Model Demonstration Project. JMG partnered with public high schools in Maine to help improve the graduation rate of students and prepare them for college or careers. JMG specialists worked with students to: develop a personalized career development plan; create job goals for a student’s preferred occupation; set and prioritize goals to improve self-determination; implement strategies to address academic, work, and emotional needs; and link students to employers and businesses for work-based learning and workshops. To be eligible for JMG, students needed to be in their final two years of high school and have an identified barrier that impacted school or work.
Features of the Study
The study used a nonexperimental design to examine the impact of the JMG model on employment and earnings. JMG was implemented in 54 high schools across Maine. The treatment group consisted of three groups of students with disabilities who applied for vocational rehabilitation (VR) from 2005 to 2016. Group 1 included 204 students who received JMG services and no VR services; group 2 included 297 students who received both JMG and VR services; and group 3 included 4,043 students who received VR services with no JMG services. The comparison group consisted of 4,076 students who did not receive VR services or JMG services. The authors matched students in the treatment group to students in the comparison group on student demographic, education, and socioeconomic characteristics and school characteristics. Data sources included JMG administrative data, the U.S. Department of Education’s Rehabilitation Services Administration (RSA-911) dataset, and the Maine Department of Labor Unemployment Insurance dataset. The authors used statistical models to estimate the differences in annual employment and earnings outcomes between the groups over three years.
Findings
Employment
- The study found that students who received JMG services (no VR services) had a significantly higher rate of employment in year three than students in the comparison group.
- However, the study found that students who received VR services (no JMG services) had a significantly higher rate of employment in years one through three compared to students in the comparison group.
- The study also found that students who received both JMG and VR services had a significantly higher rate of annual employment in years one through three compared to students in the comparison group.
Earnings and Wages
- The study found that students who received VR services (no JMG services) had significantly higher earnings in years two and three than students in the comparison group.
- The study also found that students who received both JMG and vocational rehabilitation services had significantly higher earnings in years two and three compared to students in the comparison group.
- No significant differences in earnings were found between students who received JMG services (no VR services) and students in the comparison group.
Considerations for Interpreting the Findings
The study authors could not access reliable employment data for high school students because the Maine Department of Education does not record social security numbers. To resolve this, they linked JMG records with vocational rehabilitation records to analyze employment trends. Also, the study reports a less stringent statistical significance level, considering p-values of less than 0.10 to be significant, though it is standard practice to consider statistical significance if the p-value is less than 0.05. Only results that demonstrate a p-value of less than 0.05 are considered statistically significant in this profile.
Causal Evidence Rating
The quality of casual evidence presented in this report is moderate because the study was based on a well-implemented nonexperimental design. This means we are somewhat confident that the estimated effects are attributable to JMG, but other factors might also have contributed.