Skip to main content
UNITED STATES DEPARTMENT OF LABOR

Scaling out virtual interview training for transition-age youth: A quasi-experimental hybrid effectiveness-implementation study (Smith et al., 2022)

Review Guidelines

Causal Evidence Guidelines

Absence of conflict of interest.

Citation

Smith, M. J., Sherwood, K., Ross, B., Oulvey, E. A., Monahan, J. A., Sipovic, J. E., Atkins, M. S., Danielson, E. C., Jordan, N., & Smith, J. D. (2022). Scaling out virtual interview training for transition-age youth: A quasi-experimental hybrid effectiveness-implementation study. Career Development and Transition for Exceptional Individuals, 45(4), 213-227. https://doi.org/10.1177/21651434221081273

Highlights

  • The study's objective was to examine the impact of Virtual Interview Training for Transition Age Youth (VIT-TAY) on employment outcomes for transition-age youth with disabilities. 
  • The study used a nonexperimental design to compare employment outcomes for transition-age youth who used VIT-TAY with a comparison group of transition-age youth who used Virtual Reality Job Interview Training (VR-JIT). The primary data sources for employment outcomes were 3- and 6-month follow-up surveys completed by participants' teachers. The authors used statistical models to compare the employment outcomes of VIT-TAY and VR-JIT group members.  
  • The study found a significant relationship between VIT-TAY and the likelihood of finding employment for transition-age youth who were not initially seeking employment.  
  • This study receives a low evidence rating. This means we are not confident that the estimated effects are attributable to VIT-TAY; other factors are likely to have contributed.

Intervention Examined

Virtual Interview Training for Transition Age Youth (VIT-TAY)

Features of the Intervention

Virtual learning environments (VLE) are used to help individuals with disabilities learn to navigate job interviews. One virtual learning intervention, the Virtual Reality Job Interview Training (VR-JIT), was designed to improve interview performance by enhancing interview skills through a virtual reality role playing simulation. VR-JIT was designed for adults with serious mental illnesses. 

The Virtual Interview Training for Transition Age Youth (VIT-TAY) is an adaptation of VR-JIT. VIT-TAY is designed to meet the needs of transition-age youth receiving pre-employment transition services (pre-ETS). VIT-TAY is delivered via the internet and scalable to individual needs to maximize independent learning. The virtual interviews have difficulty levels that progress from easy to hard. Depending on the difficulty level, each interview targets three to four different interview skills. 

Features of the Study

The study used a nonexperimental design to compare employment outcomes for transition-age youth who used the VIT-TAY intervention to prepare for job interviews with a comparison group of transition-age youth who used the VR-JIT intervention to prepare for job interviews. Data for the VIT-TAY intervention were collected from youth who were recruited from 32 schools with pre-ETS programs across three states (Illinois, Michigan, and Florida). Data for the comparison group came from a previously published study that examined the use of the VR-JIT intervention across 15 schools in two states (Illinois and Michigan). Teachers who worked with participants receiving the VIT-TAY intervention were trained to implement VIT-TAY in a way that closely aligned with VR-JIT implementation procedures to ensure comparability between datasets. The teachers encouraged students to complete 15 45- to 60-minute virtual interviews over 4 to 6 weeks using the VIT-TAY tool.  

Study participants were transition-age youth (ages 15 to 26) with disabilities who were receiving pre-ETS. The sample included 635 transition-age youth with disabilities, 356 participants who received the VIT-TAY intervention and 279 participants who received VR-JIT. The VIT-TAY group was predominantly male (65%), over half were White (51%), with an average age of 18.7 years. The VR-JIT group was primarily male (64%), was predominantly White (68%), with an average age of 18.6 years. The primary data sources for employment outcomes were 3- and 6-month follow-up self-report surveys completed by participants' teachers. The authors used statistical models to compare the employment outcomes of VIT-TAY and VR-JIT group members.  

Findings

Employment 

  • The study did not find a significant relationship between VIT-TAY and employment outcomes six months after the intervention. However, for youth who were not initially seeking employment, the study found a significant relationship between VIT-TAY and the likelihood of finding employment. 

Considerations for Interpreting the Findings

The authors noted that the baseline composition of the treatment and comparison groups significantly varied by student race. However, the authors did not control for the variable in the analysis. This preexisting difference between the groups—and not the VIT-TAY intervention —could explain the observed differences in outcomes. Therefore, the study is not eligible for a moderate causal evidence rating, the highest rating available for nonexperimental designs. 

Causal Evidence Rating

The quality of causal evidence presented in this is low, because the authors did not ensure that the groups being compared were similar before the intervention. This means we are not confident that the estimated effects are attributable to the VIT-TAY intervention; other factors are likely to have contributed. 

Reviewed by CLEAR

March 2024

Topic Area