Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the implementation of Heroes for Hire (H4H), an intervention that aimed to serve veterans and trade-affected or dislocated workers through skills upgrading and education in West Virginia. The intervention was implemented by three community and technical colleges and included student support services provided by faculty advisors and Veterans Coordinators.
- The authors conducted an implementation evaluation using data collected through site visits, stakeholder interviews, and document reviews. Evaluators examined four areas of implementation, including curriculum selection, creation, and use; program design, improvement, and delivery methods; assessment tools and processes; and partner contributions.
- The study found that of the 24 credentials intended for expansion or creation, two were fully implemented according to design, eight were implemented with high fidelity, six were partially implemented, and seven were not implemented at all. Intake processes were straightforward, and each site attempted to appropriately balance personnel, resources, and professional development for grant staff.
- The implementation study was somewhat comprehensive in its design, data collection, and analyses. The findings aligned with the research questions and were supported by the data. However, the authors did not discuss recruitment for participation in the implementation evaluation and the student perspective was not included in the implementation evaluation.
- The embedded impact study was reviewed by CLEAR in August 2020.
Intervention Examined
The Heroes for Hire (H4H) Program
Features of the Intervention
- Type of organization: Community colleges
- Location/setting: Multi-site in West Virginia
- Population served and scale: Adults; Veteran or military; Dislocated workers; 972 participants
- Industry focus: Manufacturing; Professional, scientific, and technical services; Health Care and social assistance
- Intervention activities: Career pathways; Student support services; Technology; Work-based learning
- Organizational partnerships: Veterans Administration; Workforce agencies; Employers; Universities; Federal Aviation Administration
- Cost: Not included
- Fidelity: Included
The Heroes for Hire (H4H) program was funded through a U.S. Department of Labor (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant. The intervention was implemented from 2014 to 2018 at three technical colleges in West Virginia: Mountwest Community and Technical College, Blue Ridge Community and Technical College, and Southern West Virginia Community and Technical College. H4H targeted veterans and trade-impacted (dislocated) workers in West Virginia and aimed to provide training that highlighted their existing credentials and provided services to enhance their skills and educational advancement. Veterans Coordinators, advisors, and instructors provided student support, such as tutoring, mentoring, and professional networking. Key H4H partners included the Veterans Administration, Workforce West Virginia, and employers. H4H colleges also partnered with Marshall University, Veterans Community Organizations, the Community College of the Air Force and the Federal Aviation Administration.
The intervention’s logic model included inputs (community colleges, program staff, USDOL, program participants, program stakeholders, businesses, and industry); outputs (individualized student support services, provision of college credit for previous work or military experience, career pathways, technology-based learning environments, and faculty professional development); outputs (technology-enabled strategies, participants completing their program of study, attaining credentials, finding employment, and receiving increased wages); outcomes (increased quality of curricula and online instruction, participants enrolling in further education, credential attainment, employment rate, and increased starting wage) and impact (lower degree completion time, improved graduation rate, and improved employment wages and job placement). External factors included other activities and programs of study at the three colleges, employment conditions, and industry outlooks.
Features of the Study
The authors conducted an implementation evaluation that included a fidelity assessment. The authors collected data through site visits, interviews with H4H staff and administrators, and document reviews. The four areas measured were based on curriculum selection, creation and use; program design, improvement and delivery methods; assessment tools and processes; and partner contributions. Fidelity outcomes were measured in three categories: high, partial, and low implementation. Low implementation meant that the career pathway did not occur, medium meant it occurred, but not as intended, and high implementation meant that it occurred as it was intended.
To conduct the document review, the evaluation team created a matrix to track key findings and program developments summarized from H4H monthly and quarterly reports, memos, course syllabi, marketing materials, individual program notes, and meeting notes and agendas. The evaluation team reviewed and analyzed these documents to assess the extent to which the project was implemented in relation to the original project plan. Analysis methods for interviews and site visits is not discussed.
Findings
Intervention activities/services
- The study found that following the grant, all three colleges in the consortium have the capacity to implement online courses that meet required standards through a hybrid or fully online instructional model.
- The study found that career pathways that leveraged a blended learning format and effectively used partnerships encouraged some students to continue their education at other institutions.
- The study found that Veterans Coordinators promoted access to education and training and assistance for veteran students. Veterans Coordinators became “one-stop shops” within the colleges and served as a connection to vital community resources and services.
- The study found that intake processes for students were straightforward and involved a career pathways advisor or a Veterans Coordinator, if applicable. However, career service activities were not formalized and upfront skills assessments that demonstrate specific workforce skills and aptitudes were not used.
- The study found that, while curricula and courses were implemented well, in some cases, labor market analysis was not adequately conducted in advance, and actual job opportunities in the more rural labor market did not justify costs of program sustainability.
Fidelity
- The study found that out of the 24 credentials intended for expansion or creation across the three technical colleges, two were fully implemented, eight were implemented with high fidelity, six were partially implemented, and seven were not implemented at all (one was identified as “not applicable” and not counted toward the final implementation rating count).
Implementation challenges and solutions
- The study found that overall enrollment was limited by delays in implementation and curriculum development.
- The study found that two of the colleges experienced high rates of staff turnover, and staff reported that neither had a formal onboarding process, leading to confusion.
- The study found that the colleges experienced challenges with a geographically limited employer base and lacked formalized engagement strategies. The consortium’s relationship with Workforce West Virginia could also be improved. However, Blue Ridge CTC was able to connect to a larger number of occupations and employers by broadening the Chemical Technology pathway to “Laboratory Technology.”
- The study found that the colleges were able to establish partnerships that provided participants with a broader system of supportive services, including with the Veterans Administration, Workforce West Virginia, Veterans Community Organizations, and employers.
Considerations for Interpreting the Findings
The implementation study was somewhat comprehensive in its design, data collection, and analyses. The findings aligned with the research questions and were supported by the data. The logic model provided the framework that guided the implementation analysis and served as a point of reference to assess the intervention’s impact. However, the authors did not discuss recruitment for participation in the implementation evaluation and the student perspective was not included in the implementation evaluation. Further, the authors did not address external validity or analytical methods related to interviews. Fidelity was assessed by the study authors and not by the CLEAR team.