Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to document how grantee activities were completed compared to the original work plan and the process, strengths, and challenges of program implementation from design to completion of the Mechatronics Re-envisioned (MRE) program, led by Central Piedmont Community College (CPCC) in Charlotte, North Carolina.
- The study authors conducted an implementation evaluation assessing whether program activities were completed as designed and the strengths and weaknesses of the MRE program using a case study design that relied on data from interviews, focus groups, surveys, observation, and document review.
- The study found that that the MRE program completed all five planned activities in the work plan including: 1) curriculum modernization and assessment development; 2) competency validation and credential alignment; 3) provision of personalized student support services; 4) innovative delivery of training; and 5) institutional and statewide policy changes to promote sustainability.
- The design of the implementation evaluation and data collection methods were reasonable to assess the research questions. The findings were aligned to the main research questions, but did not always address the sub-research questions. There was limited discussion of the quality control processes used during the evaluation and the discussion of the evaluation limitations only covered the impact evaluation, not the implementation evaluation.
- The embedded impact study was reviewed by CLEAR in May 2020.
Intervention Examined
The Mechatronics Re-envisioned (MRE) Program
Features of the Intervention
- Type of organization: Community College
- Location/setting: Charlotte, North Carolina
- Population served and scale: TAA-eligible individuals; 332 participants
- Industry focus: Manufacturing
- Intervention activities: Accelerated learning; Career pathways; Student support services; Technology; Work-based learning
- Organizational partnerships: Employers; Local workforce development agency; Digitalization vendor
- Cost: Not included
- Fidelity: Not included
Funded through a Round 4 Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant from the U.S. Department of Labor (DOL) in 2014, the Mechatronics Re-envisioned (MRE) program was led by the Central Piedmont Community College in Charlotte, NC. The primary objective was to develop a competency-based Mechatronics accelerated Associate of Applied Science (AAS) degree incorporating online learning and a Fast Track Mechatronics certification program to deliver highly trained technicians for the local energy and advanced manufacturing industries. Targeting TAA-eligible workers, the program components included: 1) reviewing and modernizing curricula and assessment development; 2) validating competencies and aligning credentials; 3) providing personalized student supports including a success coach and an apprenticeship coordinator; 4) delivering innovative technical training; and 5) institutional and statewide change to promote program sustainability. The MRE program served 332 participants over the four-year grant period.
As described in the program logic model, the planned outputs were students enrolled, courses enhanced/digitized, courses offered in a new delivery model, and students retained. The desired outcomes were completed credit hours, credentials obtained, program completion, entry into employment for non-incumbent workers, wage increases for incumbent workers, continued employment after three months, increased average earnings, and participants enrolling in further education.
Features of the Study
The implementation study used a case study design to assess whether program activities were completed as designed and the strengths and weaknesses of the MRE program implementation. Data was sourced from interviews, focus groups with faculty and staff, student surveys, meeting observation, and document review (e.g., quarterly reports, literature, websites, curricula). The single study site was Central Piedmont Community College in Charlotte, NC. Researchers conducted interviews, focus groups and meeting observation of program faculty and staff (including the project director, project coordinator, instructors, college administrators, success coach, and apprenticeship coordinator). There was also a student survey of program completers. At some program activities/events, two members of the evaluation team were able to attend and compare observations about staff interactions and draw connections to the data collected through other methods. The authors describe using an open concept with interviews and focus groups to document themes, not constrained strictly by the research questions. The evaluation team then reviewed notes to develop themes and later in the evaluation, once rapport was established, they shifted to focusing on the research questions. Findings from the organically derived theme analysis and findings related to the research questions were merged. Additional information about study participants and response rates were not provided.
Findings
Intervention activities/services
- The study found that that the MRE program completed all five planned activities in the work plan, including: 1) curriculum modernization and assessment development; 2) competency validation and credential alignment; 3) provision of personalized student support services; 4) innovative delivery of training; and 5) institutional and statewide policy changes to promote sustainability.
- The study found that despite delays due to contract issues, all planned courses were digitized. The delivery method for all MRE courses was a "web-enhanced flipped classroom layout."
- The study found that a Fast Track program was piloted and directly linked to PMMI (The Association for Packaging and Processing Technologies) credentials. The Fast Track program was viewed favorably by employees who completed the program.
- The study found that CPCC led facilitated discussions with industry partners to define needed competencies and MRE staff worked with local workforce boards to market and promote the program. Also, the advisory board for the project remained active throughout the grant period.
- The study found that staff members received professional development and found value in participating in the curricula digitization process.
Implementation challenges and solutions
- The study found that instructors perceived that the pace of the Fast Track program to be too quick for inexperienced students and recommended restricting access to more advanced students.
- The study identified several lessons learned during the implementation, including recommendations for more preparation of the faculty working on digitization; better engagement with the digitization vendor early to prevent delays; upfront definition of roles and process for digitization; more time for faculty to transition to a hybrid online/in person model, more classroom time for career option discussions, recommendations that support services evolve throughout the semester – with early intervention focused on course selection/academic guidance, mid-support on tutoring and academic support and later support on career guidance and that being engaged with industry in competency identification primed them to be more receptive to the program.
Considerations for Interpreting the Findings
The design of the implementation evaluation and data collection methods were reasonable to assess the research questions. The authors reported that all planned activities were completed, but did not provide an overall measure of fidelity. The findings were aligned to the main research questions, but did not always address the sub-research questions. There was limited discussion of the quality control processes used during the implementation evaluation. Further, the discussion of the evaluation limitations only covered the impact evaluation and not the implementation evaluation.