Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the implementation of the Florida Information Technology Careers for Rural Areas (XCEL-IT) program which sought to expand the number of specialized Information Technology (IT) offerings as well as career linkages for non-traditional and rural students.
- The study authors conducted an implementation evaluation using a review of program documents, analysis of data metrics, focus groups and interviews, annual site visits, monthly calls with program stakeholders, and an audit of the participant-level database. This was designed to understand how XCEL-IT was implemented, including lessons learned, best practices, and project outcomes. The authors evaluated how the program was developed and operated, and what factors could be changed to improve program performance through descriptive and qualitative analysis.
- The study found that while the timeframe for implementing various aspects of XCEL-IT differed across colleges, eventually all colleges satisfied the major requirements outlined within the Statement of Work (SOW). However, colleges did experience variance with implementing all components of the XCEL-IT program.
- The study provided detailed insight into each activity undertaken by XCEL-IT and the differences between the intended and actual implementation. However, it is unclear how contextual factors across the colleges influenced their varied results.
- The embedded impact study was reviewed by CLEAR in May 2020.
Intervention Examined
The Florida Information Technology Careers for Rural Areas (XCEL-IT)
Features of the Intervention
- Type of organization: Community colleges
- Location/setting: Multi-site in Florida
- Population served and scale: Adults, Non-traditional students; TAA-eligible; Veterans; Unemployed; 2,779 participants
- Industry focus: Professional, Scientific, and Technical Services
- Intervention activities: Accelerated learning; Career pathways; Developmental education; Student support services; Technology; Work-based learning
- Organizational partnerships: Employers
- Cost: Not included
- Fidelity: Not included
The Florida Information Technology Careers for Rural Areas (XCEL-IT) program was funded through the U.S. Department of Labor's (DOL) Trade Adjustment Assistance Community College and Career Training (TAACCCT) and ran for four years, from October 2013 to September 2017. The award was to a single-state consortium led by the College of Central Florida and included seven colleges. It was designed to help rural community colleges increase specialized IT offerings related to manufacturing, logistics, supply chain management, and cybersecurity. The program’s target population included post-traditional students, such as those who were older than 24 years of age, unemployed or underemployed, and who would be pursuing education while balancing work and life responsibilities. Features of the XCEL-IT program included developing refined career pathways from non-credit courses to degrees in Information Technology (IT), cybersecurity, logistics, manufacturing, and entrepreneurship; adding a developmental education course; creating an Employment Results Scorecard that documented student employment outcomes; and refining outreach techniques to better reach TAA-eligible and rural workers. Participants were eligible if they were pursing one of the above-mentioned degrees and attended one of the seven rural community colleges part of the XCEL-IT consortium.
The study included a logic model for the XCEL-IT program. Inputs included project staff, college staff, participants, student success coaches, college courses and equipment, industry partners, employers, regional Workforce Investment Boards (WIBs), local advisory committees, and college taskforces. Outputs were framed around the types of new and enhanced courses and materials, the types of outreach provided to participants, the types of participants who enrolled in the program, and the types of employer partnerships built. Expected short-term outcomes included new recruiting techniques, development of labs and other classroom resources, perceived effectiveness of new courses, new opportunities for acquiring skills and knowledge, and increased support services. Medium and long-term intended outcomes included successful recruitment and retention of participants, active engagement in program, success finding employment in in-demand fields, new patterns of collaboration across Florida colleges, refined pathways for meeting employer needs, and increased employee retention and earnings.
Features of the Study
The study used a qualitative design to examine the implementation of the XCEL-IT program. Data sources included regular progress reports (via an implementation evaluation template) and monthly calls with the XCEL-IT director, document review (meeting minutes, referrals, sustainability plans, curriculum), interviews and focus groups with key stakeholders (college coordinators, administrators, faculty, and students), program records data, participant administrative data, and twice annual site visits. The study does not mention how authors selected participants for interviews and focus groups, nor does it contain the numbers of individuals who participated. The authors used content analysis for interviews and focus group data, where two analysts read each transcript and coded them to identify major themes, which were then checked against one another for validity.
Study Sites
- College of Central Florida in Ocala, Florida
- Florida State College in Cocoa, Florida
- Florida Southwestern State College in Fort Myers, Florida
- North Florida Community College in Madison, Florida
- Palm Beach State College in Palm Beach County, Florida
- South Florida State College in Avon Park, Florida
- St. John’s River State College in Palatka, Florida
Findings
Intervention activities/services
- The study found a variety of new courses were provided to students, including non-credit courses on soft skills, boot camps, and a suite of IT courses. In addition, many students enrolled in internships and apprenticeships as part of an increased focus on work-based learning opportunities.
- The study found that it was important to prioritize staff hiring, training, and retention since the XCEL-IT program was slowed down due to staff turnover. Additionally, the evaluators learned the importance of training staff on accurate data collection (including data entry and quality control processes); insufficient and missing data made it difficult to evaluate certain components of the intervention.
- The study found all colleges provided new opportunities for students to acquire basic skills, have more opportunities for work-based learning, receive individualized guidance, and have an expanded suite of IT-related offerings. However, it was difficult to determine how beneficial any of these expanded opportunities and services were for students, as these findings were not detailed thoroughly.
Implementation challenges and solutions
- The study found that all seven colleges implemented the major aspects of the XCEL-IT program as defined within the statement of work. However, there was variation between colleges regarding how these programs were implemented, as well as the timeframes in which they occurred, which often took longer than initially expected.
- The study found that while schools were able to provide outreach and follow-up to nontraditional students and adult learners, it was often challenging to reach intended students.
- The study found while colleges were successful in hiring proactive advisors to provide individualized services to students, a lack of reliable data collection made it hard for evaluators to determine the results of these advising services on student success.
- The study found partner involvement with XCEL-IT was mixed, but colleges reported, in general, that having an individualized contact with an employer was useful. Having partners involved with planning within the college was also useful for keeping them engaged and tailoring programs to benefit their organization.
Considerations for Interpreting the Findings
The evaluation provided some insight for marketing new programs for students, though this insight does not seem specifically tailored to the target population of rural and nontraditional students. While the authors documented that most program activities were implemented across the seven colleges, they did provide information on why certain interventions varied across contexts. More generally, the authors did not mention how specific contextual factors may have influenced XCEL-IT’s implementation, such as variance across sites and across student demographics and profiles. Further, the authors did not provide explicit information on the sample used for the implementation study (such as the number of participants, how they were chosen, what colleges they attended, or any demographic information).