Absence of conflict of interest. This study was conducted by staff from Mathematica Policy Research. The review of this study was conducted by ICF Incorporated.
Citation
Highlights
- The study’s objective was to examine the implementation of Leveraging, Integrating, Networking, Coordinating Supplies (LINCS), a program offered through a consortium of nine state and community colleges, in partnership with the Council of Supply Chain Management Professionals (CSCMP), universities outside the consortium, the National Urban League, Workforce Development Boards, and American Job Centers to train early career individuals in supply chain management.
- The study authors conducted an implementation evaluation using data gathered during telephone interviews with college staff and faculty, students, employers, the National Urban League, Workforce Development Boards, and the Council of Supply Chain Management Professionals; document review; and administrative student data.
- The study found that the consortium colleges developed and implemented the program as planned and expanded the program by offering it to universities and colleges outside the consortium and through an online learning management system developed during the grant.
- The implementation study of LINCS was comprehensive in its design, which incorporated both qualitative and quantitative data. All colleges, and appropriate staff at those colleges, were included in the study, as were partner organizations and students. Evaluators used techniques and strategies to ensure the accuracy of their findings and interpretation.
- The companion impact study was reviewed by CLEAR in May 2020.
Intervention Examined
The Leveraging, Integrating, Networking, and Coordinating Supplies (LINCS) program
Features of the Intervention
- Type of organization: State and Community Colleges
- Location/setting: Multi-site in California, Florida, Illinois, New Jersey, Ohio, Texas
- Population served and scale: TAA-eligible, veterans; 3,295 participants
- Industry focus: Transportation and Warehousing
- Intervention activities: Accelerated learning; Career pathways; Student support services; Technology; Work-based learning
- Organizational partnerships: Council of Supply Chain Management Professionals (CSCMP); the National Urban League; universities outside of the consortium; Workforce Development Boards; American Job Centers; Employers
- Cost: Not Included
- Fidelity: Not Included
The LINCS program began in 2014 and was funded by a $24.5M Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant from the U.S. Department of Labor (DOL). Prior to the consortium's grant application to DOL, the Council of Supply Chain Management Professionals (CSCMP) identified eight fundamental skill/knowledge areas necessary for early- to mid-level Supply Chain Management (SCM) positions; these eight skills became the LINCS certification areas. Employer surveys conducted by the consortium prior to the grant application validated these essential knowledge/skill areas.
The LINCS program consisted of exams created to assess the skills needed for entry- to mid-level careers in supply chain management, curricula developed to teach those skills, delivery of the course materials, student supportive services (e.g., assessments, case management, resume support), and partner contributions to provide training, education, and certificates. The program was offered by both two- and four-year colleges in partnership with the CSCMP, university partners, the National Urban League, Workforce Development Boards (WDBs), and American Job Centers (AJCs). LINCS targeted individuals interested in a SCM position; additionally, WDBs and AJCs recruited TAA-eligible individuals who had lost their jobs due to international trade. The DOL grant was awarded September 2013 and concluded July 31, 2016; 3,295 students were served during this time.
The logic model shows the context (e.g., labor market conditions, employer needs, other SCM programs offered, community organizations), inputs (e.g., industry and policy maker relationships, curriculum and teaching experience, staff, student services, existing partnerships), activities (e.g., awareness, promotion, industry needs assessment, hiring, training, student recruitment and enrollment, student services, course development and instruction), outputs (e.g., agreements and contracts, certification exams, student program completion), and student outcomes (e.g., increased education, training, employment, and earnings).
Features of the Study
The mixed-method study assessed program implementation focusing on: 1) how the LINCS program was developed through collaboration; (2) how colleges and partner agencies implemented the certification tracks; 3) student participation and perceptions; and 4) employer perceptions. Researchers used data from program documents, participation on consortium phone meetings, interviews conducted via telephone, and student administrative data from each college and the National Program Office. Interviews were conducted in four rounds focusing, respectively, on program development, program implementation, program participation, and partnerships. A total of 44 college or National Program Office staff, 43 students, 16 partners, three individuals at the National Urban League, and one individual at the CSCMP participated in the interviews. All data were analyzed using qualitative analysis software, with several steps taken to ensure data quality.
Study Sites
- Broward College in Fort Lauderdale, Florida
- Columbus State Community College in Columbus, Ohio
- Essex County College in Newark, New Jersey
- Florida State College in Jacksonville, Florida
- Harper College in Palatine, Illinois
- Long Beach City College in Long Beach, California
- San Jacinto Community College in Pasadena, Texas
- St. Petersburg College in St. Petersburg, Florida
- Union County College in Cranford, New Jersey
Findings
Intervention activities/services
- The study found that within 23 months of program start, the CSCMP successfully created eight certification tracks and eight certification exams aligned to eight key courses, a Learning Management System was established, other learning tools were active, and marketing materials were in place.
- The study found that the consortium colleges implemented courses differently to allow students to take one or more courses concurrently or sequentially; additionally, colleges offered many non-traditional modes to take the courses including in-person workshops, self-paced courses, Saturday courses, and webcasts of in-person courses.
- The study found that many colleges hired additional staff to provide student supports primarily focused on career services such as mock interviews, resumes, and job searches. Colleges leveraged their relationships with partners, particularly the National Urban League, to provide services they could not--including substance abuse, childcare, housing, and criminal record expungement.
- The study found that when the study concluded in July 31, 2016, 3,295 students had completed at least one certification track course; this exceeded the target of 3,000 students by the grant end of March 2017.
- The study found that the structure of the National Program Office contributed to the program's success. Specifically, experienced staff were hired to fill director-level positions in four key areas: certification and delivery, workplace competencies, content and curriculum integration, and adaptive learning technologies.
Implementation challenges and solutions
- The study found that the challenge of National Program Office staff turnover was successfully overcome by hiring from within.
- The study found that while interviews supported the view that employers were pleased with the certification tracks, none of the employers interviewed required them for a job.
- The study found that a slow procurement process delayed the launch of the Learning Management System (LMS) and the WorkKeys assessment, which was being used to assess the readiness of students for the course materials.
Considerations for Interpreting the Findings
The implementation study of the LINCS program was comprehensive in its design, which incorporated both qualitative and quantitative data. All colleges, and appropriate staff at those colleges, were included in the study, as were partner organizations and students. Extensive detail was provided on who was interviewed, the timing of the interviews, and the topics covered. The authors noted three limitations of the design. One, the individuals interviewed may not hold the same opinions or perspectives as program participants that were not interviewed. Two, data collection ended on July 31, 2016 so the authors were unable to capture grant activities for the full period of performance (through March 2017). Third, the qualitative data were subject to recall bias. The data analysis procedures were described with sufficient detail and were appropriate to address the research questions. Evaluators used several techniques and strategies to ensure the accuracy of their findings and interpretation.