Absence of conflict of interest.
Citation
Highlights
- The study’s objective was to examine the implementation of Cleveland’s Achieve program which offered services such as office hours and group sessions to increase retention among low wage workers at long-term nursing care facilities.
- The study authors conducted an implementation evaluation using baseline data, employer records, administrative records, program participation and implementation data, the results of the ERA 12-Month Survey, and data from two waves of field research which included time studies and staff interviews.
- The study found that the program was generally well implemented, despite difficulty integrating the program into the workplace environment.
- The authors noted that employee participation in Achieve programs was less intensive than they had intended for, especially sustained engagement.
- The embedded impact study was reviewed by CLEAR in April 2016.
Intervention Examined
Cleveland Achieve Model (“Achieve”)
Features of the Intervention
- Type of organization: Social Services Organization
- Location/setting: Multi-site in Ohio
- Population served and scale: Low Wage Workers; 381 Participants
- Industry focus: Health Services
- Intervention activities: Office hours and lunch and learn group sessions
- Organizational Partnerships: Not applicable
- Cost: Annual Total Cost: $502,195; Annual Cost per Employer: $58,747
- Fidelity: Not included
The Achieve program was developed in 2000 with the belief that employers should be a partner in retention services for low wage workers. At the time of evaluation, Achieve was in what the authors considered the steady state stage of implementation. Achieve was the city of Cleveland's Emergency Rental Assistance (ERA) program which offered services to increase retention among low-wage workers at long-term nursing care facilities. The program was run by Towards Employment which provided the program services including office hours, lunch and learn sessions, and supervisory trainings. The program was funded by the Administration for Children and Families, U.S. Department of Health and Human Services with supplemental funding from the U.S. Department of Labor. The target population was recently-hired low-wage workers and was conducted in multiples locations at 22 different firms in the larger metropolitan Cleveland area.
Features of the Study
The companion impact study was a randomized controlled trial. The authors also conducted a cost analysis, staff time analysis, impact analysis, and implementation analysis. The implementation study used data from a series of site visits, program participation data and the “time study” results. During the study, the Achieve program had a relatively simple staffing structure, with 3 or 4 full-time advisors assigned to work with about 3 firms, 1 supervisor, 1 manager, and a part-time assistant. Achieve staff all had previous experience in social work and previously worked in supportive services and or crisis intervention programs. Achieve advisors were trained by their supervisors as well as the authors’ operations staff and Cygnet Associates, a consulting firm hired by the authors. Despite the organizational design being fairly simple, there was very high rates of attrition among the program advisors which caused minor delivery disruption as well as lower staff morale and team cohesiveness. The authors interviewed a number of staff members at 11 of the nursing homes which were picked to reflect a mix of settings, advisors, and time of study entry. In total, 140 staff were interviewed which included administrators, day to day managers, frontline supervisors, and entry level employees.
Findings
Intervention activities/services
- The study found that Achieve staff were able to implement the three components of the program at all of the locations.
- The study found that in Round 1, Achieve locations offered “lunch and learns” which were a half hour session that included a free lunch, a presentation about topics of employee interest and facilitated group discussion. During Round 2, Achieve locations replaced the “lunch and learns” with group sessions that were held once a week on a set day and open to all employees. The group sessions and “lunch and learns” were attended more frequently than office hours, and participants appeared to benefit from the information discussed during these sessions.
- The study found that in round 1 Achieve locations offered lunch and learns which were half hour session that included free lunch, a presentation about topics of employee interest and facilitated group discussion. During Round 2 Achieve locations replaced the lunch and learns with group sessions which were held once a week on a set day and still open to all employees. The group sessions and lunch and learns were attended more frequently than office hours and participants appeared to benefit from the information discussed during these sessions.
- The study found that Achieve program advisors noticed that advancement-focused conversations rarely occurred as participants either left the firm without notifying the advisor or had personal issues which took priority.
- The study found that across the 22 firms that were assigned to the intervention 20% of all nonresearched sample employees participated in at least one Achieve activity, and of this 20%, 82% attended a lunch and learn, 68% were referred to supportive services by a case manager and 61% had an individual session with an advisor.
Implementation challenges and solutions
- The study found that on average each Achieve staff member was assigned to provide services to 3 different firms, which meant that they were likely to spend a considerable amount of time traveling from site to site as most of the firms were located in the Cleveland suburbs.
- The study found that Towards Employment had very high levels of staff attrition, particularly of Achieve program advisors which caused minor disruptions of service delivery as well as staff morale and team cohesiveness.
- The authors noted that it was difficult to find firms that met the main eligibility criterion of having 15 recently hired (within the past 4-6 months) low-wage employees. In total, 429 firms were contacted with 52 agreeing to participate and 44 found eligible.
- The study found that due to the nature of nursing homes, it was often hard for employees to access these services while working since nursing homes require staff to be on call for patient care due to resident coverage requirements. In addition, some nursing homes required employees to stay at work longer than usual if they participated in Achieve since participation was not allowed "on the clock" which may have hindered participation.
- The study found that over time, in order to increase contacts with the participants, Achieve staff members began to seek out employees in the hallways of the nursing homes to ask how they were doing and encourage them to stop by for office hours. After instituting this practice as well as a two contact per month benchmark in round 2, advisors spent more time in contact with employees.
Cost/ROI
- The authors estimated that the Achieve program cost in total $502,195 (in 2007 dollars) with an estimated total cost per employer of $58,747 per year. The cost to employer does not necessarily represent the cost that an employer would pay for Achieve, dependent on who provides the program. The majority of the program cost is associated with staff salaries and benefits (83% of the total costs were associated with staff and program operations).
Considerations for Interpreting the Findings
The authors note that the intervention was intended to have sustained engagement over time, however, many participants only participated minimally.