WEBVTT Kind: captions Language: en 00:00:00.840 --> 00:00:06.540 Welcome. The Chief Evaluation Office is celebrating  10 years of the Clearinghouse for Labor Evaluation   00:00:06.540 --> 00:00:11.520 and Research. We would like to welcome you to  today's webinar, "Strategies to Find Evidence   00:00:11.520 --> 00:00:16.920 that Addresses Equity in CLEAR." This webinar  is the third and the final presentation of   00:00:16.920 --> 00:00:22.980 CLEAR 10-year anniversary webinar series. All the  presentations will be posted on CLEAR's website.   00:00:23.580 --> 00:00:28.320 So CLEAR is funded through the Department  of Labor's Chief Evaluation Office   00:00:28.320 --> 00:00:32.040 and is supported by various  independent contractors. Next slide. 00:00:34.860 --> 00:00:40.680 To address any issues or submit content related  questions please select MSG Webex A,   00:00:40.680 --> 00:00:44.820 host and presenter, from the drop down menu in  the chat panel on your right and enter   00:00:44.820 --> 00:00:49.560 your questions in the chat box. Please  note that this event is being recorded   00:00:50.160 --> 00:00:55.860 so by continuing to participate you are  consenting to be recorded, so you may want   00:00:55.860 --> 00:01:01.440 to turn off your video as recording will  be posted on CLEAR's website. Next slide. 00:01:03.900 --> 00:01:09.794 So before we begin with today's presentation, I'm going to turn this over to Dr. Christina Yancey, 00:01:09.794 --> 00:01:16.920 who is the DOL's Chief Evaluation Officer,  to provide brief remarks. As DOL's Chief Evaluation   00:01:16.920 --> 00:01:23.700 Officer, Dr. Yancey leads the Department's research and  evaluation and evidence-building agenda to improve   00:01:23.700 --> 00:01:31.560 programs, policies, and enforcement activities that's been DOL's mission. She also serves as a   00:01:31.560 --> 00:01:37.020 Department scientific integrity official oversing  the Department's policy to address threats to   00:01:37.020 --> 00:01:39.950 scientific integrity. Christina? 00:01:41.210 --> 00:01:45.539 Thank you so much, Chayun, and thank you for having me and welcome 00:01:45.539 --> 00:01:52.200 everyone. We are so excited to host our final  webinar of the series celebrating CLEAR's ten-year   00:01:52.200 --> 00:01:59.520 anniversary and today we have a great topic we're  going to focus on how to search strategically for   00:01:59.520 --> 00:02:04.306 research evidence and CLEAR using equity as  kind of our cross-cutting theme.   00:02:04.306 --> 00:02:10.620 By the end of today's session, you will be... you will have learned different ways to find   00:02:10.620 --> 00:02:16.740 information in CLEAR on policies and programs  and strategies that address equity and to learn   00:02:16.740 --> 00:02:21.600 what the current evidence, rigorous scientific  evidence, says about supports and interventions   00:02:21.600 --> 00:02:27.780 that promote or may promote equitable labor  market outcomes. So the Department of Labor   00:02:27.780 --> 00:02:32.760 overall is committed to improving well-being of  all communities but especially those experiencing   00:02:32.760 --> 00:02:40.080 or historically have been experiencing underserved  marginalized or excluded experiences. To that end,   00:02:40.080 --> 00:02:45.120 I would like to begin if I mentioning just  a few of the things that the DOL's Chief   00:02:45.120 --> 00:02:49.800 Evaluation Office is supporting to advance  our knowledge about equity in the workforce.   00:02:50.700 --> 00:02:57.600 First, the Chief Evaluation Officer, CEO, is working  with several of the Department of Labor and other   00:02:57.600 --> 00:03:03.960 federal agencies to ensure integration of an  equity lens into design data collection analysis   00:03:03.960 --> 00:03:11.340 of our new evaluation projects. Secondly, we're  also supporting analysis of existing data assets   00:03:11.340 --> 00:03:18.660 and resources to show how equity questions could  be answered with existing administrative data and   00:03:18.660 --> 00:03:26.880 supporting internal analyses of DOL's own data to  understand our own work in new ways, and third, for   00:03:26.880 --> 00:03:32.280 the past couple of years CEO has funded what  were called a Summer Data Equity Challenge on   00:03:32.280 --> 00:03:38.760 equity and underserved communities which is  to help both analyze existing administrative   00:03:38.760 --> 00:03:44.220 resources and data and also to explicitly  support the pipeline of researchers that are   00:03:44.220 --> 00:03:50.640 examining equity implications of DOL policies,  protections, and programs. Now, while interest in   00:03:50.640 --> 00:03:56.280 advancing equity has been steadily growing, the movement has gained significant momentum in recent   00:03:56.280 --> 00:04:03.660 years with the passage of the Executive Order  13985, which is about advancing racial equity   00:04:03.660 --> 00:04:08.400 and support for underserved communities through  the federal governments and so we we note that   00:04:08.400 --> 00:04:14.340 as a signal of the fact that there's...  there's opportunities across different federal   00:04:14.340 --> 00:04:19.140 agencies that are also reinforcing and supporting  activities that we're discussing this afternoon   00:04:19.140 --> 00:04:25.860 as well as other initiatives at the federal, state,  and local levels. However, we all know that there   00:04:25.860 --> 00:04:30.240 are challenges in being able to use research  and evaluation in your everyday decision making   00:04:30.240 --> 00:04:36.360 and CLEAR is here to help with addressing that  challenge. CLEAR sifts through tens of thousands   00:04:36.360 --> 00:04:42.420 of research studies to find and review the  most credible ones that examine effective   00:04:42.420 --> 00:04:48.720 and promising programs across a variety of labor-related areas. So our goal today is to demonstrate   00:04:49.380 --> 00:04:55.200 how you might search that information on a very  broad topic like equity that might be nested in   00:04:55.200 --> 00:05:00.720 multiple places on CLEAR so that you're able to  quickly and easily find the information that you   00:05:00.720 --> 00:05:05.580 need and we are we will also highlight some  of the findings on equity that you can find   00:05:05.580 --> 00:05:10.920 in CLEAR. So I'm looking forward to a really  robust discussion today as well to hear from   00:05:10.920 --> 00:05:14.880 you all as well so that's why it's it's great  that Chayun had pointed out that there's the   00:05:14.880 --> 00:05:20.100 opportunity for you to engage with us as we spend  the next hour or so together. So thank you again   00:05:20.100 --> 00:05:23.793 for being here and I will turn it over to Chayun now. Chayun?    00:05:23.793 --> 00:05:25.552 Thank you, Christina. 00:05:25.552 --> 00:05:30.987 So, before we begin, I would like to introduce the presenters for today. 00:05:30.987 --> 00:05:34.200 So my name is Chayun Ying. I'm a part of DOL's 00:05:34.200 --> 00:05:40.500 Chief Evaluation Office's evaluation staff and a  member of CEO's CLEAR team. In addition to CLEAR 00:05:40.500 --> 00:05:47.400 project I manage a portfolio of various projects  covering DOL's capacity assessment, restricted use   00:05:47.400 --> 00:05:53.700 data program, and CEO summer fellows program, Dr. Kristein Abner is a dissemination   00:05:53.700 --> 00:05:59.880 lead for the ICF CLEAR team. Dr. Abner supports  on this project's focus on the workforce and   00:05:59.880 --> 00:06:06.180 social programming emphasizing the intersection of  research and practice and policy and has supported   00:06:06.180 --> 00:06:13.111 CLEAR since 2016. Dr. Shelley Osborn is the Rapid  Review Lead for the ICF CLEAR team. 00:06:13.111 --> 00:06:19.920 Dr. Osborne is a survey research and program evaluation specialist, supporting the federal sector including multiple   00:06:19.920 --> 00:06:27.120 projects within DOL's Chief Evolution Office. She has supported CLEAR since 2017. Next slide. 00:06:34.020 --> 00:06:38.684 So first I would like to share a bit  more about DOL's Chief Evaluation Office  00:06:38.684 --> 00:06:42.751 for those who are not familiar with CEO. 00:06:43.111 --> 00:06:51.540 CEO is an independent Department level advising office  since 2010. We coordinate, manage, and implement   00:06:51.540 --> 00:06:59.040 DOL's evaluation programs, and we work closely  with colleagues from about 15 DOL agencies to   00:06:59.040 --> 00:07:04.200 help address their research questions and support  their learning and continuous improvement efforts   00:07:04.200 --> 00:07:11.700 by conducting evaluation studies, data analysis,  and evaluation technical assistance. Most of our   00:07:11.700 --> 00:07:18.420 work are done through third-party contracts and  we currently have about 60 active projects for   00:07:18.420 --> 00:07:25.740 working with the DOL agencies. If you want to  learn more about CEO or our current studies or   00:07:25.740 --> 00:07:33.420 view recently completed studies please check out  the links at the bottom of the screen. Next slide. 00:07:36.360 --> 00:07:39.240 So I would like to ask you a couple questions   00:07:40.140 --> 00:07:43.680 on where you're from and  your experience with CLEAR. 00:07:46.080 --> 00:07:52.680 Our poll questions are here in the  chat on your right so please click   00:07:53.400 --> 00:07:59.640 please respond in the chat. The first poll is what organization are you from? 00:08:02.520 --> 00:08:08.220 Are you from the federal government, state,  local, tribal, or other government intermediaries, 00:08:08.940 --> 00:08:16.380 DOL grantee, research university,  employer or business, or other? 00:08:17.880 --> 00:08:22.020 And then the second question is  what is your experience with CLEAR? 00:08:23.280 --> 00:08:30.300 Novice. I have never used CLEAR and I'm interested  in learning more. Or are you an occasional user?   00:08:31.020 --> 00:08:38.880 I have used CLEAR before but could use a refresher.  Or are you a regular user? I use CLEAR whenever I   00:08:38.880 --> 00:08:43.980 need evidence on what works to inform my work  and I feel comfortable navigating the website   00:08:44.640 --> 00:08:54.300 and if not sure yet. So it looks like we  have people from various organizations and   00:08:56.040 --> 00:09:01.380 many of you are CLEAR Novice. Okay, next slide. 00:09:05.160 --> 00:09:07.980 So I'll turn it over to Kristin Abner. 00:09:10.500 --> 00:09:17.520 Thank you, Chayun. Now that we know a little bit about our audience, I want to go through the goals of today's webinar.  00:09:17.520 --> 00:09:24.900 So today, using equity as an example, our team will  really demonstrate how to strategically search the   00:09:24.900 --> 00:09:30.900 CLEAR website to find information on a subject or  population of interest. So we'll share information   00:09:30.900 --> 00:09:36.960 about what CLEAR has found on equity in workforce  and employment programs but our core purpose   00:09:36.960 --> 00:09:42.600 today is to really demonstrate strategies to find  information in the Clearinghouse. Next slide, please. 00:09:45.900 --> 00:09:52.200 So we'll accomplish our goals... oh can you... that's  good, thank you... we'll accomplish these goals by   00:09:52.200 --> 00:09:57.720 providing first a summary of key concepts from  our last webinar in this series which was how   00:09:57.720 --> 00:10:02.700 to use CLEAR for decision-making. We'll then show  and discuss multiple ways to find information in 00:10:02.700 --> 00:10:07.440 CLEAR using equity as our example and this will  take us through various parts of the Clearinghouse   00:10:07.440 --> 00:10:12.420 and will give us an opportunity to demonstrate  for you all the multiple ways that information is   00:10:12.420 --> 00:10:18.300 synthesized and presented in CLEAR. We'll summarize  some findings across CLEARs products and then   00:10:18.300 --> 00:10:23.820 we'll close with a recap of the many resources  in CLEAR before we open the webinar up for Q&A. 00:10:25.020 --> 00:10:26.100 Next slide, please/ 00:10:28.920 --> 00:10:34.920 So, as Dr. Yancey mentioned this is... oh can you go  back? Thank you. As Dr. Yancey mentioned, this is the   00:10:34.920 --> 00:10:39.420 third webinar in a series that are celebrating  the 10-year anniversary of CLEAR. So our first   00:10:39.420 --> 00:10:44.340 webinar focused on CLEAR 101 and the second  webinar focused on using CLEAR for decision   00:10:44.340 --> 00:10:50.340 making and I want to share a brief recap of that  webinar before we move forward and just to note   00:10:50.340 --> 00:10:55.980 that the recording, PowerPoint presentation, and  transcript of the question and answers for   00:10:55.980 --> 00:11:00.660 the webinars will be available forthcoming  on CLEAR's website, so I will encourage   00:11:00.660 --> 00:11:05.460 you to view those for more information  when they're available. Next slide, please. 00:11:08.040 --> 00:11:13.080 So recapping the second webinar, the webinar began with a hypothetical question   00:11:13.980 --> 00:11:17.280 and a community college  administrator wondering how   00:11:17.280 --> 00:11:23.160 persistence and completion could be improved  for their college training class. Next slide. 00:11:25.920 --> 00:11:30.780 Our administrator learned that evidence on CLEAR comes in the form of evidence reviews, evidence   00:11:30.780 --> 00:11:36.480 syntheses, and profile summaries. Our speakers  explained that the resource... explain the resources   00:11:36.480 --> 00:11:40.560 and walked the audience through the Clearinghouse  to find evidence related to the policy question. 00:11:41.280 --> 00:11:45.840 We'll be discussing CLEAR's Rapid Reviews on new  emerging research forthcoming in this presentation   00:11:45.840 --> 00:11:51.780 so I'll focus on the other three for the  purpose of this recap. So evidence reviews are   00:11:51.780 --> 00:11:57.300 collections of research on a topic and  these may be defined by a population, so that might   00:11:57.300 --> 00:12:03.360 include things like low-income adults veterans or  older work, an outcome of interest, which could be   00:12:03.360 --> 00:12:10.140 literacy, re-employment, or equitable post-COVID  recovery, or an intervention type, so these might   00:12:10.140 --> 00:12:15.960 include apprenticeship or work-based training, remote post-secondary education, or remote service   00:12:15.960 --> 00:12:21.300 delivery strategies. So our evidence indices are  short narratives that summarize the findings on   00:12:21.300 --> 00:12:27.300 what works in a given topic and note gaps where  evidence is needed. Our profile summaries provide   00:12:27.300 --> 00:12:32.340 details on individual studies that CLEAR has  reviewed. So usually studies are reviewed as   00:12:32.340 --> 00:12:38.460 part of a search targeted by a topic or a time  period, occasionally a profile summary will cover   00:12:38.460 --> 00:12:44.100 a flash review of a single study that falls under  an existing evidence review. So our profiles are   00:12:44.100 --> 00:12:50.760 really short snippets of a larger study and you'll  notice on the CLEAR website that the profiles will   00:12:50.760 --> 00:12:55.500 link to that larger study if you want to go  there for more information. Next slide, please. 00:12:58.740 --> 00:13:05.160 So in our hypothetical situation our college  administrator found strategy across CLEAR's 00:13:05.160 --> 00:13:10.334 products for linked learning communities, paid performance incentives, and accelerated learning. 00:13:10.334 --> 00:13:14.100 And each of these strategies had  favorable impacts, were from credible studies,  00:13:14.100 --> 00:13:19.260 and addressed the Community College population. So with this information in hand, they were in a   00:13:19.260 --> 00:13:24.120 better position to discuss new program or program  changes to improve persistence and completion. 00:13:26.520 --> 00:13:27.720 Next slide, please. 00:13:29.760 --> 00:13:34.320 So today, our webinar is on strategies for  searching CLEAR using equity as an example   00:13:34.320 --> 00:13:39.840 so we're using equity as our example  for several reasons. As Dr. Yancey mentioned   00:13:39.840 --> 00:13:43.860 equity is a priority for the Department of  Labor and the Chief Evaluation Office is   00:13:43.860 --> 00:13:48.720 working across the department and other  federal agencies on multiple projects   00:13:48.720 --> 00:13:53.460 related to equity. Equity also crosses  different types of evidence from use in   00:13:53.460 --> 00:13:59.520 CLEAR and therefore allows us to approach a  search using multiple strategies. Lastly, equity   00:13:59.520 --> 00:14:05.460 well it doesn't have its own evidence review,  you can search using keywords, target populations,  00:14:05.460 --> 00:14:10.680 and other strategies that will demonstrate  throughout this webinar. Next slide, please. 00:14:14.460 --> 00:14:20.010 So before we dive in we want to kind of set the  stage here distinguishing equity from equality.   00:14:20.010 --> 00:14:26.280 So this image on the screen is from the Robert Wood  Johnson Foundation. The left hand slide shows a   00:14:26.280 --> 00:14:31.320 one-size-fits-all curb, that everyone needs  to negotiate across the street. On the right   00:14:31.320 --> 00:14:36.600 though the curve and the signs are adapted to  accommodate barrier, circumstances, and conditions. 00:14:37.200 --> 00:14:42.300 Equity in the context of labor market outcomes  refers to the fairness and justice in employment, 00:14:42.300 --> 00:14:47.400 education and training and earnings. It's distinct  from equality in that equity accounts for the fact   00:14:47.400 --> 00:14:52.620 that not all populations start from the  same place and interventions that promote   00:14:52.620 --> 00:14:58.980 equity adjust to counteract imbalances related to  bias, discrimination, and other systemic barriers. 00:15:03.000 --> 00:15:06.420 So now I'm going to turn the  mic over to Dr. Shelley Osborn   00:15:06.420 --> 00:15:09.420 to walk us through our strategic search. Shelly? 00:15:11.400 --> 00:15:16.217 All right thank you so much, Kristen. So next slide, please. 00:15:17.701 --> 00:15:23.464 So there are multiple places to look in CLEAR for equity or anything else that you might be interested in 00:15:23.464 --> 00:15:27.291 And as Kristin summarized, there are multiple types of revie ws in CLEAR. 00:15:27.291 --> 00:15:32.053 So for our equity example, we're going  to start with new and trending research. 00:15:32.053 --> 00:15:35.880 So CEO conducted a rapid review to identify strategies   00:15:35.880 --> 00:15:41.760 and evidence to promote equitable recovery  from COVID-19. As shown here from the home   00:15:41.760 --> 00:15:47.940 page, clear.dol.gov, you would navigate to  New and Trending Research and use the drop   00:15:47.940 --> 00:15:54.720 down menu to select Trending Research. This will lead you to the rapid reviews. Next slide, please. 00:16:01.920 --> 00:16:06.780 So in CLEAR you'll find a list of the rapid  reviews, we list them here, of which there   00:16:06.780 --> 00:16:13.140 are nine currently each with a short description  summarizes the publications reviewed. In addition   00:16:13.140 --> 00:16:17.700 to equitable recovery from COVID-19, rapid reviews address worker health   00:16:17.700 --> 00:16:23.400 and safety, distance learning, remote compliance  assistance and enforcement, just to name a few. 00:16:24.540 --> 00:16:30.840 For anything that looks promising, hyperlinks open  the document. Each rapid review follows the same   00:16:30.840 --> 00:16:36.900 general format. A guiding research question begins  the document. This is followed by an introduction   00:16:36.900 --> 00:16:43.140 of the subject and the objective of the rapid  review. In general, these types of reviews identify   00:16:43.140 --> 00:16:49.140 and summarize existing literature. The rapid  reviews are organized around strategies that   00:16:49.140 --> 00:16:55.560 have shown to be or may be promising. The reviews  conclude by describing some gaps in the research. 00:16:56.280 --> 00:17:02.160 Each rapid review also contains a supplement with  summaries of each publication that's included in   00:17:02.160 --> 00:17:08.280 the document. So each summary includes links  to the article, report, or publication, so that   00:17:08.280 --> 00:17:13.140 you can go to the primary source material.  Some of those links will take you to the   00:17:13.140 --> 00:17:21.600 full text, while others may be accessible only  through a library account. Next slide, please. 00:17:26.100 --> 00:17:32.700 So another place to look for research in CLEAR are the systematic evidence reviews. So the focus   00:17:32.700 --> 00:17:38.820 of these systematic evidence reviews are causal  studies, some include descriptive studies that use   00:17:38.820 --> 00:17:44.340 statistical techniques to another quantitative  approaches but do not attempt to assess the   00:17:44.340 --> 00:17:51.060 causal impact of a program or intervention, some  policies also include implementation studies.   00:17:51.600 --> 00:17:57.480 We list here just some of what is covered in  terms of populations outcomes and interventions. 00:17:57.480 --> 00:18:03.540 As noted in our recap of the second webinar, systematic evidence reviews can be found and   00:18:03.540 --> 00:18:08.340 CLEAR through the study profiles of the  individual studies and for many topics   00:18:08.340 --> 00:18:13.560 and in-depth synthesis of the evidence is  also in CLEAR. We're going to turn our   00:18:13.560 --> 00:18:18.600 attention now to some of the systematic evidence  reviews that touch on equity. Next slide, please. 00:18:21.600 --> 00:18:28.140 Our first is employer compliance. Here  we show what is covered by the systematic review   00:18:28.140 --> 00:18:34.500 and who is covered by the rules, policies, and enforcement activities. This is a good   00:18:34.500 --> 00:18:40.951 time to note that CLEAR is always expanding. So while there are currently 22 studies, 00:18:40.951 --> 00:18:47.160 systematic searches for new research occur regularly  and as those systematic evidence reviews are   00:18:47.160 --> 00:18:53.880 updated, CLEAR will contain additional studies. So you can check back often to see what's new   00:18:53.880 --> 00:19:00.840 in CLEAR in these for systematic evidence  reviews or rapid reviews. Next slide, please. 00:19:03.180 --> 00:19:11.220 So disability employment policy is another example  of a systematic evidence review. You'll note that   00:19:11.220 --> 00:19:17.445 unlike our last example, employer compliance, this  evidence review includes a synthesis report, 00:19:17.445 --> 00:19:24.660 which we talked a little about. So the syntheses look at the  research evidence across studies, highlights gaps   00:19:24.660 --> 00:19:31.620 in the literature, and then also suggests areas in  which further research is needed. Next slide, please. 00:19:35.520 --> 00:19:42.420 So our last example is older workers. Like disability employment policy, this topic includes   00:19:42.420 --> 00:19:49.380 a synthesis report. You'll note that the research  extends back to 2005, which is more recent than the   00:19:49.380 --> 00:19:56.280 previous two examples. Decisions on what years to  include and what studies to include are decided   00:19:56.280 --> 00:20:02.940 by DOL's CEO office and subject matter experts. These are then documented in a study protocol   00:20:02.940 --> 00:20:10.800 which can also be found in CLEAR. So pausing here  a moment provides a nice segue about how to search   00:20:10.800 --> 00:20:16.200 in CLEAR. If you are interested in older workers, you now know that there are individual studies   00:20:16.200 --> 00:20:22.200 and even a synthesis report you can review but  what if, like equity, there isn't a drop-down menu   00:20:22.200 --> 00:20:28.320 that has what you need? How can you search in CLEAR to find that information? Next slide, please. 00:20:30.960 --> 00:20:37.440 So CLEAR is currently expanding how studies are  tagged to follow the 2020 census taxonomy for race   00:20:37.440 --> 00:20:43.500 and ethnicity. Like other statistical agencies, the Census follows standards on race and ethnicities   00:20:43.500 --> 00:20:50.040 set by the U.S Office of Management and Budget  in 1997. These standards guide how the federal   00:20:50.040 --> 00:20:55.980 government collects and presents data on race  and ethnicity. As we've talked about, equity is   00:20:55.980 --> 00:21:02.160 not its own section in CLEAR, and in fact the  CLEAR products and content that include equity   00:21:02.160 --> 00:21:07.260 issues can be found across the full spectrum  of the Clearinghouse, so we want to take a few   00:21:07.260 --> 00:21:11.580 minutes to focus on different strategies  to find evidence and information in CLEAR. 00:21:13.020 --> 00:21:17.400 The first thing to note is that search in CLEAR is similar to how you would search for   00:21:17.400 --> 00:21:22.800 any information on the web. You often start with  a search term that leads to another search term   00:21:22.800 --> 00:21:30.360 that leads to still another search term. So  because authors refer to similar concepts in   00:21:30.360 --> 00:21:34.680 different words you may have to experiment  with a few different keywords before you   00:21:34.680 --> 00:21:40.980 find the information that you're looking for. In this example, I first search for in turn BIPOC,   00:21:40.980 --> 00:21:47.280 Black, Indigenous, and People of Color, and then  equity and then equitable. So while each term   00:21:47.280 --> 00:21:52.260 returned an increasing number of studies, I still  wasn't finding exactly what I was looking for.   00:21:53.100 --> 00:21:59.940 So that may be because equity is a fairly recent  term, as is BIPOC, and the term again needs to   00:21:59.940 --> 00:22:04.620 be in the publication and therefore the study  summary to be captured by the search function. 00:22:04.620 --> 00:22:10.620 So thinking about that, I began to experiment  with different search terms as listed here. 00:22:12.060 --> 00:22:18.060 So the same process could be used to search  for other racial or ethnic groups, examples of   00:22:18.060 --> 00:22:24.000 which are shown. You can also search for specific  geographic areas and a keyword such as a state if   00:22:24.000 --> 00:22:28.500 you were interested in, say, just programs that  or studies that were conducted in Wisconsin,  00:22:28.500 --> 00:22:35.160 or regions, if you were just interested in studies  conducted say in the Southwest. Next slide, please. 00:22:38.520 --> 00:22:42.780 So on the last slide, you can see, excuse me, if  you go back to the last slide for one moment... 00:22:49.620 --> 00:22:55.080 So here you can see that I searched for  Black, and then you could also search   00:22:55.080 --> 00:23:00.600 for publication years, so that's how you can  limit a search to just articles that were   00:23:00.600 --> 00:23:05.040 published since 2010 if you wanted something  more recent. Thank you, next slide, please. 00:23:06.420 --> 00:23:12.210 So you can also search by other parameters that  are shown here on the right. So in this example,   00:23:12.210 --> 00:23:19.020 I mean we find my results to the outcome that I was  interested in, limited my results to the US, and   00:23:19.020 --> 00:23:24.420 then restricted my research to studies that assess  the causal impact or programs or interventions. 00:23:25.620 --> 00:23:30.000 This is a good time to note also that  most of the studies in CLEAR are based in the   00:23:30.000 --> 00:23:37.320 United States with a few exceptions such as those  focused on child labor outcomes. Next slide, please. 00:23:39.240 --> 00:23:45.180 So as another example, I initially searched  for studies in CLEAR using the term Latinx 00:23:45.180 --> 00:23:52.380 as my only search term and I did not find many  results. So, as with BIPOC, Latinx is a newer term   00:23:52.380 --> 00:23:57.600 and again the term must be in the original  publication and CLEAR study summary or study   00:23:57.600 --> 00:24:04.020 profile so that it's captured by the search.  So I searched instead using a drop down menu   00:24:04.020 --> 00:24:10.320 for any of these words and included other terms  that researchers may have used other than Latinx.  00:24:10.320 --> 00:24:15.780 So the search returned actually dozens of studies. After reading through a few pages of results, 00:24:15.780 --> 00:24:21.540 I decided to refine my search using the filter  options. So the study shown here was just one   00:24:21.540 --> 00:24:26.506 of the studies I found after I refined my search  to those with the high causal evidence rating. 00:24:26.506 --> 00:24:31.740 As we've mentioned, the high causal evidence ratings  are given to studies in which we can be confident   00:24:31.740 --> 00:24:37.080 that the effects were caused by the intervention  examined. As you may know, CLEAR rates the strength   00:24:37.080 --> 00:24:45.300 of the causal evidence in studies as high, moderate,  or low. So from here I can search CLEAR using the   00:24:45.300 --> 00:24:50.160 study author's name just to see if they have  anything else that's included on CLEAR that   00:24:50.160 --> 00:24:55.380 was reviewed and then you'll see tags at the  bottom so I could also click on one or more   00:24:55.380 --> 00:25:02.160 of those tags and that will take me to places in  CLEAR that have related studies. Next slide, please. 00:25:04.560 --> 00:25:10.800 So up to now we've been searching using keywords  another way to search on CLEAR is by starting   00:25:10.800 --> 00:25:16.200 with one or more of the available filters. So there are multiple layers to these filters, 00:25:16.200 --> 00:25:22.440 for example on the right, other options you can  select under target population, and then within   00:25:22.440 --> 00:25:28.620 each of these are additional filters. We show some  of these on the next slide. The next slide, please. 00:25:30.480 --> 00:25:37.349 So this shows just two of the additional levels,  so what is underneath age and then employment barriers. 00:25:37.349 --> 00:25:43.274 Note that there's an option to select  older workers when you refine by the target population 00:25:43.274 --> 00:25:49.260 and then further refine by age. So doing so is another way in this case to access the   00:25:49.260 --> 00:25:55.680 systematic evidence review for older workers, which  you now know exists in CLEAR. In addition to the   00:25:55.680 --> 00:26:02.640 individual studies, currently 43, as we discussed on  a prior slide, there is an older worker synthesis   00:26:03.300 --> 00:26:10.169 also if you click on the older workers in CLEAR there is a report titled Behavioral Finance Synthesis. 00:26:10.169 --> 00:26:14.880 Many of the studies in this report  look at how individuals often make different   00:26:14.880 --> 00:26:21.660 decisions about saving for retirement than classic  economic theory would predict. Next slide, please. 00:26:23.820 --> 00:26:28.260 So let's suppose that we used one  or more of a combination of these    00:26:28.260 --> 00:26:31.920 search strategies to find evidence  in CLEAR regarding equity issues.  00:26:31.920 --> 00:26:37.021 What would we find? So we'll turn now to some highlights across CLEAR. 00:26:37.021 --> 00:26:39.021 Next slide, please. 00:26:40.440 --> 00:26:45.480 So as equity is our example today, we're going  to start by discussing the rapid review on   00:26:45.480 --> 00:26:51.780 equitable recovery from the COVID-19 pandemic. This rapid review was completed earlier this year   00:26:51.780 --> 00:26:55.980 and we continue to look for publications  that provide new insight and information. 00:26:57.360 --> 00:27:02.640 This review summarized information from dozens  of publications, some of which were causal studies,   00:27:02.640 --> 00:27:09.704 some that were descriptive studies, and others  that were literature reviews, policy briefs, and other reports. 00:27:09.704 --> 00:27:14.760 Key concepts, such as equity  in this case, are defined in the rapid review. 00:27:16.020 --> 00:27:21.045 This rapid review focuses on Black,  Indigenous, and People of Color and women, 00:27:21.045 --> 00:27:25.680 while recognizing that people may also  experience marginalization based on their   00:27:25.680 --> 00:27:32.760 sexual orientation, disability status, immigration  status, religion, income level, among other factors. 00:27:33.540 --> 00:27:38.640 BIPOC individuals and women were  disproportionately affected by the pandemic   00:27:38.640 --> 00:27:44.340 which is one reason the rapid review focused  on these populations. As noted, CLEAR is always   00:27:44.340 --> 00:27:51.540 expanding so future updates to the rapid review  may include other populations. Next slide, please. 00:27:53.760 --> 00:27:59.760 There are four strategies identified in the  literature for the rapid review synthesis. 00:28:00.900 --> 00:28:07.140 First, education and training supports, with  an equity focus, deliberately recruit, retain,  00:28:07.140 --> 00:28:13.980 and support participants from marginalized groups. Employment and other supportive services include   00:28:13.980 --> 00:28:21.180 strategies like telework, quality child care, and  mental health services. A key Federal response   00:28:21.180 --> 00:28:26.580 to support small businesses during COVID-19  was the Paycheck Protection Program or PPP,   00:28:27.360 --> 00:28:33.360 a component of the Coronavirus Aid, Relief, and  Economic Security Act, or the CARES Act, that was   00:28:33.360 --> 00:28:39.960 made available in April 2020. Place-based  interventions are programs to improve economic   00:28:39.960 --> 00:28:46.920 and social conditions in specific geographic areas  by providing resources to government, non-profit, 00:28:46.920 --> 00:28:53.820 or private sector entities instead of providing  resources directly to individuals. Examples of such   00:28:53.820 --> 00:29:00.540 programs are Community Development Block Grants,  the New Market Tax Credit, HUD Empowerment Zones, 00:29:00.540 --> 00:29:07.500 Opportunity Zones, and also Promise Neighborhood. So over the next four slides we're going to show  00:29:07.500 --> 00:29:13.440 the main takeaway findings under each  of these strategies. While detailed findings   00:29:13.440 --> 00:29:18.900 are outside the scope of our webinar, we will  provide one or two examples of research as we   00:29:18.900 --> 00:29:24.900 move through the summary. We encourage you  to review the rapid review. Next slide, please. 00:29:27.420 --> 00:29:32.041 So here we show the takeaway findings for  education and training supports. 00:29:32.041 --> 00:29:37.591 The third bullet notes that directly addressing  equity can improve access to 00:29:37.591 --> 00:29:45.000 online Career and Technical Education, or CTE,  programs, but how exactly can this be done? 00:29:45.000 --> 00:29:50.700 So one publication in this section  recommended revising coursework that   00:29:50.700 --> 00:29:56.340 requires substantial computing power, internet  bandwidth, or high digital literacy skills.  00:29:56.940 --> 00:30:03.240 Another recommended assistive technologies such  as text-to-speech functionalities, captioning, 00:30:03.240 --> 00:30:10.200 alternative text, and visual aids, so that students  with vision and or hearing impairments or limited   00:30:10.200 --> 00:30:17.040 English proficiency could access online  CTE courses or programs. Next slide, please. 00:30:19.560 --> 00:30:25.320 So listed on this slide are the main takeaway  findings included in the rapid review under   00:30:25.320 --> 00:30:30.780 employment supports and other supportive services. So the second bullet notes the   00:30:30.780 --> 00:30:36.960 importance of expanding telework and flexible  work options. Studies in the section noted that   00:30:36.960 --> 00:30:42.720 working from home differs by race and ethnicity  among occupations that can be done outside of   00:30:42.720 --> 00:30:49.920 the office and that Latino and Black workers are  overrepresented in jobs that do not offer telework. 00:30:49.920 --> 00:30:57.240 For example one study found that 16% of Hispanic  workers and 20% of Black workers compared   00:30:57.240 --> 00:31:03.660 to 30% of white workers were offered  the opportunity to telework. Next slide, please. 00:31:07.140 --> 00:31:12.720 So here we have some findings for supports  for small businesses and we want to talk a   00:31:12.720 --> 00:31:18.300 little about the second bullet. So publications  reviewed for this section of the rapid review   00:31:18.300 --> 00:31:23.700 discussed many of the difficulties with the  government's Paycheck Protection Program, or PPP. 00:31:24.540 --> 00:31:30.840 For example the nation's smallest businesses were  less aware of the PPP and less likely to apply.  00:31:31.680 --> 00:31:37.800 One of the studies discussed in this section was  a case study of pandemic relief for Latino-owned   00:31:37.800 --> 00:31:44.400 businesses in the Washington DC area. It found that  non-governmental and community-based organizations   00:31:44.400 --> 00:31:51.900 raised awareness about the available relief  options. So therefore building the capacity of   00:31:51.900 --> 00:31:57.480 these community-based financial institutions could  increase accessibility to Federal relief for those   00:31:57.480 --> 00:32:04.320 who have difficulty accessing it, including micro  businesses, which are defined as those with 10 or   00:32:04.320 --> 00:32:08.940 fewer, excuse me, with less than 10 employees, and then businesses in communities of color.   00:32:09.705 --> 00:32:11.912 Next slide, please. 00:32:13.486 --> 00:32:20.189 So the last strategy that's discussed in the rapid review for equitable recovery is place-based interventions. 00:32:20.189 --> 00:32:28.441 So recall that place-based interventions are when resources are directed to government, nonprofit, or private sector entities. 00:32:28.441 --> 00:32:35.280 Activities vary widely and include providing block grants,  offering tax incentives to businesses, and   00:32:35.280 --> 00:32:42.120 investing in local physical infrastructure, or  providing Workforce training. Regarding the second   00:32:42.120 --> 00:32:48.060 bullet, one of the publications reviewed in this  section noted that place-based programs rarely   00:32:48.060 --> 00:32:54.616 include components that explicitly target local  conditions caused by racially biased policies. 00:32:54.975 --> 00:33:01.980 For example, opportunity zones are designed to attract  outside investment. They do not specifically target   00:33:01.980 --> 00:33:07.636 wages, creating jobs, or supporting local  ownership of business. 00:33:07.636 --> 00:33:09.591 Next slide, please. 00:33:11.520 --> 00:33:16.140 So we turn our attention now to another  rapid review in CLEAR, one that's focused   00:33:16.140 --> 00:33:22.620 on remote post-secondary education. This rapid  review addresses the research question: what is   00:33:22.620 --> 00:33:27.540 the evidence on technology-based or distance  learning that can inform efforts to provide   00:33:27.540 --> 00:33:29.936 post-secondary education remotely? 00:33:29.936 --> 00:33:33.720 So equity considerations is a section in this rapid review. 00:33:35.040 --> 00:33:41.280 Among some of the findings emerging in this  review, researchers noted that remote CTE compared   00:33:41.280 --> 00:33:46.860 to in-person CTE attracted more working  professionals, students who were currently   00:33:46.860 --> 00:33:53.100 employed, students outside of that school's  district, single parents, and part-time students. 00:33:53.640 --> 00:33:58.920 So the publication's authors noted that these  dimensions of difference should be kept in   00:33:58.920 --> 00:34:05.700 the forefront when designing education and  training courses and programs. Next slide, please. 00:34:07.980 --> 00:34:13.685 We highlight here some findings that address  equity from three more rapid reviews. 00:34:13.685 --> 00:34:21.119 So the first is from return to work. The guiding research  question here was in response to the COVID-19 pandemic. 00:34:21.119 --> 00:34:26.460 What strategies can be effective for  helping unemployed individuals return to work? 00:34:27.540 --> 00:34:33.840 The second is from the worker health and safety  rapid review. That addressed the research question: 00:34:33.840 --> 00:34:39.420 What strategies can promote worker health  and safety during the COVID-19 pandemic?  00:34:40.140 --> 00:34:44.340 And the remote service delivery rapid  review address the research question: 00:34:44.340 --> 00:34:49.140 What is the evidence on remote service  delivery programs and strategies to   00:34:49.140 --> 00:34:55.200 inform DOL's efforts to provide remote  employment services. Next slide, please. 00:34:58.620 --> 00:35:03.384 So we want to return for a moment to the rapid  review for equitable recovery. 00:35:03.384 --> 00:35:11.880 So there's a section, the fifth in the rapid review, that discusses ways to measure equitable recovery that have   00:35:11.880 --> 00:35:18.480 been or could be used to assess recovery among  women, BIPOC, and other marginalized population.  00:35:19.200 --> 00:35:24.871 Historically, employment and earnings have  been used to measure labor market recovery. 00:35:24.871 --> 00:35:32.580 As important and illuminating as these can be, some experts posit that they provide an incomplete   00:35:32.580 --> 00:35:40.200 picture of economic recovery from an inclusive or  equitable perspective. So listed here on the slide   00:35:40.200 --> 00:35:46.860 are the main ideas included on the rapid review  for different ways to measure equitable recovery   00:35:46.860 --> 00:35:54.360 outside of, again, labor market earnings. And this  could be applicable for multiple topic areas and   00:35:54.360 --> 00:35:59.940 for equity more generally. So I'll pause here for  just a little bit so you can read through them. 00:36:07.320 --> 00:36:08.880 Okay, next slide, please. 00:36:11.880 --> 00:36:16.800 Lastly, we list some research gaps that  were identified across several of the   00:36:16.800 --> 00:36:22.440 rapid reviews that address equity. The first  research tap comes from the CLEAR rapid review   00:36:22.440 --> 00:36:27.240 on remote secondary, remote post-secondary  education, which we've talked a little about. 00:36:28.080 --> 00:36:34.200 The second is from the worker, excuse me, the rapid  review on worker health and safety during COVID-19. 00:36:34.200 --> 00:36:39.780 again, which we've mentioned, and the third  is the rapid review that presented evidence   00:36:39.780 --> 00:36:45.180 on targeted or remote compliance assistance  and enforcement strategies that could assist   00:36:45.180 --> 00:36:51.300 employers in conforming to new and existing health  and safety and work requirements under COVID-19. 00:36:52.200 --> 00:36:56.700 The last is from the equitable recovery rapid  review which we've talked quite a bit about. 00:36:57.780 --> 00:37:04.140 We focused on the rapid reviews since these  synthesize existing evidence. We encourage   00:37:04.140 --> 00:37:08.940 you to explore CLEAR using the strategies  we've outlined today to find research for the   00:37:08.940 --> 00:37:14.700 populations and outcomes that you're interested in. I will now turn things back over to Chayun. 00:37:18.360 --> 00:37:20.580 Thank you Shelley. Next slide. 00:37:23.820 --> 00:37:27.300 So I would like to share  additional resources before   00:37:27.300 --> 00:37:34.080 we dive into questions and answers  part of the presentation. Next slide. 00:37:38.700 --> 00:37:44.820 So here we have the links to some of CLEAR resources. We already had some   00:37:44.820 --> 00:37:51.920 questions about whether this presentation or the slides will be posted on CLEAR's website? 00:37:51.920 --> 00:37:58.620 You can check out CLEAR's website.   All three webinars... the recordings will be   00:37:58.620 --> 00:38:04.140 posted we'll also have a questions and answer  transcript and the PowerPoint slides there. 00:38:04.740 --> 00:38:12.900 You can... if you want to hear more from CEO, you can join us for future CEO events by   00:38:12.900 --> 00:38:18.240 clicking through the links, and you can  sign up for CEO's newsletter. Next slide. 00:38:22.200 --> 00:38:27.960 And there are additional equity resources as  Christina mentioned, Department of Labor takes   00:38:27.960 --> 00:38:39.660 seriously the serving of underserved  communities... of underserved, 00:38:39.660 --> 00:38:45.629 marginalized, or excluded communities. So please  check out DOL's Equity Action Plan to find out more 00:38:45.629 --> 00:38:52.380 as well the latest papers and data sets  from CEO's Summer Data Challenge on Equity and   00:38:52.380 --> 00:38:59.340 Underserved Communities is available on CEO's  website, so please check it out. Next slide. 00:39:03.060 --> 00:39:10.859 So before moving on to discussions and questions, let's take a quick poll on the quality of today's presentation. 00:39:10.859 --> 00:39:15.509 First, how would you rate the  quality of speakers content of today's seminar? 00:39:15.509 --> 00:39:19.860 Very high quality, high quality  neutral, low quality, or very low quality. 00:39:56.040 --> 00:39:57.240 Okay next question. 00:40:03.360 --> 00:40:08.290 Overall, how would you rate the relevancy  of today's topic of your work? 00:40:08.290 --> 00:40:13.320 Is it very relevant, relevant, somewhat relevant, or irrelevant? 00:40:45.780 --> 00:40:48.960 And Brittni, whenever you're ready  you can advance the next question. 00:40:54.600 --> 00:41:00.180 The last question: what specific areas would you  suggest CLEAR feature or investigate further? 00:41:00.180 --> 00:41:06.840 And you can select multiple options. More research on equitable recovery, 00:41:07.500 --> 00:41:12.305 more evidence on remote education training, more on implementation research, 00:41:12.305 --> 00:41:19.620 more on the employment services and eligibility assessment  program, or other. And you can specify in the chat. 00:41:52.440 --> 00:41:56.820 And Brittni, you can close the chat  once the analysis is complete. 00:42:29.820 --> 00:42:34.980 Okay so the poll has ended. Next slide. 00:42:39.780 --> 00:42:45.780 Okay so I'll open up the floor for  any questions or discussions... 00:43:09.480 --> 00:43:12.022 Okay so I have a question. 00:43:12.022 --> 00:43:17.520 Will CLEAR consider adding a tag or a way to filter for additional terms such as equity? 00:43:17.520 --> 00:43:19.860 Is there a process to submit recommendations? 00:43:23.640 --> 00:43:26.700 And Kristin and Shelley, could you guys take this? 00:43:31.620 --> 00:43:38.340 Definitely. I think the CLEAR as we  noted in the presentation CLEAR's collection is   00:43:39.120 --> 00:43:49.560 growing constantly and so as we are moving  and adding new things to the collection we   00:43:49.560 --> 00:43:56.520 think about how that... how we might tag and filter  for additional topics. So it's definitely under   00:43:56.520 --> 00:44:01.680 consideration. If folks have recommendations, on the CLEAR website you can contact us through   00:44:01.680 --> 00:44:08.460 the form on the website which will come to the  team at CEO and we'd be happy to take those. 00:44:11.760 --> 00:44:13.564 Chayun, do you have anything to add? 00:44:13.564 --> 00:44:14.940 No, thank you. 00:44:16.320 --> 00:44:22.320 I have another question. What are the sources in general of the research on CLEAR? 00:44:24.420 --> 00:44:26.460 Is it only DOL grantees? 00:44:29.040 --> 00:44:36.840 So CLEAR conducts, when we do our  systematic evidence reviews, we conduct a   00:44:36.840 --> 00:44:43.320 very wide reaching search to gather all of  the causal evidence in a given topic area.  00:44:44.520 --> 00:44:49.500 And one of the things that you'll learn  from the previous webinars is that CLEAR also   00:44:49.500 --> 00:44:55.800 includes descriptive and implementation studies,  but we cast a really wide net, so you'll   00:44:55.800 --> 00:45:01.380 get dissertations that will be included. It's not just limited to DOL publications,   00:45:02.640 --> 00:45:08.520 DOL grantees, so we do... we capture  great literature from organizations that may post   00:45:09.540 --> 00:45:15.420 reports on their websites but we look at the academic literature, of course,  00:45:15.420 --> 00:45:20.535 journal articles, dissertations, like I  mentioned, gray literature,   00:45:20.535 --> 00:45:25.740 other federal agencies and departments would be included  if it would fit the criteria for that search. 00:45:30.120 --> 00:45:35.400 Thank you. So, someone asked how are tags, study type, etc. created for these studies? 00:45:41.880 --> 00:45:48.060 So if you go on the CLEAR website, we have... one of the goals of CLEAR is to be transparent in how   00:45:48.060 --> 00:45:57.060 we tag and filter our resources and so we have  a tag and filter guidelines and they were created   00:45:57.060 --> 00:46:03.659 really thinking about the kind of the broad scope  of literature around employment and labor related programs. 00:46:03.659 --> 00:46:08.940 So those are how the tags and filters  are initially created, so you can go on the   00:46:08.940 --> 00:46:21.240 website and see how we tag and filter resources  and we, like I said, we really we think   00:46:21.240 --> 00:46:27.660 about how we can always improve our tag and  filter process to help users find what they need. 00:46:29.160 --> 00:46:36.360 Thank you. Does CEO plan any future research  specifically focused on equity in workforce program? 00:46:49.260 --> 00:46:53.820 Does Shelley or Kristin... do you want to take that? 00:46:59.760 --> 00:47:04.681 I can't speak for DOL, but I'll speak for CLEAR. We're currently, as Shelley mentioned, 00:47:04.681 --> 00:47:13.140 we're updating evidence reviews currently at the  disability employment evidence review is   00:47:13.140 --> 00:47:17.280 currently being updated a lot of the other  evidence reviews on CLEAR are being updated. 00:47:18.300 --> 00:47:23.340 And you know... I'll let Shelley, do you want to  talk a little bit about our rapid reviews and how   00:47:23.340 --> 00:47:25.925 those sort of address new and upcoming topics? 00:47:25.925 --> 00:47:32.024 Certainly and one thing I wanted to go back to about the search terms that we use 00:47:32.024 --> 00:47:39.880 in the rapid reviews we do list, I didn't mention it during the presentation, but in addition to the strategies, 00:47:39.880 --> 00:47:43.620 (coughs) excuse me, and the gaps in research 00:47:43.620 --> 00:47:49.380 and all of the study summaries, there is also  a section that explains what search terms and   00:47:49.380 --> 00:47:53.940 keywords we searched for. So for folks that are  really interested in kind of how that search   00:47:53.940 --> 00:47:57.573 was done, they can look for those. 00:47:57.573 --> 00:48:05.922 And then, the rapid reviews in general, every, and again this is a fairly complicated process that I won't go into great detail, but, 00:48:05.922 --> 00:48:13.860 every few weeks, we are sort of looking for, again, using a  variety of different keywords in a variety of   00:48:13.860 --> 00:48:19.560 different searches, searching on Google Scholar  for newly released information about the topics   00:48:19.560 --> 00:48:25.440 for those rapid reviews. So it was, you know, sort  of one sentence in the presentation about how   00:48:25.440 --> 00:48:31.440 those are continually being updated, but just  to stress even more sort of every couple weeks   00:48:31.440 --> 00:48:37.620 we are looking for additional information and  as Kristin mentioned those cover dissertations,   00:48:37.620 --> 00:48:43.560 subject matter expertise, blog posts, newspaper  articles the search is really really broad. 00:48:43.560 --> 00:48:51.060 And then we sort of winnow that down as we find  things that are most relevant for that topic area.  00:48:52.140 --> 00:48:58.080 And I can add a little more. As Kristin and Shelley walked us through today, 00:48:59.520 --> 00:49:05.760 some of the topic areas and then the rapid review  synthesis wouldn't state equity, like it wouldn't   00:49:05.760 --> 00:49:12.240 spell out "equity", but then a lot of the rapid  reviews also cover some sections on equity or   00:49:12.780 --> 00:49:19.500 you know specific target population  or programs. So you might want to check   00:49:19.500 --> 00:49:28.080 different reports, different topic areas across  CLEAR, because it's a cross topic subject. 00:49:33.900 --> 00:49:39.900 And then, you know, and DOL while you can  check out the publicly available 00:49:39.900 --> 00:49:47.280 evidence building plan too, the Equity Action Plan that  that was linked in the PowerPoint slide. 00:49:51.780 --> 00:49:55.080 And then I had another question. 00:50:02.340 --> 00:50:04.762 Sorry, I lost the question. 00:50:05.751 --> 00:50:09.428 Chayun I have one about the searching, 00:50:09.428 --> 00:50:18.488 so let me talk a little bit more about that. The question was about sort of if the searches search existing synthesis reports or sort of a larger group of studies? 00:50:18.488 --> 00:50:22.717 So when you search in CLEAR, and Kristin you can add to this of course too, 00:50:22.717 --> 00:50:32.040 when you search in CLEAR, you're generally searching, if you're searching by keywords or if you're filtering or refining by the the ways that we mentioned you can do, 00:50:32.040 --> 00:50:38.460 you're generally finding causal evidence  studies that have been reviewed and those are   00:50:38.460 --> 00:50:45.660 going to go a very very thorough and systematic  review and that's where we list, you know, the   00:50:45.660 --> 00:50:51.660 type of study that it is, whether it's causal or  descriptive or other, you know, what methods they   00:50:51.660 --> 00:50:57.869 used profiles are created... so generally when you  search in CLEAR, those are what you're searching for, 00:50:57.869 --> 00:51:02.520 which is why we, you know, sort of discussed  different ways like the new and trending research   00:51:03.600 --> 00:51:09.540 that you can find that synthesize in the case  of the rapid review for equitable recovery from   00:51:09.540 --> 00:51:17.220 COVID-19 there were 91 publications that were  reviewed and sort of included in that larger   00:51:17.220 --> 00:51:21.480 synthesis document. So when you're searching  in CLEAR specifically you're usually searching   00:51:21.480 --> 00:51:27.900 just individual studies, but if it's a particular  topic area like for older workers there may also   00:51:27.900 --> 00:51:31.922 be a synthesis report that you would find when you search. 00:51:31.922 --> 00:51:35.700 That's a nice segue to the next question. 00:51:36.660 --> 00:51:41.400 One of the audience members asked how  long does it take to make a synthesis report? 00:51:43.620 --> 00:51:49.320 Now that is an excellent question, and I did  not, you know, we didn't track our time. 00:51:50.220 --> 00:51:54.840 It takes a long time. It takes  a really long time. So again for the   00:51:54.840 --> 00:52:01.972 equitable recovery one there were 91 studies that  were included in the document. 00:52:01.972 --> 00:52:07.996 Multiple times that many were reviewed for sort of background information 00:52:07.996 --> 00:52:14.940 reviewed and then ultimately not included, perhaps because they didn't really describe well enough a strategy that could   00:52:14.940 --> 00:52:22.920 be implemented or used. So a lot of iterations  and a lot of decisions and a lot of time goes   00:52:22.920 --> 00:52:29.880 into which ones are included, how they're  included, under which topic area they best fit,   00:52:30.840 --> 00:52:38.820 which strategy, you know, etc. So quite  a bit of time and awful consideration goes   00:52:38.820 --> 00:52:44.400 into the rapid reviews and also again the  individual study summaries and reviews. 00:52:50.820 --> 00:52:58.739 But I think we can also add that we are  targeting six months after the completion 00:52:58.739 --> 00:53:03.660 of the evidence review... sometimes  shorter, sometimes longer, so it depends   00:53:03.660 --> 00:53:12.000 on the evidence and the topic area, like  how much evidence is there? How many reports we're   00:53:12.000 --> 00:53:20.640 synthesizing? And also in like rapid review and  the synthesis we periodically make updates as well. 00:53:24.420 --> 00:53:33.960 And then next question. Someone asked about small business... sorry. 00:53:35.322 --> 00:53:36.684 Let me check... 00:53:41.040 --> 00:53:43.740 Okay, okay there you go.   00:53:44.700 --> 00:53:49.560 So is there anything that would particularly  be useful in assisting small businesses? 00:53:54.480 --> 00:54:02.940 Yeah I'll start so the equitable recovery  rapid review has a section in supports for   00:54:02.940 --> 00:54:08.700 small businesses and again when we were  creating that rap... that   00:54:08.700 --> 00:54:13.560 particular rapid review, most of what was  available at the time was about the PPP or   00:54:13.560 --> 00:54:19.320 the Paycheck Protection Program, but  that section does mention other strategies   00:54:19.320 --> 00:54:27.540 again that either have been shown to be or are  hypothesized to be helpful. So that's one   00:54:27.540 --> 00:54:32.100 place I know that you can search  in CLEAR for information for small businesses. 00:54:34.620 --> 00:54:42.480 We also have like the topic areas on  entrepreneurship or self-employment   00:54:42.480 --> 00:54:49.140 evidence topic area, so that may be of interest  to small businesses. We have synthesis on those as   00:54:49.140 --> 00:54:54.420 well so you can check out these topics, and Kristin, were you gonna say something else? 00:54:55.560 --> 00:55:01.800 I was going to add that you can use, you know, for  a topic like I want information on small   00:55:01.800 --> 00:55:07.200 business, the information shared on this webinar  can really be be applied in terms of searching for   00:55:07.200 --> 00:55:13.560 information on a particular topic. So you can browse, if you go to the CLEAR website, you can   00:55:13.560 --> 00:55:18.495 browse through our topic areas and kind of scroll  through and see if there's relevant information to you.   00:55:18.495 --> 00:55:23.820 You know, if you go to search for studies  and search for keyword "small business"   00:55:23.820 --> 00:55:28.708 and then you want to winnow down using the  filters, you can do that like, you know, 00:55:28.708 --> 00:55:33.480 I'm interested in, you know, I'm in a keyword search  small business but then I'm really interested   00:55:33.480 --> 00:55:40.200 in outcomes related to compliance. So you can  keyword search and then filter down as well   00:55:41.040 --> 00:55:46.620 to get kind of the information you need. Or, you know, I'm interested in small business maybe   00:55:46.620 --> 00:55:51.540 in rural areas so there's the ability  to filter down to find what you need   00:55:52.500 --> 00:55:55.200 in the search for studies  section of the website as well. 00:56:01.320 --> 00:56:09.420 And Shelley, you briefly mentioned about why  you focus on women and BIPOC in the equitable   00:56:09.420 --> 00:56:18.060 recovery review and someone wants to know, you know, like why you didn't mention   00:56:18.060 --> 00:56:24.700 like older workers or people with disabilities  or any other, you know, population. 00:56:24.700 --> 00:56:31.740 Yeah, in this case, in that particular case, we were really  driven by what we were finding, and because   00:56:32.880 --> 00:56:39.120 Black, Indigenous, and People of Color and women  were particularly disproportionately impacted 00:56:39.120 --> 00:56:44.040 that, as I mentioned, that's one of the reasons  that we focused on them. There also happened to be   00:56:44.040 --> 00:56:51.480 a great deal more information about those groups  than there were other groups, such as LGBTQ+   00:56:51.480 --> 00:56:58.320 individuals or older workers and low-income  like sort of other vulnerable populations in   00:56:58.320 --> 00:57:04.860 general, so that's the primary reason is that was  the bulk of the research that we were finding were   00:57:04.860 --> 00:57:12.660 with those groups, but as we've mentioned, we are  continually searching for, and it's possible that   00:57:12.660 --> 00:57:19.440 there may be sort of enough evidence and enough  strategies for particular populations or groups   00:57:19.440 --> 00:57:24.420 that when that review gets updated  we can include additional groups. 00:57:27.780 --> 00:57:34.740 Thank you. Maybe Kristin, you can take this. If someone wants to make a   00:57:34.740 --> 00:57:40.380 recommendation for publication to include  in CLAER, what's the process of sharing? 00:57:42.420 --> 00:57:49.800 So CLEAR has the ability to contact us on the website if you have a suggestion for   00:57:49.800 --> 00:57:58.380 a publication inclusion you can email us through  the contact CLEAR portion of the website and   00:57:58.380 --> 00:58:05.400 what we would do is our team would review it  against our causal evidence guidelines and   00:58:05.400 --> 00:58:12.120 our policies and procedures to make sure it fits  with the collection and and go from there so...   00:58:12.120 --> 00:58:17.820 We do include, as briefly mentioned, kind  of what we call flash reviews, so if something   00:58:17.820 --> 00:58:23.220 comes up and it fits within kind of the scope of  CLEAR we will review it if it's causal through   00:58:23.220 --> 00:58:28.140 our causal evidence guidelines and assess  the strength of the quality of the evidence. 00:58:31.560 --> 00:58:37.680 Thank you, and I think there's a question about  the timeline of when the PowerPoint slides will be   00:58:37.680 --> 00:58:44.880 posted online and we are aiming for early next year. So you should definitely check back CLEAR. 00:58:54.000 --> 00:58:55.500 Okay any other questions? 00:59:14.760 --> 00:59:17.700 Okay I think there may be a question coming in. 00:59:25.380 --> 00:59:35.760 Okay is there a way to export the list of  findings from a search? Like titles and DOI? 00:59:41.160 --> 00:59:48.600 So, within topic area there is a button to  export the results which would go into a CSV file. 00:59:48.600 --> 00:59:55.920 I do not believe it includes DOI, but it would  include the titles and how the profiles   00:59:55.920 --> 01:00:01.634 are tagged. I don't believe... oh yeah, you can also  export under search for studies as well. 01:00:01.634 --> 01:00:03.420 So... 01:00:05.880 --> 01:00:11.400 clicking on it now, so yeah so that'll export  it into a CSV for you and you can filter by topic. 01:00:11.940 --> 01:00:19.740 I am... I don't believe it has DOI, but it  should have the other citation information.  01:00:21.120 --> 01:00:27.180 And I'll add for the rapid reviews the  DOI is included, so as I mentioned that the   01:00:27.180 --> 01:00:34.020 supplement that accompanies every rapid review  has basically the authors, the title, the type   01:00:34.020 --> 01:00:40.320 of study, and then it lists the the website  link which also often is the DOI and then   01:00:40.320 --> 01:00:45.300 it includes a summary as well, and again those  hyperlinks will at least generally take you   01:00:45.300 --> 01:00:51.960 to the study abstract with perhaps a little bit  more information, and then sometimes the full   01:00:51.960 --> 01:00:58.440 text is available because it's not behind any  kind of a paywall, and then sometimes it is so   01:00:58.440 --> 01:01:04.980 you may have to access it, you know, request it  through like an interlibrary loan or log   01:01:04.980 --> 01:01:10.620 in if you have sort of library access to EBSCO or one of the other larger search engines. 01:01:15.420 --> 01:01:20.580 Yeah, so if you go to like a topic area if you're, I don't know if you're on the CLEAR website but   01:01:20.580 --> 01:01:29.340 there's the orange button it says export results and you can download the... 01:01:40.260 --> 01:01:42.480 Yeah, download the results... 01:01:46.620 --> 01:01:54.600 And also as a reminder, you know the goal  of CLEAR is to make research accessible to   01:01:54.600 --> 01:02:04.260 researchers and you know other users.  So we do the summaries for you. 01:02:04.260 --> 01:02:10.740 So we summarize our studies, and you don't  need a library membership if you use CLEAR so... 01:02:25.380 --> 01:02:29.928 Okay I don't think we have any other question... 01:02:30.522 --> 01:02:33.894 Let's wait for a couple minutes and then... 01:02:47.220 --> 01:02:48.840 Okay, Brittni, next slide. 01:02:53.460 --> 01:03:00.420 Okay so if you haven't... if you weren't able to  ask us a question or if you just browse through a   01:03:00.420 --> 01:03:05.880 CLEAR and then come up with a question, feel free  to reach out to us. You can contact us through   01:03:05.880 --> 01:03:15.420 DOL CEO's email address or the CLEAR contact us link  in the PowerPoint slide. So thank you again for   01:03:15.420 --> 01:03:22.380 joining and this concludes today's webinar. The materials for this webinar will be the recording   01:03:22.380 --> 01:03:29.100 PPT and transcript will be available online  early next year so feel free to check it out. 01:03:31.740 --> 01:03:33.825 Thank you. 01:03:33.825 --> 01:03:34.740 Thank you, everyone