This study was conducted in part by staff from Mathematica Policy Research, which administers CLEAR. Therefore, the review of this study was conducted by an independent consultant trained in applying the CLEAR causal evidence guidelines.
Citation
Benus, J., McConnell, S., Bellotti, J., Shen, T., Fortson, K., & Kahvecioglu, D. (2008). Growing America Through Entrepreneurship: Findings from the evaluation of Project GATE. Columbia, MD: IMPAQ International.
Highlights
- The study’s objective was to examine the impacts of the Growing America Through Entrepreneurship (Project GATE) demonstration project, which offered entrepreneurship training and business counseling to low-income individuals, on participants’ employment, earnings, and public assistance benefit receipt.
- The authors randomly assigned study participants to a treatment group, which could participate in Project GATE, or a control group, which could not participate. The authors used Unemployment Insurance (UI) wage records and survey data in their analyses.
- The study did not find any statistically significant effects of Project GATE on employment, earnings, or UI benefit receipt in any of the four quarters after random assignment, using UI wage record data.
- The quality of causal evidence presented in this report is high for outcomes measured using UI wage record data because they were based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects would be attributable to Project GATE and not to other factors. However, the study did not find statistically significant effects. Outcomes measured with survey data received a low evidence rating because there is high attrition and insufficient adjustment for differences in previous employment and earnings history.
Intervention Examined
The Project Growing America Through Entrepreneurship (GATE) Demonstration
Features of the Intervention
Project GATE was a demonstration project that operated in Philadelphia and Pittsburgh, Pennsylvania; Minneapolis–St. Paul and northeastern Minnesota; and three cities in Maine from fall 2003 to summer 2005. It was open to anyone older than 18 who resided in one of the participating sites and had a business idea. Applicants had to provide information on their personal characteristics and business ideas. Very few people were denied participation based on their business ideas or their qualifications for starting a business (three applicants whose business ideas were not appropriate for federal funding were deemed ineligible).
As part of the demonstration, the authors randomly assigned 2,095 Project GATE applicants to a treatment group that could participate in Project GATE and 2,103 applicants to the control group that could not participate but could access other employment services available in the community. Members of the treatment group underwent an assessment interview with a Project GATE assessment counselor and together they selected the most appropriate services to meet the participant’s self-employment needs. The counselor referred the participant to an appropriate service provider—a One-Stop Career Center, nonprofit community-based organization, or small business development center—that offered entrepreneurship training, one-on-one business counseling, and help applying for business loans.
Study participants were 42 years old on average. Slightly more than half were white and non-Hispanic, and about 30 percent were black and non-Hispanic. On average, sample members had completed two years of postsecondary education. Slightly less than half had received UI in the past year, and more than one-third received it when they applied. Slightly more than one-third had been self-employed in the past, and about one-fifth were self-employed when they applied.
The authors obtained UI wage record data for all participants for four quarters following random assignment to measure employment, earnings, and UI receipt. The authors also conducted two follow-up surveys (6 and 18 months following random assignment) to measure employment and earnings, from both self-employment and wage and salary employment, as well as public benefit receipt. The authors compared the outcomes of treatment and control group members in each quarter following random assignment.
Findings
- Employment. The study found no statistically significant effects of Project GATE on employment during the first four quarters after random assignment using data from UI wage records.
- Earnings. The study found no statistically significant effects of Project GATE on earnings in any of the four quarters after random assignment, based on UI wage record data.
- Public benefit receipt. The study found no statistically significant effects of Project Gate on UI benefit receipt in any of the four quarters after random assignment, based on UI wage record data.
Considerations for Interpreting the Findings
Because both follow-up surveys had high attrition, analyses involving these follow-ups were reviewed as nonexperimental designs. In their analysis of the survey data, the authors did not account for existing differences between the groups before random assignment. These existing differences between the groups—and not Project GATE—could explain the observed differences in outcomes. Therefore, all outcomes using survey data in this study receive low causal evidence ratings.
Causal Evidence Rating
The quality of causal evidence presented in this report is high for outcomes measured with UI wage record data, because these outcomes were based on a well-implemented randomized controlled trial. This means we would be confident that any estimated effects for these outcomes are attributable to Project GATE, and not to other factors. However, the study did not find statistically significant effects. Outcomes measured with survey data received a low evidence rating because there was high attrition and insufficient adjustments for previous employment and earnings history. This means we would not be confident that any estimated effects are attributable to Project GATE and not to other factors.