In this post CSBDF’s Jason Sabatelle (Volunteer Senior Research Fellow), Adonis Caramintzos (Volunteer Research Fellow), and Emily Rivera (Loan Servicer) join Vice President of Policy & Research Jamie McCall to show why both economic and social impact measurement is important (….and hard to do). For more details about the findings of this research, please see the 2-page brief and full report.
Throughout the pandemic, many programs have emerged to help small businesses across the country. Many of these efforts - including distributing both loans and grants - have been led by community development financial institutions (CDFIs) like CSBDF.[1] For us, it has been a privilege to provide help to the main engine of the state’s economy over the past 18 months. And while the economic recovery from COVID-19 has started, we think much more help is needed to ensure North Carolina’s small businesses have the tools they need to succeed in the long-term.
For three decades CSBDF has been here to help small businesses. But to do that well, we must understand how what we do affects both entrepreneurs and the communities they live in. In that spirit we examined our impact amidst the pandemic, finding that CSBDF’s lending helped small businesses be more financially stable. We are now pulling back to explore our impact in a different context and over a much longer time.
Estimating Our Economic Impact is Important…
Time and again, we have been inspired by the ability of the state’s small businesses to prosper despite what seems like overwhelming challenges. The stories we hear from entrepreneurs who receive assistance from CSBDF put a face on the data we publish about our economic impact. CSBDF’s ability to create positive financial ripples is important and is indeed part of our story.
Yet we have also acknowledged that measuring our economic impact is a challenge. We issue millions of dollars in affordable loans each year, but North Carolina has an economy totaling $600 billion. In such large and complex systems,[2] trying to identify how 1, 50, or even 100 loans move the needle on economic outcomes is not an easy task.[3] How do we know whether it's our loan or another variable (or both – or neither!) that is causing observed positive economic effects?[4]
…But Only Part of the Story
If putting a number behind our economic impact is hard, trying to show our impact on community social systems is even harder. How do you quantify small businesses giving a neighborhood a sense of place and character?[5] How do you put a dollar value on the relationships entrepreneurs build with each other and their community over time? How do you measure the social worth of main street firms as pillars of their community?[6]
As part of our ongoing commitment to impact measurement and program evaluation, we’re excited to share a new exploratory analysis on how CSBDF’s work creates positive social impacts across a 10 year period. This research looks at how affordable financing activities might shape community-level social capital. We wanted to know: is there a correlation between CSBDF’s lending and increasing levels of trust, reciprocity, and social connections at the community level?
In short, the answer is: yes (with caveats, of course!). Using a CSBDF dataset which covers 1,068 lending transactions, we find that lending activity does appear to be occurring in tandem with an increase in community-level social capital. Correlation is not causation, but a pattern of correlation over many loans and a long time period does hint at a positive social impact. The 2-page brief and report give more detail, but a few high-level findings stand out:
First, we looked to see if the demographics and characteristics of the firms we serve were correlated with county social capital levels. The data show CSBDF’s lending to businesses owned by racial minorities has a positive association with community social capital. In fact, that is the only variable we tested at the firm-level which seemed to have a positive association. Importantly, that does not mean other variables have a negative association (or even no association). It just means we cannot distinguish whether other variables have effects which are not due to statistical noise.
Second, though we included traditional measures of economic impact like job creation and retention, neither measure had a significant relationship with social capital. We think this is interesting because employment is one of the primary ways CDFIs talk about their impact.[7] That includes CSBDF, which engages in extensive efforts to collect accurate jobs information from our borrowers. Our research makes the case for why CDFI impacts should be reimagined to have a focus on both economic and social outcomes.
Slight Technical Detour - Why This is Hard
Though we think our findings are positive, they also illustrate some of the inherent complexity involved in trying to measure CDFI impacts. For example, you may notice the table above says income inequality has a positive relationship with social capital and racial diversity has a negative relationship. How could it be that higher income inequality and less diverse communities lead to more trust, reciprocity, and social networks?
The good news is that isn’t what is actually happening. But it is easy to interpret the data that way - showing why this kind of research is hard. There are multiple types of social capital,[8] and the index measure used in our analysis does not capture all those categories (no measure could). The index used in our research tends to be a good measure of bridging social capital, which are networks of trust involving people/organizations with heterogeneous (different) social, economic, and/or demographic characteristics.
The index does not do as well at capturing bonding capital, which represents trust across people/organizations with homogeneous (similar) social, economic, and/or demographic characteristics. The debate about how these different types of social capital relate to positive community outcomes is extensive and ongoing.[9] But we believe our results support research showing how bonding capital – especially in the distressed/marginalized communities[10] – helps promote an array of desirable socioeconomic outcomes.[11]
Economic and Social Impact Measurement is Key
The best things in life are never easy, and so it goes with trying to measure the social capital impact of CDFIs. Though the analysis is exploratory, we believe it shows that across the past decade CSBDF loan interventions are “moving the needle” in terms of promoting positive social outcomes at the community level across North Carolina.
[1] Eric Hangen and Michael Swack, “CDFIs Can Make the SBA PPP Loan Program Work for Smaller, Minority-Owned, and Women-Owned Small Businesses,” Research Brief (Durham, NH: University of New Hampshire Carsey School of Public Policy, April 7, 2020), https://carsey.unh.edu/publication/CDFIs-can-make-SBA-PPP-Loan-Program-Work. [2] Mary L. Ohmer, “Assessing and Developing the Evidence Base of Macro Practice Interventions with a Community and Neighborhood Focus,” Journal of Evidence-Based Social Work 5, no. 3–4 (September 5, 2008): 519–47, https://doi.org/10.1080/15433710802084284. [3] L. Owen Kirkpatrick, “The Two ‘Logics’ of Community Development: Neighborhoods, Markets, and Community Development Corporations,” Politics & Society 35, no. 2 (June 1, 2007): 329–59, https://doi.org/10.1177/0032329207300395. [4] Timothy Bartik and Richard Bingham, “Can Economic Development Programs Be Evaluated?,” in Dilemmas of Urban Economic Development: Issues in Theory and Practice, ed. Richard Bingham and Robert Mier, vol. 47, Urban Affairs Annual Reviews (Thousand Oaks, CA: Sage, 1997), 246–90; Roger R. Stough, “Endogenous Growth Theory and the Role of Institutions in Regional Economic Development,” in Theories of Endogenous Regional Growth: Lessons for Regional Policies, ed. Börje Johansson, Charlie Karlsson, and Roger R. Stough, Advances in Spatial Science (Berlin, Heidelberg: Springer, 2001), 17–48, https://doi.org/10.1007/978-3-642-59570-7_2; Michael Woolcock, “When Do Development Projects Enhance Community Well-Being?,” International Journal of Community Well-Being 2, no. 2 (July 1, 2019): 81–89, https://doi.org/10.1007/s42413-019-00031-z. [5] Matthew Freedman, “Place-Based Programs and the Geographic Dispersion of Employment,” Regional Science and Urban Economics 53 (July 1, 2015): 1–19, https://doi.org/10.1016/j.regsciurbeco.2015.04.002. [6] R. Allen Hays and Alexandra M. Kogl, “Neighborhood Attachment, Social Capital Building, and Political Participation: A Case Study of Low- and Moderate-Income Residents of Waterloo, Iowa,” Journal of Urban Affairs 29, no. 2 (May 1, 2007): 181–205, https://doi.org/10.1111/j.1467-9906.2007.00333.x. [7] Coastal Enterprises Inc., “Measuring Impact in Practice: Reflections from Coastal Enterprises, Inc.’s Experience,” February 2006, http://www.ceimaine.org/wp-content/uploads/2014/02/Measuring-Impact.pdf; Giuseppe Gramigna, “Evaluating SME Policies and Programmes—Micro-Level Datasets, Analytical Toolkits and Institutional Factors,” Journal of Entrepreneurship and Innovation in Emerging Economies 3, no. 2 (July 2017): 134–42, https://doi.org/10.1177/2393957517721845; James L. Greer and Oscar Gonzales, Community Economic Development in the United States: The CDFI Industry and America’s Distressed Communities (New York City, NY: Springer, 2016); Pacific Community Ventures, “Moving Beyond Job Creation: Defining and Measuring the Creation of Quality Jobs,” April 2016, https://www.pacificcommunityventures.org/2016/04/14/defining-and-measuring-the-creation-of-quality-jobs/. [8] Maureen Berner et al., “Building Bonds and Bridges (and Leveraging Links): A Place-Based Mobility Strategy Based on Social Capital Creation” (Federal Reserve Community Development Research Conference, Washington, DC, 2019), https://doi.org/10.46712/building-bridges-bonds; Laurie E. Paarlberg, Michele Hoyman, and Jamie McCall, “Heterogeneity, Income Inequality, and Social Capital: A New Perspective,” Social Science Quarterly 99, no. 2 (2018): 699–710, https://doi.org/10.1111/ssqu.12454. [9] Hilde Coffé and Benny Geys, “Toward an Empirical Characterization of Bridging and Bonding Social Capital,” Nonprofit and Voluntary Sector Quarterly 36, no. 1 (2007): 121–39; Jessica A. Crowe, “In Search of a Happy Medium: How the Structure of Interorganizational Networks Influence Community Economic Development Strategies,” Social Networks 29, no. 4 (2007): 469–88; Roger V. Patulny and Gunnar Svendsen, “Exploring the Social Capital Grid: Bonding, Bridging, Qualitative, Quantitative,” International Journal of Sociology and Social Policy 27, no. 1/2 (2007): 32–51; Andrew Woodhouse, “Social Capital and Economic Development in Regional Australia: A Case Study,” Journal of Rural Studies 22, no. 1 (2006): 83–94; Michael Woolcock, “The Rise and Routinization of Social Capital, 1988–2008,” Annual Review of Political Science 13 (2010): 469–87. [10] Joyce Mandell, “Picnics, Participation and Power: Linking Community Building to Social Change,” Community Development 41, no. 2 (2010): 269–82. [11] Shahzad Ansari, Kamal Munir, and Tricia Gregg, “Impact at the ‘Bottom of the Pyramid’: The Role of Social Capital in Capability Development and Community Empowerment,” Journal of Management Studies 49, no. 4 (2012): 813–42; Daniel Brisson and Charles Usher, “The Effects of Informal Neighborhood Bonding Social Capital and Neighborhood Context on Homeownership for Families Living in Poverty,” Journal of Urban Affairs 29, no. 1 (2007): 65–75; Mark R. Warren, J. Phillip Thompson, and Susan Saegert, “The Role of Social Capital in Combating Poverty,” in Social Capital and Poor Communities, ed. Susan Saegert and J. Phillip Thompson (New York City, NY: Russel Sage Foundation Press, 2001), 1–28.
Comments