Program Evaluation > Evaluation Reports > Survey Data Validity
This report assesses the extent to which CSBDF’s survey of loan borrowers is generalizable. The validity of survey data hinges on the extent to which the sample of respondents broadly reflects the targeted population. The data suggest the survey respondent pool is representative in nature. There is no statistically significant difference in response rates across most characteristics tested (startup firm status, employment levels, demographics, geographic location). However, respondents tend to disproportionately represent larger businesses and tend to be in a favorable repayment status.
Why This Matters
The use of survey data to measure CDFI impacts is commonplace. But the ability to draw conclusions about survey findings depends on whether the data can be generalized. Analysis of survey response factors is imperfect – the number of variables which might influence response rates are difficult to capture. Interactions between variables that might promote or suppresses responses are even more complex. Still, we believe that the assessment of impact data collection processes is needed to promote a cycle of continuous program evaluation improvement.
Carolina Small Business Development Fund. 2020. Loan Client Survey Data Collection Process: Analysis of Generalizability. http://carolinasmallbusiness.org/evaluation/survey-validity/.