Measuring Women's Enterprise Outcomes Without a Baseline
Sakhi Capital is a for-profit social enterprise that on-lends to women micro-entrepreneurs through joint liability groups. It has disbursed loans to 6,000 women over three years at an average ticket size of INR 40,000, with a reported repayment rate of 98%. It intends to raise funds through the Social Stock Exchange and has asked you to prepare its impact assessment. Its impact narrative claims "6,000 lives impacted", "incomes doubled" and "women empowered", citing a 2018 internal study of 50 borrowers and a 92% "satisfaction score" collected by loan officers. There is no baseline survey. The management information system records only loan size, disbursement date, repayment status, group, and a free-text "enterprise type" field. Field staff incentives are tied to disbursement volume. You have eight weeks and INR 12 lakh.
1. Critique the existing impact claims and explain what separates them from genuine impact evidence.
Read the worked answer
Each claim collapses a different rung of the results chain into "impact".
- "6,000 lives impacted" is an output count - loans disbursed. It is not the number of lives changed; some borrowers may have used the loan for consumption, some enterprises may have failed, some women may hold two loans. It is also inflated by counting the same household once per loan cycle if repeat borrowers exist.
- "Incomes doubled" is an outcome claim with no baseline, no definition of income (enterprise net profit? household income?), no time period, and no counterfactual. It rests on an internal study of 50 self-selected borrowers conducted eight years ago and cannot support a present-tense claim.
- "Women empowered" is a construct requiring a defined, measured index - decision-making, mobility, control over loan use, asset ownership - not a satisfaction score.
- The 92% satisfaction score is collected by the same loan officers whose incentives depend on disbursement, which is a structural bias, not a measurement.
Genuine impact evidence needs: a defined indicator, a comparison over time or against a credible comparison group, a stated attribution assumption, and disclosure of who collected the data and how. The absence of any of the first three means the report should reframe all three claims as outputs or aspirations.
2. Design a measurement plan that fits eight weeks and INR 12 lakh. What do you measure, on whom, and against what comparison?
Read the worked answer
Design. A cross-sectional survey with a retrospective baseline and a comparison group, plus a short qualitative strand.
Sample. For a population of 6,000, a 95% confidence interval at ±5% needs roughly 360 completed interviews. Stratify by disbursement cohort (so you have first-, second- and third-year borrowers) and by enterprise type. Add a comparison group of approximately 120–150 women from the waitlist or from groups whose loans were sanctioned but not yet disbursed - imperfect, because of selection, but far better than nothing, and its limitations can be stated.
Indicators. Enterprise net monthly profit (with a documented calculation approach), number of income sources at household level, loan use as actually applied versus as stated at application, a short women's agency index (decision-making on purchases, mobility, control over loan use, contribution to household expenditure), and enterprise survival/continuity.
Method. Ten-minute structured interview on a tablet, with 20–25 semi-structured interviews for mechanism and adverse findings (over-indebtedness, group pressure, repeat borrowing to repay).
Retrospective baseline. Ask recall questions anchored on dated events, and triangulate with the MIS and group records. Disclose recall bias.
Budget and time. Fieldwork four weeks, two weeks cleaning and analysis, two weeks reporting. The survey and comparison group are affordable at this budget; a randomised design is not, and should not be promised.
3. The chief executive insists the report retain the phrase "6,000 lives impacted". How do you handle this?
Read the worked answer
Distinguish the underlying number from the verb attached to it. The 6,000 is a real and verifiable count - of borrowers. "Impacted" is the problem, because it asserts a change you have not measured and cannot verify.
Propose approved wording that keeps the number: "6,000 women accessed credit from Sakhi Capital between 2021 and 2024", followed by "of the 360 borrowers surveyed, X% reported an increase in enterprise profit of at least Y% over the period, and Z% showed improvement on the agency index". This gives the reader the scale and the evidence side by side, and it is more persuasive, not less, because it is checkable.
Explain the risk of keeping the original phrase. On the Social Stock Exchange the impact report is a public disclosure relied on by donors and investors; an unsupported impact claim is a misrepresentation exposure, and it is exactly the kind of claim a diligent reader will attack first, taking the rest of the report down with it.
If the chief executive overrules you, record the disagreement in the engagement file and in the report's basis-of-preparation section. If the final published document still carries the unsupported claim over your objection, withdraw the report rather than let it appear with your assessment attached.