Read the original article published on 23 November 2011 on the Association of Charitable Foundations blog

It’s an interesting question. In a decidedly sceptical article for the Chronicle of Philanthropy earlier this year, William Schambra of the Bradley Center for Philanthropy and Civic Renewal  argued that “the nonprofit world […] has been saturated for more than 100 years with ever more elaborate schemes for ensuring measurable outcomes”. Schambra argued that having to measure and report on outcomes has been a serious drag on the work of non-profit organisations, not least because different metrics fade in and out of fashion, and non-profits must spend an ever larger portion of their time mastering these methods. More damning still, Schambra suggests that there is little evidence that this culture of measurement has done anything to improve the practice of grant-making over the years.

These points were elaborated upon by Larry McGill, Deputy Director of the Foundation Centre, in a more recent article for PhilanTopic, and were subsequently debated on several panels at the ARNOVA conference in Toronto last week.

It is hard to argue with Schambra’s first two points. There is much anecdotal evidence to suggest that charities and non-profits, particularly smaller ones, often face reporting requirements that they find difficult to meet. Many frameworks to capture outcomes – and, over time, impact – are time-consuming, expensive, and disruptive to an organisation’s normal practice. And there are in fact hundreds of such frameworks and methods, some of which have their moment in the limelight only to be superseded by other, newer methods.

However, I would argue that this means we need a joined-up conversation about about measurement and reporting frameworks that are proportional to a grantee organisation’s size and capacity to administer. There is also a question about the reports that funders ask of their grantees, and what purpose these are designed for. Scrapping outcome measurement as a reporting requirement altogether would be throwing out the baby with the bathwater: there is good evidence that grantee organisations often benefit from measuring their outcomes, as long as the systems as proportional, and, crucially, embedded within the non-profits’ work.

Measuring and analysing outcomes can help grantee organisations better understand how they deliver value, which of their activities are most effective and to what extent they are delivering on their core mission. For instance, and as outlined in New Philanthropy Capital’s new report A Journey to Greater Impact, non-profits such as the homelessness charity the Edinburgh Cyrenians use evidence of their outcomes to improve on areas which aren’t working as well as others, while using positive results to make a case for additional funding. In fact, for the Cyreneans, embedding systematic measurement and data collection has actually reduced the organisation’s reporting burden, as they do a small amount of data collection every week rather than spending several days producing quarterly reports for funders.

I am intrigued by Schambra’s argument that years of impact assessment and measurement haven’t added up to better practice in grant-making as a discipline. Is it possible that nothing has been learnt in all these years of collecting data? There is at least anecdotal evidence that by committing to measuring grant recipients’ outcomes and longer term impact – and by extension their own – grant-makers are able to allocate funding more strategically in line with their own goals – the experience of The Diana, Princess of Wales Memorial Fund is one example, and others are set out in work by the Centre for Effective Philanthropy . I would be interested in to know if ACF members have a different opinion about this, or any experiences to share.

In any event, I second McGill’s call for honest discussion of the challenges we face in measuring outcomes and impact in the non-profit sector. The Social Impact Analysts Association (SIAA) aims to do just that. We are a new professional body which connects and supports practitioners worldwide and across different sectors. We want to improve the flow of information about social impact analysis to better understand when it is helpful and when it is not; and, where appropriate, to devise strategies to mitigate its costs and make its benefits more accessible. We don’t profess to have all the answers, but we do want to start this conversation with as many stakeholders as possible, and warmly invite you to join in.

SIAA launches at an inaugural ‘unconference’ on Thursday 1 December in central London. For more information, please visit www.siaassociation.org or contact [email protected].