Selecting the Appropriate Measurement System

by Simon Anderson, Social Asset Measurements

Socially oriented organizations, whether they are non-profits, foundations or social purpose businesses, seek, now more than ever, to articulate how they create social change. Understanding the change that is created is important as it assists staff in understanding their organization’s performance, where adjustments can be made, and enables them to showcase and promote their work to the public.

In order to develop the required assessment framework, an organization must consider its core goals and objectives. This process involves identifying the organization’s key activities and intended areas of influence. In addition, another important stage is to determine what kind of indicators to track, whether they are input, output, or impact oriented indicators. This is an interesting decision because there is not a “one size fits all” approach as the correct framework depends on several internal factors. The following will discuss this decision making process, drawing from Social Asset Measurement’s experience working with organizations across Canada.

Given the current interest in measuring impact, organizations are often drawn to developing an impact assessment framework. While measuring impact ensures that the organization captures the change it is creating, it should be recognized that in order to develop and implement a credible system, an organization needs to devote significant internal resources to the project. This issue highlights the first consideration when developing a measurement system: organizational capacity. While it would be ideal for all organizations to have dedicated staff for developing and implementing an impact assessment framework, in reality, this is not always the case. Therefore, if an organization recognizes that it does not have the resources for impact assessment, an alternative is to develop a framework that captures the outputs from their programs. While this does not provide information on the efficacy of the program, it may be a better option than pursuing an impact assessment that is implemented without adequate resources.

Another key consideration is the structure of the program in question. In some cases an organization may maintain strong connections with their program participants after the program is complete. If this is the structure, then capturing data on outcomes that materialize in the future is possible. However, in other contexts, organizations may lose contact with program participants, and while the infrastructure can be adjusted to facilitate ongoing connection, until that is achieved, it may be difficult to capture outcomes that that arise after the program is completed. If this is the case, indicators which capture change that occurs within the duration of the program should be selected.

The final element I will speak to is that of program type.  This issue affects whether an impact analysis can be extended to a social return on investment (SROI) analysis (where a financial value is applied to some of the indicators to enable the calculation of a benefit cost ratio). For some programs, such as employment training or education promotion, there are well defined financial proxies that can be used to assign a financial value to the indicators. However, in other situations, such as an after school art program, it is more difficult to identify a financial value. In these cases it is better to focus on the impact rather than force the assessment into a SROI analysis because using poor financial proxies taints the credibility of the assessment at large. To this end, when determining whether a SROI analysis is applicable, the nature of an organization’s programs needs to be considered.

It is important for organizations to capture data on their work to better understand their programming and how they create change. In doing so, it is an imperative to develop the most appropriate system for an organization as this ensures the organization has the capacity and required infrastructure to utilize the framework, and, so potential data collection opportunities are not left unused. What are your thoughts regarding the selection of a measurement framework? What needs to be considered? What have been your experiences? It would be great to hear your comments below!

---

 

Simon is the manager of Western Canada at Social Asset Measurements (SAM) where he works with non-profits and foundations to develop and implement impact measurement frameworks. Prior to joining SAM, he worked with Human Resources and Skills Development Canada where he focused on social finance policy issues.

Simon holds a Master of Public Policy (University of Toronto) and B.A. in International Development and Economics (University of King’s College).