What evaluation has to say on serving the needs of decision-makers

by Benoît Gauthier, President, Canadian Evaluation Society

The focus of SIAA’s 2014 Annual Conference is on “measurements that deliver the right message”. The conference program cites examples like writing for funders, communicating to internal decision makers, and telling stories to stakeholders. Ultimately, this conference’s theme reflects a concern with bringing information to life, and making it usable and used.

Utilization is a long-standing preoccupation of evaluators – the community that I represent. The seminal work by Michael Quinn Patton, “Utilization-Focussed Evaluation”, dates back to 1978, soon to be 40 years ago. Four decades of research on evaluation focussed on utilization have led to a number of firm observations.

The key determinants of evaluation utilization comprise: the readiness of the organization, the engagement of decision-makers, the selection of the issues pertinent to decision-makers, the information organized as meaningful stories respectful of the context, quality of communication of results (timing, method, content), and availability of the analyst to support decision-making.

In terms of reporting, our experience as evaluators indicates that a focus on utilization stems from several factors: being intentional about reporting, that is, determining who the audience is and stay true to that purpose, staying user-focused, using graphics and other visuals to communicate succinctly and powerfully, preparing users for counter-intuitive findings, and distinguish dissemination from use.

So, what does that mean in relation to the theme of this conference? The presentation of data is a fundamental ingredient in the recipe for influencing decision-making and knowledge utilization. However, it is only one of several: I would contend that great presentation of poorly targeted evidence is less efficacious than poor presentation of well chosen evidence. But I wholeheartedly agree that optimal utilization of evidence occurs when all the utilization drivers are aligned.

Also, utilization comes in a number of forms, from instrumental use to process use, conceptual use, and other uses. Before determining what the best strategies are to maximize utilization, it is important to clarify what type of use is aimed for. Also, management science tells us that decision makers are not looking for certainty; they strive for a reduction of uncertainty. Managers also need information that is credible in their eyes and actionable in their lives.

Benoît Gauthier will be talking more about what evaluation has to say on serving the needs of decision-makers at SIAA’s 2014 Annual Conference, Talking Data: Measurement with a message, on November 3rd. This blog was originally posted here.