Conventional monitoring and evaluation

In ‘conventional’ monitoring and evaluation, we are used to measuring progress towards predetermined outcomes and ‘verify’ if a programme’s objectives were reached. After having developed a set of core-indicators for each of the outcomes, a baseline (‘snapshot’ of the situation at programme’s start) is typically established to create a benchmark against which the programme’s outcomes can be assessed. It is a great way of determining if and to what extent the objectives have been reached.

Complex programmes require a different approach

In complex environments, however, programmes often lead to a non-linear stream of outcomes which are difficult to predict. The results chain between activities and outcomes if typically unclear, multiple actors are involved in creating change, and activities often contribute only indirectly or partially to one or more outcomes. Think about lobbying and advocacy programmes for example. Assessing the results of such complex programmes through a linear, cause-effect approach in which progress towards predetermined outcomes is measured is not only challenging, but also unlikely to provide insight in a programme’s true outcomes. A different approach is therefore needed.

Outcome harvesting

Unlike conventional methods, Outcome Harvesting collects evidence of what has been achieved. Evaluators first look for demonstrated, observable changes (‘outcomes’) and then work backwards to determine if and how the programme contributed to those outcomes. Assessment is not limited to predefined results and change patterns. This way, both predicted and unpredicted, positive and negative, direct and indirect outcomes can be captured. A set of principles and techniques is used to answer questions such as:

• What changed?
• How did the change happen? Who / what contributed to the change?
• How do we know this? What evidence do we have?
• What is the importance / significance of the change? Why?

Six steps

Outcome Harvesting consists of six iterative steps:

1. Design the outcome harvest

Research questions to guide the harvest are identified, the boundaries for the outcome harvest are defined and key-informants are selected.

2. Gather data and draft outcome descriptions

The evaluators look for changes (outcomes) that have occurred and how the programme contributed to those changes. Information is gathered by reviewing reports, evaluations, press releases or through interviews, surveys and any other relevant sources. The evaluators then write preliminary outcome descriptions to be reviewed.

3. Engage change agents in formulating outcome descriptions

‘Change agents’ (people working for the programme) are engaged to review outcome descriptions and identify additional outcomes. This results in a more complete and specific overview of relevant outcomes.

4. Substantiate

Independent individuals (outside the programme) knowledgeable about the outcomes and the way they were achieved are invited to reflect on the outcome descriptions. It is an important step in validating and enhancing the harvest’s findings.

5. Analyse and interpret

The outcomes are organised in such a way that they can be analysed, interpreted and used to provide evidence-based questions to the research questions. The way the analysis is conducted if different in every outcome harvest.

6. Support use of findings

Translate the findings in evidence-based, useful and actionable answers.  

Why Outcome Harvesting?

Compared to conventional approaches, an outcome harvest provides a much broader and deeper understanding of a programme’s outcomes. It does not only help understand patterns of change in complex environments where relations of cause and effect are not fully understood, but also reveals a programme’s contribution to those changes. It is useful when outcomes are not (yet) specific enough or measurable when an intervention or programme is designed, making it well-suited for evaluations in dynamic and complex (often uncertain) environments.

Can we support you?

At Sinzer, we strongly believe in the potential of innovative approaches such as Outcome Harvesting. Would you like to learn more about it? Feel free to get in touch with Vincent Belgraver (impact consultant) through vincent@sinzer.org / + 31 6 1484 3954.

Vincent Belgraver has been trained in Outcome Harvesting at the University of Bologna by Ricardo Wilson-Grau, the ‘founding father’ of this evaluation method.