Measuring impact – and getting better at it
We started our impact journey in 2009. It is only through looking back that we realised how far we have come. We did not set out to change the world, simply to answer a single question: What difference did we make?
Since then, we have assessed more than 600 programmes worth more than R3 billion. We have assessed development programmes over 15 portfolios – from education to health, enterprise to sports development, and literally everything in between. We have crossed the continent in search of scalable and replicable successful programmes to understand what works (or not) in the development space. In the process, we have identified more than 15 dimensions of impact and return, and developed a library of more than 5 000 quantitative and qualitative indicators. And what have we learned?
- Any resource can be measured – from books and wheelchairs to money, water pumps and schools.
- All programmes have impact – but not all impact is positive.
- In order to have a really sustainable programme, you must consider economic, environmental and social dimensions.
- One size does not fit all – one type of evaluation does not work for everything and, more importantly, there is a big difference between monitoring and evaluation. And monitoring and evaluation are two separate activities with different outcomes, purposes and importance.
- Trying to measure and define “change” is difficult. Understanding, defining, qualifying and quantifying long-term social change is an incremental effort and ongoing process.
- The same project can deliver varied results for different funders.
- How and what you spend the money on (inside the programme) has a direct influence on the impact and return of such a programme.
- The strategy and focus areas must clearly define the return and impact required – upfront. Sustainability must be clearly defined for exit and completion. Indicators must be developed, agreed and documented as part of the contractual phase. Internal monitoring and external evaluation processes must be established and adhered to – impact assessment does not replace evaluation and monitoring.
- You must consider the impact of the impact assessment on many levels. As a result of our work, we can categorically state that most programmes have only short-term impact. Those with medium-term impact are not necessarily sustainable. The long-term impact is mostly social impact, as opposed to economic impact that really contributes to poverty eradication.
- It is possible to determine impact and return on investment of social and community programmes. The real value lies in independent, verifiable assurance of social investment expenditure, programme results, outcomes and impact.
This presentation shows our progress over time and reflects the impact assessments we conducted during 2015. It is our way of sharing our knowledge and contributing to the growing field of practice. We would love to hear from and engage with other practitioners in this space. Communicate with us via rrossouw@nextgeneration.co.za.