Brightside Head of Finance Rachel Pigott describes how an external analysis of our evaluation data is helping us to achieve our charitable mission to help young people.
Back in 2015 we developed a theory of change that articulated what we are trying to achieve at Brightside. We defined our mission as ‘connecting young people with inspirational role models who can help them to make confident and informed decisions about their next steps in life’ and from Autumn 2016 we have had the frameworks in place to test that we were achieving this. Roll forward to 2018 and we were ready to bring in the statisticians and have an independent audit of our impact. As Head of Finance at Brightside, I personally find it really exciting that we now give our impact data the same attention and external scrutiny that our finances get, something that I believe is long overdue for many impact-focused third sector organisations. This is what we learnt from our audit.
At the end of our programmes, the young people we work with report an improvement in how confident they feel in their decision on what to do after school. We now have independently verified, statistically significant evidence that our programmes are achieving the ultimate outcomes in our theory of change – encouraging stuff.
The evidence shows that Brightside mentoring has a big impact on human and social capital, (in line with our expectations, but reassuring none the less) a smaller but still positive impact on coping and hope (also very promising) but that we currently aren’t influencing self-efficacy or growth mindset. While disappointing to learn we are not consistently hitting every behavioural outcome in our theory of change, this gives us a baseline from which we can redesign our project models in order to better target these outcomes.
The numbers only tell you so much and it’s important to understand the stories behind the numbers before making major programme changes. The impact audit didn’t tell us why we are hitting some outcomes more than others or why some project models are more successful – but it’s given us a road map to guide our qualitative analysis to help us answer these questions and improve our programmes.
In order for our mentoring to have an impact, the young people we work with must engage with our programme and have a high quality experience. These are the factors we can influence directly and therefore both activity and quality data should always form part of any conversation about impact. For us, managing impact is about continually monitoring our programmes to learn what works and using this to improve delivery and achieve the best possible outcomes for young people, not just looking at impact in isolation.
For us at Brightside, robust measurement and evaluation is not a goal in itself but a tool to help us manage and improve our impact on young people. An external audit provides a crucial role in independently verifying impact, but should be used alongside ongoing, more granular measurement. We are building a live performance management system that will allow us to “course correct” our projects based on real time feedback of the young people we are working with. We are committed to this as we believe it is the only way to ensure our mentoring is really helping the young people that need it most make confident and informed decisions about their future.
We are grateful to CAN Invest for their work on the impact audit and their continuing support on our impact journey from 2015.
We are committed to an open and honest approach to impact measurement and if you would like to speak to us about this please get in touch with firstname.lastname@example.org