Title: Measuring Outcomes in Youth Mentoring Programmes: What Should Be Measured and How?
Facilitator and room number: Heide Busse; 1.201
Documented by: Alena Lohnert
Number of participants: 18
Who are you and to which institution do you belong to?
My name is Heide Busse and I am a PhD student based in the Faculty of Health Sciences at the University of Bristol, United Kingdom. I am working within the Public Health team at the University of Bristol and am part of the Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), who specialise in evaluating interventions such as mentoring.
My research focussed on the role of mentoring for young people in the United Kingdom and I am particularly interested to see how the effectiveness of mentoring programmes can be best measured and in the effectiveness of mentoring with regard to the health and wellbeing of young people.
Starting hypothesis:
What is measured in each mentoring programme varies widely as well as their method of measuring. A general discussion of “what should be measured and how” may show commonalities and differences in interest and aim of outcome measurement between the mentoring programmes and may reveal capability and deficits in recent practice.
Sequence of content/methods:
After a short introduction of Heide Busse about the need for outcomes to be measured, the group discussed in two rounds: (1) what outcomes should be measured and (2) how could the measurement be realised?
Main arguments presented by the facilitator:
Measuring the outcomes is essential to each mentoring programme. For practitioners, outcome measurement is a control for quality and effectiveness of the programme. For researchers, outcomes show in which way and for whom an intervention method is working. Finally, the “proof” that the mentoring programme is effective is important to justify funding to the commissioners.
The interest in specific outcomes generally differs between practitioners and researchers. Whereas practitioners often prefer short-term outcome measurement focusing on “soft” outcomes (personal changes, experiences), researchers favour long-term outcome measurements concentrating on “hard” outcomes. This also leads to the use of different measurement methods: qualitative measurement (practitioners) and quantitative measurement (researchers).
Two main points of the discussion:
- The participating mentoring programmes are mainly interested in developmental and educational outcomes for the mentee. Concerning the developmental outcomes, participants named more specifically the measurement of changes in regards to self-esteem, self-efficacy, self-confidence, social capital, satisfaction, health, emotional stability and resilience. Concerning educational outcomes, the programme focus on language skills, school grades, communication and problem solving skills.
- The used or imagined measuring methods ranged from project diaries, individual and group interviews, supervision through phone calls or emails to online surveys and app programmes.
Results of the Session:
- Until now, outcome measurement mainly focuses on the mentee of the mentoring programme. However, in the future, there should be a bigger focus on the effects of mentoring relationships for the mentors as well as for the society.
- Quantification of outcomes is really hard and takes a lot of time. Every mentoring programme focuses on different outcomes and thus uses different measuring methods, which make their results hardly comparable in research. Furthermore, there is a general lack of competence in quality measurement. If there were training programmes for outcome measurement, almost every present practitioner would like to participate.
Three central statements highlighting the result/s of the discussion:
Practitioners and researcher have different considerations of outcome measurement.
Outcome measurement is really hard to conceptualise and realise for practitioners.