RQ1. What is the current understanding of the social impact of youth digital projects?
Youth workers’ and young people’s official understanding of social impact is largely influenced by the power dynamics in the evaluations system. Youth workers’ interpretations of the real value of the ‘change’ associated with social impact revealed levels of both confusion and frustration in the digital youth work field in Scotland. The results indicate that despite providing many positive examples and stories of positive social impact, youth workers were equally concerned about the negative and often unreported impacts of digital youth projects.
Positive social impact is considered as the official and reported social impact in evaluation. The negative impact is seen as linked to youth workers’ personal opinions, and therefore less likely to be offered in evaluations.
Youth digital projects participants are more likely to talk about positive and negative social impacts. In their views, both are of equal value for their project development. Nonetheless, the results of this project also indicate that young people perceive social impact as something that does not directly belong to them. According to youth digital projects participants, social impact is something that is externally managed and defined by adults in authority, such as youth workers, funders, teachers, or government bodies.
.png)
RQ2. What are the approaches used to evaluate the social impact of digital youth culture co-creation in Scotland?
This research found that three types of evaluation methods are currently used to collect and analyse the social impact of digital youth culture co-creation:
-
Surveys
The findings of this project indicate that traditional evaluation tools (such as surveys, questionnaires) are the most commonly tools used to evaluate digital youth projects in Scotland. Both groups (youth workers and young people) view questionnaires as a necessary formality guided by pre-agreed indicators. Consequently, there is a sense that currently they have no choice but to use these existing surveys to sustain their funding.
2.Participatory and creative tools
A wide range of creative and innovative methods to evaluate impact was identified and used. The analysis revealed that creative methods can be divided into the following three categories: (1) digital, (2) mixed (using digital and offline methods), and (3) digital. Among some of the most commonly cited in this project were digital quizzes, participatory videos, and photography.
3.Observations and conversations
Whilst least cited in the project, forms of observation and conversations (such as case studies) are also used in the context of Scotland’s digital youth projects evaluation. Youth workers tend not to refer to these activities as evaluation methods, ; however, the analysis of their accounts indicates that youth workers frame their understanding of social impact and youth development while discussing and observing their development. Stories of social impact reported in this project provide some of the richest descriptions of how digital youth projects affect young people’s development and social connectedness.
RQ3. What are the experiences and perceptions
of social impact evaluation among digital youth culture co-creation projects participants and projects facilitators in Scotland?
The results of this project reveal three dominant themes in how youth workers and young people experience evaluation of digital youth projects.
Firstly, both groups are uncertain about the meaning of evaluation. Youth workers are concerned about the lack of clarification of what digital means in the context of a youth project and how digital impact should be evaluated. Young people are concerned about the lack of transparency during the evaluation process. They do not know how their evaluation data is being analysed and if/how it is being used to improve future digital youth projects.
Both groups report feeling pressure to report only the positive impacts of evaluation. Youth workers believe that evidence of positive impact is required to sustain future funding. Young people feel that providing positive examples of social impact is what is expected/required from them during evaluation.
Both groups also feel disempowered during the evaluation process of digital youth project, which they perceive as a control and accountability mechanism imposed by the funders.
RQ4. To what extent could digital youth practitioner-led and youth-led social impact evaluation recommendations alter current evaluation practices?
The analysis of the data suggests that young people’s and youth workers’ insights extend and might improve the current evaluation of youth digital culture
co-creation projects in Scotland. To improve the current evaluation system, project participants propose that evaluation approaches of youth digital projects should be:
-
Accessible
-
Anonymised
-
Digital
-
Encouraging critical thinking
-
Informed
-
Independent of funding
-
Participatory
-
Playful
-
Serendipitous
-
Well-timed