12. Measuring the difference
Commentary
If we want to measure young people’s progress, it is important to have a starting point or baseline. This may take the form of achievement tests. It may be records of achievement or examples of young people’s work. It may also be evidence of attitudes, collected by means of questionnaires or written open-ended statements made by the young people involved. Parents' views may also be collected.
Before collecting any baseline data, however, it is vital for people to know why information is being collected and how it will be used. Thought should be given beforehand to how it will be analysed and who will take on that responsibility. If it is to be a useful exercise it is vital that it is carried out in a climate of trust and it should not do anything which would undermine relationships or the ethos of the school.
Questions to ask
- Which outcomes do you wish to measure? (ie changes in behaviour, attendance, attitudes, attainment)
- What baseline evidence will be needed? How can this be obtained? Who will be responsible?
- What evidence is there of the quality of OSHL provision at different times? How does this inform future developments?
- How are the purposes and the results of the evidence collection discussed and shared?
- How reliable is the evidence that is available?
- How is evidence used and how are results analysed to inform learning in the future?
- How might the skills of staff and students in using evaluation be developed?
- How are results used to inform whole school planning and development?
Evidence to show
Examples of documentary evidence
- surveys/questionnaires which explore young people's or tutors’ skills, qualifications, roles and attitudes at a given point
- surveys/questionnaires which explore tutors', staff's or young people’s attitudes to improvement
- existing school data systems, eg for attendance, attainment, attitudes to study, self, school
- photographs or videos showing development over time
- new instruments customised and developed by staff or young people, eg self-evaluation, key skills maps, target setting, suggestion boxes
- attainment test results
- reports analysing findings from data.
Testing the evidence
Examples of requests a visitor might make
- Tell me what kind of information you use to measure young people’s progress.
- How useful is baseline data and how might it be improved upon in the future?
- Explain to me how data was gathered and what steps you took to ensure that the conditions were fair and equitable for all. Describe how the data was analysed.
- What alternative types of evidence have you gathered or might you gather in the future?
- In what ways is the data used as a diagnostic or developmental tool?
- How are you made aware of the findings of the data?
Emerging practice
Emerging centres recognise the importance of evaluation and of collecting simple baseline data. They create a supportive climate for the gathering of data. They keep staff informed of its value and purposes and explain it to young people. They use the information gained to develop more effective programmes.
Key indicator
A supportive climate for effective data gathering.
Case study
Measuring the difference - Emerging
Established practice
In established centres evaluation is highly valued and a strong emphasis is placed on informing practice and robust baseline data. In school-based centres, the collection of baseline data for OSHL is seen as part of a whole school approach to the evaluation of learning and teaching, and is incorporated into the school development plan.
Key indicator
A whole school approach to gathering and using data.
Case study
Measuring the difference - Established
Advanced practice
In advanced centres staff, parents and young people see the exploration of data as a significant learning opportunity. They are involved in planning data-gathering, monitoring and reviewing tools, and discussing implications. They find opportunities to customise tools and instruments which truly reflect their own purposes and priorities.
Key indicator
Using data for more effective learning.