Virtual Advisory Service (Pilot) Phase One Evaluation

Introduction

The Virtual Advisory Service (VAS) replaces the traditional model of subject advisers embedded within each local authority in Scotland. Between January and April 2007, a pilot study into the VAS was conducted in four local authority areas, spanning early years, primary and secondary education. Specific subjects covered included science, literacy and numeracy. Based on the results of this second stage of research, a decision on whether to fully implement the VAS will be made.

George Street Research was commissioned to evaluate the initial pilot study on behalf of Learning and Teaching Scotland, from February to July 2007.

Evaluation aim

The aim of the research was to evaluate the initial VAS pilot study to assess usage of the service, its impact and levels of satisfaction, in order to inform future developments.

Research method

The research involved two main phases.

Phase one involved a quantitative telephone survey of users of the service, undertaken just after the launch of the VAS. At the same time, qualitative telephone interviews were undertaken with those involved in setting up or providing the service.

Phase two was conducted later in the pilot study and involved another set of telephone interviews with VAS users, along with a further set of interviews with those responsible for delivering the service.

As well as these interviews, six focus groups were held with participants in the pilot, to collect more in-depth information on aspects of the Virtual Advisory Service.

Key findings

Usage

• Patterns of usage of the Virtual Advisory Service varied and the majority of respondents logged on outwith the school day. Highest usage was for browsing enquiries (96%), reading a discussion forum (82%) and Who’s Online (71%).
• A large majority of respondents were positive about the VAS in terms of its attractiveness, considering it well laid out, easily navigated and easy to use.
• The majority of enquiries (80%) were answered within the target timescale of 72 hours.
• When assessing a range of factors in relation to responses to enquiries, respondents were very positive, with over 9 in 10 rating each dimension as very or quite good (overall quality of response, applicability of information provided, clarity and depth of information, and speed of response).

Impact

• Key strengths of the VAS were 24-hour accessibility, quick responses, obtaining expert advice and the opportunity to ask detailed questions.
• A majority (77%) of those who had browsed enquiries found the enquiries relevant or useful to them in their work.
• A majority of respondents (86%) claimed they would be likely to recommend the service to a friend.

Future usage and potential impact

• Significant proportions expect usage to increase in the future (43%) or to remain the same (45%) and 75% claimed to be likely to make enquiries in the future.
• Views were largely positive about the future impact of VAS, with 48% considering it would provide useful support, 38% that it was not essential but was beneficial and 10% that it would provide essential support for practitioners.
• VAS was seen to work well alongside Curriculum for Excellence as a source of support and advice.

Challenges and future options

• There is a need to increase awareness of VAS through promotion, a network of established users or increased involvement of local authority co-ordinators.
• Potential users need to be made aware of how to access the service and submit queries.
• There may be a future need for input from facilitators, in order to ensure the VAS remains up to date.
• Speed of response should be maintained at 72 hours where possible.
• Since more users of the VAS browse content rather than submit enquiries, there is a need to ensure that all users can browse the service at leisure.
• It is expected that the Virtual Advisory Service will sit under the umbrella of Glow, but there is a preference for it to retain its own brand and identity.