Skip to navigation | Skip to content

DCSF

RedEye optimum.web effectively test usability for DCSF Diplomas website

Background

In September 2007 The Department for Children Schools and Families (DCSF) launched a new programme offering young people aged 14-19 years, from selected schools and colleges, the opportunity to study for a new ‘Diplomas’ qualification.  The teaching of the first five Diplomas (Construction and the Built Environment, IT, Creative and Media, Health and Development, Society and Engineering) started in September 2008.

Before teaching began DCSF needed to increase awareness and understanding of the new Diplomas and stimulate demand from young people.  This was (and continues to be) delivered by an integrated communications campaign managed by the Central Office of Information (COI).

As part of this campaign a new Diplomas website has been developed as a main source of communication regarding information on the programme for young people.  To ensure the initial Diplomas website launched in September 2007 was effective, and met the needs of the target audience, COI recommended usability testing.  

The solution

Usability testing focused on The Diplomas website’s principal target audience (young people aged 13-16 years).

The purpose of the usability testing was to rigorously assess how well the new website was meeting its objectives from a communications and user experience perspective.

RE-OW recommended 20 1:1 user testing sessions of 1.25 hours each, be carried out across two venues, London and Leeds. This would provide an exceptionally robust sample for usability testing and would allow any potential regional findings variances to emerge. Recruitment was undertaken by RE-OW’s professional recruitment specialists, who work with strict adherence to the Market Research Society Code of Practice. A detailed recruitment brief was developed by RE-OW and approved by the client as follows:

  • 5 users in each  4 school year bandings (years 8, 9, 10 and 11)
  • Half (10) to be interested in working in one of the five Diploma subject areas (2 per subject)
  • Gender split to be equal across the sample and with a 2 : 3 split within each school year band
  • 12 users to come from C2DE backgrounds; 8 from ABCI
  • Total sample to represent a good mix of key British ethnic groups (white, Asian, Afro-Caribbean)
  • 12 sessions to take place in London; 8 to take place in Leeds; and all sessions to start after the end of the school day
  • All users to be chaperoned by an adult; all to be provided with appropriate incentives and the Market Research Society Code of Practice to apply

The 1:1 (moderator:user) testing sessions were undertaken using RE-OW’s optimum improvement methodology.  This involved the use of a specially-tailored testing script and set of pre-agreed information-gathering tasks. A contextual enquiry approach (verbal protocol) was also put in place to identify the following;

  • Any issues / problems experienced when using The Diploma website (e.g. content and functionality gaps, clarity of language, possible enhancements)
  • Users’ overall understanding and impressions of The Diplomas concept
  • Users overall impressions of the website

The sessions were led by a senior HCI (Human Computer Interaction) consultant with strong experience of moderating sessions with users of different ages and ethnic backgrounds. All sessions were audio and video recorded for analysis purposes and were undertaken in a usability laboratory environment that provided remote viewing facilities.  In addition, a support moderator undertook real–time key quotes capture and key issues analysis.  Once all 20 sessions were completed an Evidence Analysis meeting involving both moderators provided the foundation on which RE-OW then based its detailed report and recommendations.

The Results

The usability testing sessions clearly identified the site’s key strengths – not least of which was the use of video (this proving the most engaging and successful communications medium for the site’s principal target audience).

At the same time, the results highlighted a number of issues where enhancements and / or changes would improve significantly both the overall user experience and, critically, key targets’ understanding of The Diplomas concept.

The website continues to grow with the campaign and key learnings from the usability testing have been incorporated to the site’s benefit.


To find out more call 020 7730 9958 or submit an online enquiry.