Usability testing policy
This policy covers all usability testing which takes place at the University of St Andrews and applies to all staff who are involved in any usability testing process. The policy applies to in-person and distance testing, as well as for testing on mobile or desktop devices.
Usability testing is open to all to request but please involve the Digital Communications team before starting so they can help you make it a success.
The intention of this policy is to highlight the rules associated with organising, conducting and reporting on usability tests.
-
Organising and conducting usability testing in the University requires good communication between the Digital Communications team and the School, department or unit who wish testing to take place. This means there needs to be at least one key contact in the relevant department with whom Digital Communications can liaise.
Any change to the point of contact must be shared with the Digital Communications team.
-
If a School, department or unit wants to test their web pages or an app, they first need to decide whether they want to conduct the testing themselves or have Digital Communications run the tests.
-
A range of audiences can be used to test a University website or app, such as:
- current students
- current members of staff
- prospective students
- members of the public.
However, you must not test:
- any member of staff who is involved in the project or web pages
- anyone who has seen the testing script before the test
- anyone who has undertaken the same test before.
Within reason, participants should know as little about the tasks and the theme of the test as possible before testing takes place.
-
Incentives are a way to encourage people to test your website or app. Incentives are optional, and whether you need them will depend on the type of people you require for testing. If an incentive is needed, this must be supplied by the School, department or unit who wish to undertake usability testing.
If you are testing a member of staff at the University, you must not provide a monetary-value incentive. This is because staff are already paid by the University and the testing is an extension of their role.
The public and current students can be given an incentive, and this can be determined by any budget you have access to in your School, department or unit. For example, teams have provided vouchers for students or gift bags filled with University merchandise.
-
A usability testing script must be created so all tests are standardised. This will make the data more accurate. When testing, the script must be followed exactly.
Digital Communications will provide a template for the script. The team will also highlight how to create non-leading tasks for participants during the initial meeting and during the usability testing training session. It is the responsibility of the School, department or unit to create the first draft of the testing script, which must then be signed off by Digital Communications.
Scripts must follow the template format.
-
You should record individual usability tests when possible. By recording tests, you will be able to assess any issues later and use recordings as evidence to back up findings in reports and meetings.
However, a participant must not be recorded without their written permission. This includes recording their visual reactions, actions, or their voice. To ensure you have the explicit permission of the participant, they must sign a recording consent form.
The recording consent form is a document owned by Digital Communications, and no other consent form should be used for usability testing. The form is provided to the School, department or unit who want to conduct their own testing prior to any tests taking place.
Storing the recording consent forms
If a School, department or unit wants to conduct its own testing, all recording consent forms must be returned to Digital Communications.
Upon completion of the testing, all recording consent forms must be labelled with the name and date of the testing, scanned as a PDF, and stored in OneDrive by Digital Communications. After all the forms are uploaded to the drive, the physical copies must be shredded.
-
Lookback is the only tool which should be used to record usability tests. Please see the Lookback access policy.
-
If staff wish to conduct their own testing, they must have completed the digital visa training course: ‘Usability testing: understand how your website works’.
The key contact must also have a suitable space in which the testing can take place. Ideally, this will be quiet and have minimal distractions for participants. If testing is conducted in-person, it must be on campus in a University building.
If testing is conducted through moderated remote testing, this must be in a place which has a strong internet connection. However, the tester should not be connected to eduroam.
If part of a recorded test asks a participant log in to a system such as Outlook, MMS or Moodle, for example, the participant must be informed at the beginning of the test that they must log out of all systems at the end of the test.
Hardware used for testing
If conducting in-person testing, the Digital Communications team will provide the following hardware for testing:
- University laptop
- mouse
- microphone.
The items will be provided to the key contact when the School, department or unit is given access to Lookback.
The microphone must only be used to record the participant’s feedback if they have agreed to be recorded.
All participants should be given the option to use a mouse instead of the laptop's trackpad.
-
The level of involvement from Digital Communications in any usability testing will largely depend on whether a School, department or unit wishes to conduct the usability testing themselves or if they need help.
The amount of time the Digital Communications team will spend will be decided during the initial meeting and will be documented in the follow-up summary email. This allocation of time is fixed. -
When Digital Communications conduct testing
If the Digital Communications team are conducting testing on behalf of a School, department or unit, Digital Communications will be responsible for reporting any findings. The School, department or unit may select the method that the findings are reported from a set of options.
An initial debriefing meeting with the main contact will be organised by Digital Communications shortly after the last testing session to discuss the main findings and confirm the next steps.
Staff may wish to also choose one from:- A short report covering the main usability issues and some potential solutions.
- A longer, more comprehensive report covering what the testing was for, what the usability outcomes were, recommendations and individual user feedback analysis.
You may also choose to have an in-person presentation to your School, department or unit covering outcomes, recommendations, and snippets of user experience videos.
Digital Communications aim to report official findings to the key contact within a month of the final usability test. However, this may be subject to change based on the reporting options chosen by the School, department or unit.
When a School, department or unit conducts testing
It will be the responsibility of the School, department or unit to report any usability findings to the relevant people. There are several options to consider for reporting, all of which are covered in greater detail in the training course.
During the reporting process, Digital Communications are happy to consult on any findings and advise on recommendations and next steps. -
The findings of the usability tests may require a number of changes to be made. These changes may be ‘quick fixes’ but they also may be more complex and require significant developer or content editor resource.
If a School has changes that need to be made, the School’s computing officer should do them. If the School does not have a computing officer, they will need to send all small changes to itservicedesk@st-andrews.ac.uk.
The same applies for units in the University. If units have access to developer resource, any reasonable changes should be put forward to them. If units do not have developer resource, they will need to contact itservicedesk@st-andrews.ac.uk.
If the test results show there needs to be a dramatic overhaul of a website, web or mobile app, this will need to be raised with the University’s business transformation board.