Ergonomic evaluation

The ergonomic evaluation consists in reviewing each component of the interface in order to identify usability problems.

Advantages

  • The main advantage of the ergonomic evaluation is its level of detail. It provides a deep analysis of each component of the software.
  • This method is a good means to identify the main usability problems. It does not replace the user test. But it allows to conduct a more efficient test, because better prepared.

Drawbacks

  • The ergonomic evaluation presents the disadvantage of not being done in real situation.
  • It is based only on the judgement of the evaluator which is not always closed to the one of the user.

Protocol

The ergonomic evaluation consists in reviewing each component of the user interface in order to check whether they respect a set of ergonomic criteria.

Schemata: Principle of the evaluation by inspection
The ergonomic evaluation consists in evaluating each component of the software or of the web site with respect to a checklist of ergonomic criteria.

Several checklists are available in order to evaluate software usability:

  • The Index of Usability of Purdue University. Gary Perlman proposes a web version of this questionnaire.
  • Dominique Scapin's ergonomical guide. The complete guide is available on Interaction website.
  • A checklist based on ISO 9241 standard. A demo version of the questionnaire, called IsoMetrics, is proposed on Osnabrück University site.

However, it is sometimes useful to build his own checklist by taking into account the specific features of the application and the user profile. For instance, specific checklists have been developed for Air Traffic Control systems: ATC Human Factors Checklist.

The evaluation can also be based on the only expertise of the evaluator and on his knowledge of similar systems. It is called heuristic evaluation. In this case the applied ergonomic criteria are not explicitly stated.

Example of evaluation checklists

The following checklist, inspired from [Nielsen 93], presents the advantage of being very short. It could be employed easily and allows to quickly identify the main problems:

  • Is the dialogue simple?
  • Does the interface speak the users' language?
  • Does the interface minimize the users' memory load?
  • Is the interface consistent?
  • Is feedback visible?
  • Are the exits clearly marked?
  • Do shortcuts exist?
  • Are the error messages explicit?
  • Are there means to prevent the errors?
  • Does an online help exist?
  • Is the software documented?

In the same way, the following checklist, from IBM Web Design guidelines, allows to identify the main usability problems of a web site:

  • Is the purpose of the site clear?
  • Does the site clearly address a particular audience?
  • Is the site useful and relevant to its audience?
  • Is the site interesting and engaging?
  • Does the site enables users to accomplish all the tasks they want or need to accomplish?
  • Can the users accomplish their tasks easily?
  • Does the information and the order in which it is presented suite the purpose?
  • Is the important information easy to find?
  • Is all information clear, easy to understand and easy to read?
  • Do you always know where you are, or how to get where you want to go?
  • Is the presentation attractive?
  • Do the pages load quickly enough?

For the web, a complete checklist (112 criteria) allowing to evaluate the whole design process of a web site is available in Software Ergonomics and Web Design.