Expert review of a website user experience context

A talk by Tom Van de Zande

I devised a new approach to perform an expert review of quantifiable context factors of a user experience. The goal was to unify existing heuristics, principles and checklists into a framework for evaluation.

It is by definition impossible to measure a unique user experience through an expert review, but it is possible to measure the quantifiable context factors of a user experience. These context factors are our best approximation of a unique user experience.

Ingredients for the framework are existing research, heuristics, guidelines and checklists from the last 25 years. About 100 items were selected for the construction of the new model. Tagging was used to visualize the underlying relations between the heuristics and guidelines. 34 tag categories emerged during this extensive analysis.

34 categories allow for detailed evaluation, but are way too much for fast comprehension. I played with it, and drew a model where I bundled 28 categories around the 6 outer principles from Morville’s Honeycomb. This master/detail approach hits the sweet spot between detailed evaluation and fast comprehension. As a bonus, the flexibility of this approach allows reconfiguration of the 34 categories around other starting points.

An extensive checklist was developed around the model. 196 questions are linked to one or more categories. For example, a question regarding alt tags on images relates to both the SEO (external findability) category and the disability compliance category. Each question is scored on a 5 point Likert scale. Means are calculated for each category and subsequently for each facet. Both the total facet score and the detailed category scores are visualized in one comprehensive radar graph. This allows to see the bigger picture of what is good, and where improvement is needed.