The Evaluator Effect in Usability Tests
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
The Evaluator Effect in Usability Tests. / Jacobsen, Niels Ebbe; Hertzum, Morten; John, Bonnie E.
CHI'98 Conference Summary. New York : ACM Press, 1998. p. 255-256.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - The Evaluator Effect in Usability Tests
AU - Jacobsen, Niels Ebbe
AU - Hertzum, Morten
AU - John, Bonnie E.
PY - 1998
Y1 - 1998
N2 - Usability tests are applied in industry to evaluate systems and in research as a yardstick for other usability evaluation methods. However, one potential threat to the reliability of usability tests has been left unaddressed: the evaluator effect. In this study, four evaluators analyzed four videotaped usability test sessions. Only 20% of the 93 unique problems were detected by all four evaluators and 46% were detected by only a single evaluator. Severe problems were detected more often by all four evaluators (41%) and less often by only one evaluator (22%) but a substantial evaluator effect remained
AB - Usability tests are applied in industry to evaluate systems and in research as a yardstick for other usability evaluation methods. However, one potential threat to the reliability of usability tests has been left unaddressed: the evaluator effect. In this study, four evaluators analyzed four videotaped usability test sessions. Only 20% of the 93 unique problems were detected by all four evaluators and 46% were detected by only a single evaluator. Severe problems were detected more often by all four evaluators (41%) and less often by only one evaluator (22%) but a substantial evaluator effect remained
M3 - Article in proceedings
SP - 255
EP - 256
BT - CHI'98 Conference Summary
PB - ACM Press
CY - New York
ER -
ID: 113158447