Evaluation Strategies for HCI Toolkit Research

Research output: Contribution to book/anthology/report/proceedingArticle in proceedingsResearchpeer-review

DOI

  • David Ledo, Department of Computer Science, University of Calgary, Canada
  • ,
  • Steven Houben, Lancaster University, United Kingdom
  • Jo Vermeulen
  • Nicolai Marquardt, UCL Interaction Centre, University College London, UK
  • ,
  • Lora Oehlberg, Department of Computer Science, University of Calgary, Canada
  • ,
  • Saul Greenberg, Department of Computer Science, University of Calgary, Canada

Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementa-tion of interactive systems. For publication, the HCI commu-nity typically expects toolkit research to include an evalua-tion component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and dis-cussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strat-egies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.

Original languageEnglish
Title of host publicationCHI 2018 - Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems : Engage with CHI
EditorsRegan Mandryk, Mark Hancock
Number of pages17
Place of publicationNew York, NY
PublisherAssociation for Computing Machinery
Publication year20 Apr 2018
Pages36:1-36:17
ISBN (print)978-1-4503-5620-6
ISBN (Electronic)9781450356206, 9781450356213
DOIs
Publication statusPublished - 20 Apr 2018
EventACM SIGCHI Conference on Human Factors in Computing Systems: Engage with CHI - Palais des Congrès de Montréal, Montreal, Canada
Duration: 21 Apr 201826 Apr 2018
https://chi2018.acm.org

Conference

ConferenceACM SIGCHI Conference on Human Factors in Computing Systems
LocationPalais des Congrès de Montréal
LandCanada
ByMontreal
Periode21/04/201826/04/2018
Internetadresse
SeriesCHI '18

    Research areas

  • design, evaluation, prototyping, toolkits, user interfaces, Design, Evaluation, Toolkits, User interfaces, Prototyping

See relations at Aarhus University Citationformats

ID: 127852562