Documentation, Informaiton & Knowledge ›› 2023, Vol. 40 ›› Issue (2): 10-19.doi: 10.13366/j.dik.2023.02.010

Previous Articles     Next Articles

Measuring Human-AI Interaction Experience: Implications for Conducting Controlled Experiments

JIANG Tingting, TIAN Huiyi, XU Yanrun, FU Shiting   

  • Online:2023-03-10 Published:2023-05-09
  • Contact: Correspondence should be addressed to JIANG Tingting, Email:tij@whu.edu.cn, ORCID: 0000-0002-5310-2073
  • Supported by:
    This is an outcome of the Major Project "Information Service System Restructuring and Application Driven by Human-Centered Artificial Intelligence"(22&ZD325)supported by National Social Science Foundation of China and the project "The Formation Mechanism and Synesthesia-Based Intervention of Information Cocoons"(72074173)supported by National Natural Science Foundation of China.

Abstract: [Purpose/Significance] Human-AI interaction(HAII)has become a frontier research theme in the field of human-computer interaction. The research of Human-AI interaction focuses on the processes and outcomes of human using artificial intelligence to accomplish specific tasks and explores the function and mechanisms of each element in the process. This study aims to reveal the application of controlled experiments in existing foreign HAII research and to acquire implication, which would provide guidance for future experimental studies and accelerate the development and popularization of human-centered AI. [Design/Methodology] This study conducts a systematic review on 114 experimental research articles on Human-AI interaction experience, which were published between 2007 and 2022. It proposes a five first-level categories coding table including experimental samples, experimental tasks, experimental materials, experimental observation dimensions and data analysis methods, based on which the content analysis on the experimental design part of the literature has been conducted. [Finding/Conclusion] Previous researches have shown the characteristics such as single sampling method, low reusability of experimental materials, insufficient use of objective experimental observation tools, and prospective experimental tasks. [Originality/Value] The research focus on the emerging research direction of human-AI interaction experience, is the first to comprehensively review and analyze the research methodology and contribute a complete set of basic framework for experimental design, which helps to improve the standardization and scientificity of human-AI interaction experience research.

Key words: Human-AI interaction experience, Research method, Controlled experiment