Abstract: [Purpose/Significance] Human-AI interaction（HAII）has become a frontier research theme in the field of human-computer interaction. The research of Human-AI interaction focuses on the processes and outcomes of human using artificial intelligence to accomplish specific tasks and explores the function and mechanisms of each element in the process. This study aims to reveal the application of controlled experiments in existing foreign HAII research and to acquire implication, which would provide guidance for future experimental studies and accelerate the development and popularization of human-centered AI. [Design/Methodology] This study conducts a systematic review on 114 experimental research articles on Human-AI interaction experience, which were published between 2007 and 2022. It proposes a five first-level categories coding table including experimental samples, experimental tasks, experimental materials, experimental observation dimensions and data analysis methods, based on which the content analysis on the experimental design part of the literature has been conducted. [Finding/Conclusion] Previous researches have shown the characteristics such as single sampling method, low reusability of experimental materials, insufficient use of objective experimental observation tools, and prospective experimental tasks. [Originality/Value] The research focus on the emerging research direction of human-AI interaction experience, is the first to comprehensively review and analyze the research methodology and contribute a complete set of basic framework for experimental design, which helps to improve the standardization and scientificity of human-AI interaction experience research.

Keywords: Human-AI interaction experience, Research method, Controlled experiment