图书情报知识 ›› 2019, Vol. 0 ›› Issue (2): 102-110.doi: 10.13366/j.dik.2019.02.102

• 情报、信息与共享 • 上一篇    下一篇

网络搜索引擎自然语言问答能力的评价研究

赵一鸣,夏雪,孙永强   

  • 出版日期:2019-03-10 发布日期:2019-03-10

Evaluation of the Natural Language Question Answering Ability of Web Search Engines

  • Online:2019-03-10 Published:2019-03-10

摘要:

[目的/意义]以Google为代表的主流网络搜索引擎正在从基于关键词匹配的检索系统演变为兼具检索与自然语言问答功能的综合平台,亟需引入新的搜索引擎评价维度和方法以适应这种转变。[研究设计/方法]使用文本检索会议自动问答系统评测中的自然语言问题,从定量和定性两个方面,以Google为例对网络搜索引擎的自然语言问答能力进行评价。 [结论/发现] 定量评价结果显示,Google已经具备了一定的自然语言问答能力。在定性评价方面,通过对搜索结果页面中的答案框进行内容分析,揭示了搜索引擎问答能力的特点以及存在的不足,为搜索引擎问答能力提升提供了建议。[创新/价值]为搜索引擎评价研究提出了自然语言问答能力这一新的评价维度。

关键词: 搜索引擎, 自然语言问答, 信息检索评价, 搜索引擎评价, 问答服务

Abstract:

[Purpose/Significance]New measurements for search engine evaluations should be investigated to deal with the shift that Web search engines like Google are evolving from the keyword-matching retrieval systems to the comprehensive platforms with functions of searching and natural language question answering(QA).
[Design/Methodology]Questions used in TREC QA Track were used to test the QA ability of Google both from quantitative and qualitative aspects.
[Findings/Conclusion]The results of qualitative evaluation show that Google has already acquired some QA ability in natural language. Based on content analysis of answers, the characteristics and the existing problems of search engines' QA ability have been revealed. Meanwhile, suggestions on how to promote the QA ability have been provided. [Originality/Value]This paper has proposed a novel measurement to evaluate search engines from the perspective of natural language question answering ability.

Keywords: Search engine, Natural language question answering, Information retrieval evaluation, Search engine evaluation, Questionanswering service