Abstract: [Purpose/Significance] Differences in the scope of indexing among various policy citation databases may lead to different results in the policy metric analyses of scientific papers. In order to better support the measurement and evaluation of the policy impact of scientific research outcomes, a comparative study is needed to understand the differences in the data foundations provided by different databases. [Design/Methodology] This study compared the similarities and differences in the range of policy documents included in Altmetric and Overton, two mainstream policy citation databases, during the same period. It also compared the policy citation data provided by Altmetric and Overton for approximately 17 million scientific papers, focusing on aspects such as data coverage, data volume, and disciplinary distribution. [Findings/Conclusion] Compared to Altmetric, Overton provides a broader range and greater number of policy documents and policy citations, demonstrating a significant data advantage. However, only a small portion of the policy documents included in Altmetric and Overton overlap, representing different policy viewpoints and behaviors of different policymakers regarding the use of scientific evidence. Therefore, these databases are difficult to substitute for one another and can be complementary. [Originality/Value] In the quantitative evaluation of the policy impact of scientific research achievements, we should pay attention to and warn the potential impact of the choice of policy citation database on the results, and develop reasonable data acquisition strategies to achieve more responsible evaluations.

Keywords: Policy metrics, Research evaluation, Societal impact, Scientometrics, Altmetrics