An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications
Altmetrics indices are increasingly applied to measure scholarly influence in recent years because they can reflect the influence of research outputs more timely comparing with traditional measurements. Simultaneously, artificial intelligence (AI), as an emerging interdiscipline, has a rapid development in these years. Traditional indices can’t reflect the influence of the AI research outputs quickly, thus more timely altmetrics indices are needed. In this paper, we conduct four studies about altmetrics indices and AI research outputs based on the datasets collected from Altmetric.com and Scopus database. First, we provide a review of the research status in the AI field. Second, we show the AI researches that attracted the most attention. Third, we demonstrate the general effectiveness of altmetrics indices in the AI field. Last, we examine the effectiveness of altmetrics indices for different levels of AI journal papers and AI conference papers. Our results indicate that there is a rapid increase of AI publications and the public has paid more attention to AI research outputs since 2011. It is found that altmetrics indices are effective to discriminate highly cited publications and publications whose citation counts increase quickly. Among all Altmetric sub-indicators, Number of Mendeley readers is the most effective. Moreover, the results indicate that altmetrics indices are more effective in high levels of AI journal papers and AI conference papers. The main contribution of this paper is investigating the effectiveness of altmetrics indices from the perspective of different levels of publications. This study lays the foundation for further investigations about effectiveness of altmetrics indices from new perspectives, and it has important implication for the studies about the impact of social media on the scientific community.
KeywordsAltmetrics Bibliometrics Artificial intelligence Highly cited publication Increase of citation count Citation analysis
The study is supported by funds from National Natural Science Foundation of China (Nos: 71722005 and 71571133 and 71790594 and 71790590). And from Natural Science Foundation of Tianjin (No. 18JCJQJC45900), the Humanities and Social Sciences Foundation of the Ministry of Education, China (Project No. 16YJC870011). We are grateful to Altmetric.com for providing the data.
- Altmetric.com. (2017). Sources of attention-altmetric track a unique range of online sources to capture the conversations relating to research outputs. https://www.altmetric.com/about-our%20-data/our-sources/.
- Bando, K. (2014). Importance of Mendeley readership on altmetrics based on the Altmetric score. Journal of Molecular Structure, 435(2), 123–132.Google Scholar
- Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Paper presented at the IEEE computer society conference on computer vision and pattern recognition.Google Scholar
- Harvey, C., Morris, H., Kelly, A., & Rowlinson, M. (2010). Academic journal quality guide. London: The Association of Business Schools.Google Scholar
- Hassan, S. U., & Gillani, U. A. (2016). Altmetrics of “altmetrics” using Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki.Google Scholar
- Hu, B., Lu, Z., Hang, L., & Chen, Q. (2014). Convolutional neural network architectures for matching natural language sentences. In Paper presented at the international conference on neural information processing systems.Google Scholar
- Kemp, C., Tenenbaum, J. B., Griffiths, T. L., Yamada, T., & Ueda, N. (2006). Learning systems of concepts with an infinite relational model. In Paper presented at the national conference on artificial intelligence. Google Scholar
- Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. In International conference on learning representations. arXiv:1412.6980.
- Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Paper presented at the Advances in neural information processing systems.Google Scholar
- Lavin, A., & Gray, S. (2016). Fast algorithms for convolutional neural networks. In Paper presented at the computer vision and pattern recognition. Google Scholar
- Ling, C. X., Huang, J., & Zhang, H. (2003). AUC: A better measure than accuracy in comparing learning algorithms. In Paper presented at the Canadian society for computational studies of intelligence conference on advances in artificial intelligence. Google Scholar
- Lyder, A. (2008). Mechanical design of Odin, an extendable heterogeneous deformable modular robot: Intelligent robots and systems. In Paper presented at the IEEE/RSJ international conference on intelligent robots and systems, 2008.Google Scholar
- Ming, J., Han, J., & Danilevsky, M. (2011). Ranking-based classification of heterogeneous information networks. In Paper presented at the Acm Sigkdd international conference on knowledge discovery and data mining.Google Scholar
- Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(7431), 159.Google Scholar
- Pooladian, A., & Borrego, Á. (2017). Methodological issues in measuring citations in Wikipedia: A case study in library and information science. Scientometrics, 113(3), 1–10.Google Scholar
- Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7). Available at: https://firstmonday.org/article/view/2874/2570/.
- Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv:1203.4745.
- Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. http://altmetrics.org/manifesto/.
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Paper presented at the proceedings of the IEEE conference on computer vision and pattern recognition. Google Scholar
- Wouters, P., & Costas, R. (2012). Users, narcissism and control—Tracking the impact of scholarly publications in the 21st century. Available at: https://apo.org.au/node/28603/.
- Zahedi, Z., Costas, R., & Wouters, P. (2013). What is the impact of the publications read by the different Mendeley users? Could they help to identify alternative types of impact. In Plos Alm workshop.Google Scholar