Scientific papers with AI references receive more citations

Scientific papers with AI references receive more citations

Studies with titles or abstracts that specify Artificial intelligence (AI) methods are more likely to be in the top 5% cited papers in their field than those that do not reference these techniques. These jobs also receive more Citations from other disciplines than studies that do not use AI terms.

But this “citation boost” is not noticed equally by all authors. The analysis also shows that researchers from Groups that have been historically underrepresented in science, do not receive the same citation growth as their colleagues when they use AI tools in their work - suggesting that AI exists Inequalities could exacerbate.

The findings come from a study aimed at quantifying the use and potential benefits of AI in scientific research. However, the latest report in Nature Human Behavior also raises concerns. Scientists could be lured into using AI solely as a means to increase their citations—regardless of whether the AI ​​tools actually improve the quality of papers, says Lisa Messeri, a science and technology anthropologist at Yale University in New Haven, Connecticut. “We want to make sure that when we [invest] in AI, we don’t neglect other approaches,” she says.

The study also provides much-needed quantification of how AI is changing scientific research says Dashun Wang, co-author of the study and a science researcher at Northwestern University in Evanston, Illinois. “Now we finally have systematic data,” Wang said, which will be critical to addressing disparities related to the use of AI in science.

Tracking the rise of AI

To measure scientists' engagement with AI, the authors identified AI-related terms - such as 'machine learning' and 'deep neural networks' - in the abstracts and titles of nearly 75 million papers published from 1960 to 2019 across 19 disciplines. Wang acknowledges that due to the deadline, the study ignores current developments in AI, including the rise of large language models such as ChatGPT, which is already changing the way some researchers do science, cannot fully grasp.

According to the study, scientists in all 19 disciplines have increased their use of AI tools over the past two decades (see 'Use of AI is increasing'). However, there are significant differences: computer science, mathematics and engineering show the highest levels of AI use, while history, art and political science show the lowest. The rates for geology, physics, chemistry and biology are in between.

Um the potential benefits of AI For each discipline, the authors first identified research-related tasks that AI can perform. They then tracked the growth of these skills over time by tracking specific verb-noun pairs, such as 'analyze data' and 'generate image', in publications about AI between 1960 and 2019. By examining how much these terms overlapped in AI-related publications compared to the fundamental tasks of a given research field over time, researchers were able to assess whether AI's capabilities could meet the evolving needs of that field.

Once again, computer science, mathematics and engineering were associated with the highest potential benefits, while history, art and political science had the lowest.

Marinka Zitnik, a biomedical informatics specialist at Harvard Medical School in Boston, Massachusetts, explains that the study's approach is interesting because it allows for systematic analysis across multiple scientific disciplines. However, it also has limitations. “Because the authors wanted to conduct a very broad, systematic study, they couldn't necessarily go into detail and fully understand the specific reasons why a particular verb or noun appeared in a paper,” she says. Just because certain verbs and nouns appear together in a paper doesn't mean that if AI can perform the task described, it will necessarily be useful for that field, she notes.

1. Gao, J. & Wang, D. Nature Hum. Behavior https://doi.org/10.1038/s41562-024-02020-5 (2024).

   Article  
   Google Scholar  

Download references