Recently, paper mills in China, which fabricate or sell authorship of articles, have been mass-producing forged academic papers — up to 30 per week — for each person who avails of their services. They are doing all this with the help of generative AI tools. While this story was on the news, a scientific conference, Agents4Science 2025, was held on 22nd October, welcoming papers from every sector of science, with one condition: AI must do the majority of the work.
AI was used to formulate hypotheses and analyze data, and was also the main driver of the first round of peer reviews, before human review took over. Out of 314 papers, 48 made the cut, and each paper had to describe how AI was used in their research process and writing.
“We’re seeing this interesting paradigm shift,” said James Zou, a computer scientist at Stanford University who co-organized the conference. “People are starting to explore using AI as a co-scientist.”
The papers spanned from subjects like economics to biology. One paper concluded that waiving towing fees would help people on lower incomes, and it collected its data from car-towing statistics in San Francisco.
Most publishing companies have strict restrictions on the use of AI for co-authoring and peer review. According to Zou, this conference was an experiment conducted to explore the viability of AI in publishing scientific papers. All the articles are publicly available.
Risa Wechsler, a computational astrophysicist at Stanford, was one of the human reviewers, and according to her, AI agents aren’t good enough to produce a robust scientific paper, as the papers were neither interesting nor important. There is also a lack in AI agents’ ability to raise good scientific questions that can guide future work in the field.
Amidst the mixed results, one paper on AI reasoning in the job marketplace stood out and ranked among the top three. It was generated using ChatGPT, and the results were novel. AI is still in its infancy. With further developments, it can truly shine in the field of scientific publications. However, AI should complement what scientists write, and not be the sole driver of scientific writing.
Contributors
Copyright @smorescience. All rights reserved. Do not copy, cite, publish, or distribute this content without permission.
