Medical journals, law reviews, and literary magazines are grappling with AI-generated submissions

For more than 50 years, the political and cultural magazine Boston Review has given emerging and established writers a platform to go deeper than the headlines on the world’s most pressing issues: war, the rise of authoritarianism, growing inequality. Typically, the publication’s three editors will receive 500- 700 unsolicited submissions annually through its online portal. But in the past two years, the number of submitted drafts has exploded, tripling for the first six months of this year compared to the same period in 2023. The new influx isn’t likely due to a revived interest in critical analysis. Editors at the Boston Review suspect a sobering reality: a flood of artificial intelligence-generated submissions. “It’s a huge drain on our small staff,” said executive editor Matt Lord. “Practically speaking, there’s no world in which we can continue reading every single submission we get very closely and not make faster judgments in some cases.” The people who run Boston’s prominent literary magazines , university-affiliated academic journals , and scientific publications, such as the New England Journal of Medicine , say they’re increasingly struggling to tell the difference between human and AI writing. While these publications serve relatively narrow audiences, their contributions are key for new thinking about the world. And their struggles are part of a broader battle over how to preserve human understanding in the age of AI. “Embodied human experience will always remain a human strength,” said Bill Pierce, editor of the Boston-based literary magazine AGNI. He said AI-generated writing simply cannot convey the depths of what a person can. “Someday maybe it will be able to tell us what the hell it is,” he said of AI. “But it would still not be telling us what we are.” But some say AI can be beneficial for writing when the value of an article is the original, scientific research. “If someone gets help with AI to do the writing, particularly if English isn’t their first language and it communicates authentic data, then fine. I’d rather have a well-written article,” said Dr. Eric Rubin, editor-in-chief of the New England Journal of Medicine. “There’s no question that articles from foreign authors have improved tremendously.” That journal, however, also publishes opinion pieces and letters to the editor . That’s where AI writing can be harmful. “We don’t really care about what ChatGPT’s opinion is,” Rubin said. Up until very recently, the prospect of using AI to generate competent prose seemed far off. But new scandals have emerged, including one at the New York Times, which cut ties with a freelance book reviewer who used AI to help write a book review, and at a British Commonwealth short story competition, where an author was accused of using AI to write a prize-winning short story. AI-generated books are also flooding Amazon and even seeping into bookstores . Even publications that haven’t seen a massive uptick are aware of the threat of machine-made submissions. Many are wrestling with what guidelines to implement for writers and researchers. Like other publishers, MIT Press adopted general rules when large language models first began to take off. It prohibits applicants from listing AI models as an author, and requires authors to be completely transparent about how they used AI in the research and writing process (some individual MIT journals also have implemented more specific guidelines). But editors across the organization’s 43 journals kept receiving papers they suspected were produced with the help of AI, minus the disclosures. Some of the signs are well-documented, such as the frequent use of certain phrases, a heavy reliance on em-dashes , and syntax such as “it’s not X, it’s Y.” Not only has there been what Nick Lindsay, the MIT Press director of journals, calls ‘a significant uptick’ in papers they suspect are written by AI, but also a few instances where an author has submitted up to five papers to a journal in a single day, which is a massive number. “That can be a real problem,” Lindsay said. “Each submission actually costs the publisher something,” adding that the platforms MIT Press use for submission, which help them manage a high volume of papers, typically charge the organization anywhere from $10 to $32 each time someone submits an article. While AI is hampering some publications, others aren’t as concerned. And in some cases the tech can be beneficial, such as in fact-checking large datasets, and translations that Rubin, of the New England Journal of Medicine, referenced. Shyam Madhani, editor-in-chief of the Yale Law Journal, said the publication took an open approach to crafting its first AI policy , which it created this year and requires writers to disclose how they used the technology, adding that its usage would not deter the journal from considering their work. “There’s no consensus either inside the journal or outside in legal academia on how to evaluate AI use in scholarship,” he said. He argued that some of the concerns typically raised, such as prose being too formulaic or arguments that have already been published by other authors, are not AI problems, but rather issues with the quality of writing or novelty of the paper itself. Those are factors for which the journal is already reviewing, so the use of AI is secondary. Of course, that puts a greater emphasis on reviewing submissions to ensure that they are sound to publish. “We’re trying to make sure we adapt as things go on,” Madhani said. The editorial staff has established an approach “mostly not on how to evaluate pieces that use AI writing, but how to enforce a policy where our main job as editors is to ensure the accuracy of the pieces.” At the New England Journal of Medicine, Rubin said that editors are implementing AI into its own review process . That includes statistical editors, who use the tech to help fact-check models. But there are other challenges. Rubin said there is little the journal can do if the science itself is AI-generated. Images are another problem. In May, the NEJM had to retract a case study regarding black “bronchial casts” that were removed from a man’s lungs following a forest fire, after its authors admitted to using artificial intelligence to manipulate the image. At issue was a crooked ruler placed at the top of the samples that authors used AI to straighten. “That was a kind of dumb one,” Rubin said. “It seems like we’re always adjusting to the last crisis.” To help with the opinion pieces, Rubin said the journal is experimenting with AI detectors, but is unsure of their efficacy. He declined to disclose which tools his team is using, out of fear that researchers and experts would use them ahead of submitting their papers. Despite the challenges AI is causing and the potential impact it could have going forward, Rubin believes people will always be at the heart of the journal’s work — a sentiment that was shared across publications of various disciplines. “We’re primarily a clinical research journal. You can’t do clinical research without sitting down with patients,” he said. “Not everything is built into the literature.” Aidan Ryan can be reached at aidan.ryan@globe.com . Follow him @aidanfitzryan .