Some writers worry that AI detection software could falsely flag their work, potentially ruining their reputation or job opportunities before they have a chance to prove the detectors wrong.
cafeteria/Getty Images
hide title
cafeteria/Getty Images
Ghostwriter Sally McGraw hoped to get a job developing a book proposal for a potential client. But after submitting a writing sample, he received an unexpected response: an email telling him that multiple AI detection tools had flagged his work as machine-generated.
“I was stunned because I hadn’t used AI to generate the text,” McGraw said. “I had written it all myself.”
Having spent more than 20 years as a professional nonfiction writer, McGraw said she was angry that her work had been dismissed as written by AI.
“You know where you see a cartoon character turn red from the chin to the top of the head? I felt like my ears were starting to burn,” he said. “I thought, what’s going on?”
McGraw said she assured the client that she had written the proposal herself, but decided not to press it because she considered the work “minor.”
Sally McGraw, a ghostwriter, said she changed her writing style after a potential client suggested her work was generated by AI.
Sally McGraw
hide title
Sally McGraw
In an email to NPR, the client confirmed that it had reviewed McGraw’s work for AI. But he said results were not the determining factor in choosing to hire a different writer, instead citing experience and suitability. McGraw says he was paid for his writing samples, but believes the detector results cost him the opportunity to write the full book proposal.
As publishers struggle to keep up with the growing influx of AI-generated content, some are turning to screening tools to help sift through the growing number of submissions. But these tools aren’t perfect: They have been known to falsely flag human-written work as AI, which can leave writers with little idea what triggered the result and how to prove otherwise.
“Everyone knows that these evaluation methods are not 100% accurate,” McGraw said.
Editors are overwhelmed by AI-written submissions
Neil Clarke, editor and publisher of ClarkesmundoHe says a flurry of AI presentations at the science fiction magazine have put pressure on the editorial staff.
In 2021, the publication received approximately 1,100 submissions per month. By February 2023, after AI agents like ChatGPT, Claude, and Gemini began to become widely available, that number had doubled, forcing Clarke to temporarily close its submission portal. He estimates his own workload has increased by about 30%.
“It has introduced an element of distrust into the process,” he said.
When AI write detectors disagree
While AI detectors have become more sophisticated in recent years, the companies behind the detectors acknowledge that the tools are not foolproof. Researchers have found that different AI detection tools can reach dramatically different conclusions about the same piece of writing.
In a study published in June in the International Journal for Educational Integrity, researchers compared the accuracy of four widely used detectors. They tested 160 academic articles comprising four different categories: written entirely by humans, written entirely by AI, a combination of the two, and AI material rewritten by a human.
Among the four detectors, Pangram was the most consistently accurate, correctly identifying AI use in 37 of the 40 articles with the text merged and in 37 of the 40 articles where a human rewrote the AI-generated text.
Pangram CEO Max Spero, in an emailed statement, said his company’s detector can make mistakes, especially when there isn’t much writing to evaluate.
Previously published writings may pose an additional problem, says Chris Callison-Burch, a professor of computer and information sciences at the University of Pennsylvania.
“Many AI detectors work by asking a large language model how predictable a passage is,” Callison-Burch said. “The problem is that language models also tend to find the text in their training data unusually predictable.”
That could explain what science blogger Janelle Shane found in late 2022 when she tested excerpts from her 2019 book on how AI works. You look like something and I love you. One detector reported 98.7% confidence that a passage was generated by a machine.
“This is a clear, glaring, blinking light false positive,” Shane said. “I had written this text myself.”
An unpublished draft like McGraw’s writing sample wouldn’t face that particular problem, Callison-Burch said, but it could still be falsely flagged based on patterns in writing style.
Detectors look for clues in word choice, punctuation, and the way sentences are composed. The word “dig deeper,” frequent scripts and ideas grouped in lists of three can trigger a positive outcome for AI, he said. The same goes for opening sentences with adverbs like “notably” or “crucially,” or posing a question immediately followed by answering it.
At the same time, those are familiar habits for human writers. Callison-Burch says the detectors weigh patterns across an entire piece, so an em dash or a particular word alone doesn’t necessarily mean a bot was used.
What is at stake for writers?
The Authors Guild, an organization that advocates for writers’ rights, said in May that the use of AI detectors “creates an environment reminiscent of witch hunts,” and writers worry about false accusations and how to defend themselves.
“A false positive can cost an author their publishing contract and reputation, and a false negative, when AI-generated work slips through reputable publications unnoticed, can cause readers to lose trust in authors and the industry,” the Guild wrote.
Clarke, the editor of the science fiction magazine, falls somewhere in between. He says that while he doesn’t personally use AI detectors to screen incoming manuscripts, he believes certain screening tools could help overwhelmed editors. manage the volume, as long as they check the detector results. But he warns against turning suspicions into accusations that could damage a writer’s reputation.
“Submissions should end in a rejection, not an accusation,” he said.
McGraw says the fear of another accusation has changed the way he writes. She says she now avoids using certain words and sentence constructions, and occasionally omits commas she used to include, hoping the inconsistency will help prove that a human wrote the text.
“I miss my em dash. It was something I learned in journalism school and I’ve used it for decades,” he said.
The ghostwriter also sees the irony. Great language models learn from human prose, but she says she feels pressure to make hers less like AI.
“It’s like trying to trick a machine into thinking I’m not a machine,” McGraw said.
And he worries about facing another accusation when a bigger opportunity is at stake.
“I’m preparing for the time when it’s a task I’m really looking for and the outcome will make or break me,” he said.
