Proving Humanity: The Failure of AI Detection Tools

This piece was written by our summer intern, Kalani Gaviola. We’re grateful for her contribution to the blog.

OpenAI publicly launched ChatGPT on November 30, 2022. Five days later, Sam Altman, CEO and co-founder of OpenAI, tweeted the following: “I am a stochastic parrot, and so r u”. 

The tweet is a reference to a 2021 AI ethics paper, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”, which describes LLMs as stochastic parrots—able to convincingly repeat information based on probability, but unable to truly comprehend their outputs. While the paper ultimately uses this metaphor to frame its argument on the limitations and downsides of AI, Altman’s motives are more opaque. Mere days after the release of the most powerful AI chatbot to date, the words are eerie—are they a joke? A warning? A prophecy? 

An informed reader might say all of the above. 

I am a stochastic parrot, and so r u. 

Indeed, in the years following the release of ChatGPT and the AI boom that followed, plagiarism has become one of the most pressing issues in academic spaces—and beyond—as writers use the ever-increasing capacities of AI to create in their stead, parroting AI outputs instead of truly processing and analyzing material. According to recent research from the College Board, college faculty estimate that nearly three quarters of students use AI to write papers or essays. Universities have adjusted plagiarism policies in the face of automated writing. In May 2026, for example, Berkeley Law enacted one of the strictest AI policies to date. This policy functionally bans use of AI in any part of completing course work aside from “identifying sources”.

In this scramble to combat academic misconduct, AI detection tools have become our best solution and de facto detectives. But how good are they, really? And how does the never-ending investigation of wrongdoing and ultimately, AI’s writerly abilities, affect students and authors alike?

In January 2023, Princeton student Edward Tian created GPTZero, a leading AI detection tool that he began coding mere weeks after the release of ChatGPT in 2022. 

GPTZero “analyzes linguistic and statistical signals learned from human and AI text to estimate the likelihood that text was generated by an LLM”—a method used by various similar tools such as QuillBot, ZeroGPT, Originality AI, and more. As stochastic parrots, AI models function as prediction engines and tend to produce outputs with a statistical regularity human writings usually lack. For GPTZero, the signals to catch a stochastic parrot include perplexity (how predictable a sequence of words is), burstiness (variation in sentence length and structure), along with five other “indicators.” 

Quickly, however, many pointed out the issues with this type of tool. While some markers may be indicative of AI use, large language models are ultimately trained on human writing and mimic it accordingly. In April 2023, several screenshots of the U.S. Constitution being flagged by GPTZero as “likely to be written entirely by AI” went viral, thus throwing the effectiveness of the entire enterprise into question. A few months later, a Stanford study showed that these tools are biased against non-native speakers, whose writing often scored higher in predictability and thus was flagged consistently as AI by detectors. 

Even if one assumes accuracy, these tools are not necessarily effective. Gen Z is a tech-savvy generation. For every AI detector used by a teacher, there’s a tool that “humanizes” AI writing by purposely regenerating text to avoid AI markers, and often, you don’t have to look far. Even GPTZero, with the stated mission to “preserve what’s human,” recently joined Superhuman, the AI productivity suite behind tools like Grammarly, which allows users to “humanize” up to 1500 AI-generated words for free in mere seconds. The AI that enables detection can equally enable the generation of text specifically optimized to evade such detection. 

Other methods are similarly unreliable. Authory claims to verify human writers by creating a custom style fingerprint that compares your writing to itself to avoid the pitfalls of generalization—but if all your available works were created using AI, the accuracy seems dubious. Some propose implementing AI text watermarks, which requires “embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens.” But the watermarks are easy to remove. Other tools track the writing process by analyzing typing patterns, but, so long as you have some awareness of these tools, it seems easy enough to avoid pasting from AI and instead type it yourself while simply looking at what Claude generated on another tab. 

Despite the unreliability of these tools, they have gained widespread popularity. In 2023, a UC Davis student’s story went viral after a professor reported him for cheating using AI. The professor’s accusation was corroborated by multiple detection tools including GPTZero, but was ultimately dropped after the student provided his document history and testified with the university’s honor court. The student claimed the experience caused him to have “full-blown panic attacks“, and if he had been less savvy in proving his innocence, he would have been subject to both failing the class and facing possible academic probation. A professor at Texas A&M similarly failed an entire class based on false AI use claims, causing the students to be denied their diplomas. 

These types of stories have become commonplace—anecdotally, many students have been accused of using AI in some capacity. In academic spaces, the restrictions of AI seem reasonable; the purpose of education is to learn, and studies show that LLM users underperform at “neural, linguistic, and behavioral levels.” But some AI witchhunts have gone beyond academia, into fiction and creative spaces. 

If you believe creative media’s goal is to entertain or make you feel something, it seems at least certain AI use should be encouraged. After all, AI’s narrative writing is now good enough that many are fooled by short passages from more advanced LLMs, even preferring them to human writing. Still, the internet crucifies the Duffer Brothers for allegedly using AI to write the famously underwhelming Stranger Things final season. A popular self-published novel, Shy Girl, is pulled by its publisher post-AI allegations. Despite the fact it’s impossible to know definitively whether or not AI was used in the creation of these pieces, the accusations continue. There are practical arguments against AI use in the creative space, whether that be via algorithms naturally creating less unique stories or LLMs stealing jobs from human artists, but the outrage seems to go beyond that.

Fanfiction is an artform that generates no money for the writers involved. It is known for being trope-filled and easy to read– you don’t have to be Margaret Atwood or Toni Morrison to have thousands of people read and enjoy your work. Still, many in the fan fiction community are ardently against AI use. 

For example: in June 2026, an X (formerly Twitter) user called John Doe (@heatedrivalryai) created a thread informing netizens that they can search the HTML code of the popular fanfiction site Archive on Our Own for “font-claude-response-body” to identify if the author of a work pasted directly into the site from Claude. They also created a site skin that automatically turns the webpage red when a user clicks on a work that has this code embedded. Their post reached over 3M views within a week. 

Though the method is shaky and does not in fact detect a mass of AI use (for example, if the author didn’t paste directly into the site from Claude, or simply used another LLM, it is ineffective), the fandom space was still taken by storm. The thousands of users in replies and quote retweets alike swarmed authors found guilty of AI usage. While some brought up the practical argument, condemning AI use generally for environmental reasons or citing fanfiction authorship as a valuable learning opportunity for young authors, many appealed to the intrinsic sanctity of human authorship. 

“nobody is forcing ppl to write fics!!!” one wrote. “i don’t understand why ppl need ai to do it like pls just write fic from the heart.”

These tools and their ineffectiveness will surely continue to fester in our culture, but the extent of the outrage is interesting. It seems our discomfort with AI goes beyond practicality, and perhaps it should. 

To many, our writing and our art is a fingerprint of our humanity—a transcription of our thoughts, and thus our personhood. It is our emotion, however predictable and replicable it may be, and our connection, and while some don’t mind having AI girlfriends, most people would prefer our relationships stay human. 

 You shouldn’t stop caring about human writing. You can use AI in moderation, as an additive tool instead of a replacement, but you should prize your effort, your art, and your learning, because without it, what are you? 

The same goes for the AI detection tools—if you use these tools, use them as one piece of evidence instead of a death sentence, and use them only if you understand their efficacy and limitations. 

Sam Altman may be a stochastic parrot, but you don’t have to be.