I Tested Human and ChatGPT Writing With 5 AI Detectors. Here’s What Happened
Disclosures: I am associated with AI Checker Pro, which is one of the five tools in this test. An AI assistant helped turn my real test notes into the first draft of this article. I reviewed, edited, and fact-checked the final version before publishing.
I gave five AI detectors the same human-written text and the same ChatGPT-generated text. Four of them scored the ChatGPT sample at 100% AI. Grammarly gave it 61%.
My own writing scored between 0% and 2.8% AI.
That gap made me curious. The tools agreed on the big picture, but did they agree on the numbers? I wanted to see how closely the five tools would agree.
Why I ran the test
AI detector scores are often discussed as if they are definitive. A number appears, and people treat it as a verdict.
I wanted to see what happens when the exact same samples go through several popular tools. That was the whole idea behind this AI detector test.
How I ran the experiment
I used two samples.
The first was written by me. It was a 252-word paragraph about how I use AI in my daily work. I did not use ChatGPT, Grammarly, or any other AI tool to write or correct it before testing.
The second was a 251-word passage generated entirely by ChatGPT. I did not edit, rewrite, humanize, or correct it.
Neither sample was changed after creation. I tested both with the same five detectors:
- AI Checker Pro
- GPTZero
- Copyleaks
- ZeroGPT
- Grammarly AI Detector
Here is the full set of AI detection results in one place:
Round 1: my human-written sample
I started with my own paragraph.
Four tools returned 0% AI: AI Checker Pro, GPTZero, Copyleaks, and Grammarly. ZeroGPT returned 2.8% AI.
That small number didn’t change the outcome. ZeroGPT’s final classification was still “Human written.” So all five tools ultimately classified my writing as human.
For the other details, Copyleaks showed all 252 words as human text. GPTZero showed 100% human. Grammarly showed 100% with no AI text patterns found. AI Checker Pro showed 0% AI and 100% human.
ZeroGPT did highlight a couple of lines in my text. Its overall score stayed low.
AI Checker Pro classified my 252-word human-written sample as 100% human and 0% AI.AI Checker Pro: 0% AI, 100% human for my 252-word sample.
GPTZero classified the same human-written sample as 100% human and 0% AI.GPTZero: 0% AI, 100% human.
Copyleaks found no AI content in my 252-word human-written sample.Copyleaks: 0% AI, no AI content found, 252 words classified as human.
ZeroGPT gave my human-written sample a 2.8% AI score but still classified it as human-written.ZeroGPT: 2.8% AI, with a final classification of human written.
Grammarly AI Detector reported 0% AI patterns in my human-written sample.Grammarly AI Detector: 0% appears AI-generated, 100% no AI text patterns found.
Round 2: the ChatGPT sample
Next, I tested a fresh 251-word passage generated entirely by ChatGPT. Again, I did not edit it.
Get Harshil Barvaliya’s stories in your inbox
Join Medium for free to get updates from this writer.
Four tools gave it 100% AI: AI Checker Pro, GPTZero, Copyleaks, and ZeroGPT.
Copyleaks counted 251 AI words and 0 human words. GPTZero classified the text as AI generated. ZeroGPT classified it as AI/GPT generated.
Grammarly gave a different number. I cover that below.
AI Checker Pro classified the 251-word ChatGPT-generated sample as 100% AI.AI Checker Pro: 100% AI, 0% human for the ChatGPT sample.
GPTZero classified the same ChatGPT-generated sample as 100% AI.GPTZero: 100% AI, 0% human. Classification: AI generated.
Copyleaks classified all 251 words in the ChatGPT sample as AI-generated.Copyleaks: 100% AI, with 251 AI words and 0 human words.
ZeroGPT gave the ChatGPT-generated sample a 100% AI/GPT score.ZeroGPT: 100% AI/GPT. Classification: AI/GPT generated.
Grammarly AI Detector reported 61% AI patterns in the ChatGPT-generated sample.Grammarly AI Detector: 61% appears AI-generated, 39% no AI patterns found.
The biggest difference I noticed
The biggest difference was Grammarly’s score on the ChatGPT text.
Four tools said 100% AI. Grammarly said 61%.
I’m not saying Grammarly was wrong. It still found a substantial share of AI signals in the passage. The tools simply did not report the same level of confidence for the same text.
I can’t say why from this test alone. In general, different detectors may use different models, thresholds, scoring methods, and definitions of what a percentage means. A 61% from one tool may not mean the same thing as a 61% from another.
The key observation is simple. Same text, same word count, different confidence scores.
What this small test actually shows
Here is what I can say from these two samples:
- All five tools classified my human sample as human.
- All five found AI signals in the ChatGPT sample.
- Confidence scores differed between tools.
Here is what I can’t say. A detector percentage should not automatically be treated as proof of authorship. It is a signal, not a verdict.
Two samples are also far too few to measure AI detector accuracy. I tested one human paragraph and one ChatGPT passage, both around 250 words. I can’t claim these results apply to every human or AI-written text.
Other kinds of writing could produce different results. That includes heavily edited AI text, human writing that happens to sound formulaic, or text in another style or length. I did not test any of those.
I also won’t name a winner. The tools didn’t clearly separate from each other here, apart from the one score difference on the ChatGPT sample. That is not enough to rank them.
Final thoughts
Running this made one thing clearer to me. A single percentage can look more certain than it is.
AI detectors can give useful signals. But readers, students, teachers, and editors should look beyond one number. Checking more than one tool, reading the text itself, and talking with the writer all add context a score can’t.
My two samples showed broad agreement, with one visible difference in confidence. Treat that as an observation from a small experiment, not a conclusion about every detector.
