I Asked an AI to read 17,000 Theses
A quiet shift in Bachelor and Master Theses writing in Sweden
Anyone who grades student work has noticed it by now. Homework reads smoother than it used to, lab reports have acquired a confident academic register that nobody taught in the first lecture, and somewhere over the last two years the theses started to read that way too. I noticed it in the theses I supervise and examine, and since I mostly read theses from computer science, I had a theory about where it came from: us. We are the early adopters almost by definition. Our students had AI coding assistants in their editors before most people had heard of ChatGPT, they are comfortable with anything that runs on a computer, and every study of published research I had read put computer science at the top of the list of fields where AI-assisted writing shows up. If anyone was polishing their thesis with ChatGPT, it was surely my own field.
Before you read on, make your own guess. If you could look at every thesis written at a handful of Swedish universities over the last six years, would you expect to see AI in the writing at all? In how many of them? And in which fields would you expect it most — computer science, engineering, business, the social sciences? Hold on to that answer for a few paragraphs.
The problem is that you can’t check this one thesis at a time. Human readers can’t reliably tell AI-assisted text from student-written text, commercial detectors are unreliable and biased against people writing in a second language, and the AI companies seem to have drawn their own conclusions: this year both Anthropic and OpenAI started hiding invisible watermarks in the text their models produce. Watermarks only work from now on, of course. They say nothing about the three years of theses already written, graded, and archived. But those theses are still there, and while you can’t ask a single thesis whether AI touched it, you can ask a few thousand of them at once.
So that is what I did. Sweden has a national open repository, DiVA, where universities deposit their bachelor’s and master’s theses. The library at Dalarna University was kind enough to compile the full list for me, about 35,000 theses, and from that list I ended up analysing about 17,000 English-language theses from six universities, spanning 2020 to 2025. The release of ChatGPT in November 2022 sits right in the middle, which turns the whole archive into a natural experiment: three years before, three years after. From each thesis I took three random 500-word excerpts from the middle of the text — no abstracts, no introductions, just the body where the actual writing happens — and replaced every year from 2020 to 2025 with 20XX, so the model reading them couldn’t tell when they were written. Then I asked a language model to score each one from 0 to 10 on grammar, vocabulary, academic quality, and wordiness, and, for good measure, on how likely it thought the text was to be AI-generated. I also sorted the theses into six topical groups by their abstracts, which ended up labelled Sustainability, Society, Engineering, Energy, Interaction, and Informatics.
None of this is heroic computing. It took a few overnight runs on my laptop and on the small Mac cluster at our CHAI center, plus a stack of API calls. But it means that a machine read, in a consistent way, more theses than any examiner will read in a whole career. That is the point of the exercise, and it is the kind of quant-of-qual work that makes some colleagues in the humanities visibly nervous. An examiner reads one thesis and judges it as a whole, which is exactly right for grading and exactly wrong for this question, because every single thesis looks like a thesis. Whatever changed only becomes visible when you line up thousands of them by year. (This is, incidentally, the paper that got me stuck between two support bots, the one in the submission system and the one on the help line [link]. Publishing it was its own small study in automation.)
Get Arend Hintze’s stories in your inbox
Join Medium for free to get updates from this writer.
The first result was the one I expected. From 2020 to 2022 the four writing scores are essentially flat, and after 2022 they bend upward — grammar, vocabulary, academic quality, and conciseness, all together. These are scores given by an AI, not by a human reader, and that cuts both ways. A language model is not a better judge of writing than an experienced examiner, and it has its own preferences, which I will come back to. But it is a remarkably consistent one: it applies the same rubric to the first thesis and the seventeen-thousandth, it doesn’t get tired or irritated, it has no idea which university or which year a text comes from, and it doesn’t know what I am hoping to find. For comparing thousands of texts with each other, that consistency matters more than taste. The size of the change is moderate, about a third of a standard deviation over the period, but the shape is striking: in every one of the six topical groups, on every one of the four writing dimensions, the slope after ChatGPT is steeper than the slope before, typically by a factor of two to seven, and the difference is statistically significant in 22 of those 24 cases. Whatever happened, it happened everywhere at roughly the same time.
Before I get to the part that surprised me, there is the obvious objection, and it deserves a straight answer. The pre-ChatGPT years are also the COVID years. Maybe writing was depressed by remote teaching and simply recovered, and I am mistaking a recovery for AI. I take that seriously, but it doesn’t fit the shape of the data well. A recovery should have started when teaching went back to normal in early 2022, so the 2022 theses should already be climbing, and they aren’t; a recovery also tends to be gradual and uneven across universities, not a single bend in every field at once. What I can’t rule out is that 2020 to 2022 was itself an unusual baseline, because I have no data from before 2020. So this is an argument from timing, not a proof. The one corpus-wide event that lines up with a corpus-wide bend in 2022 is the release of ChatGPT, and I find that the most plausible explanation, but it remains an explanation, not a measurement of anyone’s AI use.
Now for the surprise. When I ranked the six groups by how much their writing changed, the order came out backwards. The largest shift was in Sustainability, where the post-2022 slopes run at about a quarter of a point per year on every writing dimension, followed by Society. Informatics and Engineering, my supposed early adopters, changed the least, at roughly half that rate. In published research it is the other way around: computer science leads, every time.
I don’t think this means computer science students use AI less, and the data can’t tell me that anyway. There are at least three readings. Theses in technical fields may have started closer to the ceiling of these scores and had less room to improve. They also contain less prose — a thesis that is half code, equations, and tables gives a language model less text to polish, even if the student used AI just as heavily, maybe on the code. And students in different fields may simply use these tools for different things. What the data do show is narrower and, to me, more interesting: the trace of AI in thesis writing is not concentrated where I was looking for it. It is in every field, and it is most visible in the ones I wasn’t watching.
How many theses are we talking about? This is where it is easy to say too much. A distribution-based lower bound puts it at at least one in seven theses after 2022 whose scores come from a shifted distribution. To get a feel for the upper end, I ran a small control: I took 18 theses from 2020, which predate ChatGPT, and had three different AI models polish them, excerpt by excerpt, the way a student might paste one passage at a time into a chat window. Polishing raised the writing scores by between two thirds of a point and a full point. The corpus as a whole moved by about a fifth to a quarter of a point, which is roughly what you would see if a quarter to a third of the theses had been fully polished. Reality is almost certainly messier, with more students using AI lightly rather than a few using it for everything. Surveys put the share of Swedish students who have used ChatGPT well above half, while the share who disclose it in submitted work, where disclosure is asked for, is much lower. Either way, every one of these theses passed and was approved by an examiner.
And then there was the strangest result, the one I hadn’t gone looking for. Remember the fifth score, the model’s own guess at how likely each text was to be AI-generated? It turned out to run in the opposite direction from the other four. The better the writing, the less likely the model thought it was AI. That was not something that appeared after ChatGPT; the same negative relationship is already there, at the same strength, in the 2020 theses, before there was anything to detect. And when I polished those 18 theses from 2020 with AI and rated them again, the model’s writing scores went up clearly, while its AI-likelihood score didn’t move at all. I didn’t trust a single rater, so I had two other commercial models and seven open-weight models score a subset of the theses with the same instructions. They broadly agreed on the writing scores. On the AI-likelihood score they agreed on nothing, not even the original model with itself when it saw different excerpts from the same thesis. The part of the instrument designed to detect AI was the one part that couldn’t be trusted, while the plain questions about grammar and vocabulary picked up the change.
One boundary to be clear about: this study looked only at theses written in English. I did not analyse theses written in Swedish and make no claims about them.
So here is where I ended up. I went in looking for the computer scientists and found the change everywhere instead — in sustainability reports and discourse analyses, in accessibility studies and grid decarbonisation, all bending upward at the same time, all approved. No single examiner could have seen it, and no detector would have flagged it. I am not sure yet what it means for what a thesis is supposed to examine, when the writing itself can no longer tell us much about the writer. I’d be curious what you see in your own students’ work, and whether it matches what the archive shows.
The full paper is open access in Social Sciences & Humanities Open: https://doi.org/10.1016/j.ssaho.2026.103905
