Everyone has a game they say they don’t make anymore. The one you played until three in the morning, the one whose map you could still draw, the one your friends still bring up. Set it beside this year’s releases and the verdict feels obvious: games were better before.

Memory isn’t a scoreboard, though. Steam has kept one for more than a decade, written by millions of players, so I wanted to know whether it agrees.

Before trusting it, a warning from the movies.

Movie critics’ scores have been climbing. A recent analysis on Reddit’s r/dataisbeautiful found the average Rotten Tomatoes score up about 15 points in twenty years, most of it since 2015. One explanation it offers: Rotten Tomatoes now counts far more critics per film, and a bigger, different crowd votes differently.

Steam looks different. Its first-year ratings sit higher for games launched after late 2019, and since then they’ve barely moved.

So does that settle the old argument that games were better before? I went through millions of Steam reviews and tried three scoreboards.

They don’t agree.

Line games up by launch year, and newer games look better. Ask QG Score, and older games look better. Compare games at the same age, and older games edge ahead again, but only just.

Short answer: old games weren’t more loved. Newer games do score higher, but that lines up with Steam changing who writes reviews in 2019, not with anything about the games. Put old and new on equal terms and they come out about level. What the reviews do show clearly is something else: in every era, games are rated highest when they’re new, and then slowly slide.

Here’s what sits behind each scoreboard.

Scoreboard 1: newer games get better first-year reviews

Start with the obvious chart: how often players recommended a game in its first year, by the year it came out.

I only count engaged players here: people with at least two hours of play when they wrote the review. That filters out quick first impressions.

Every launch year since 2020 beats every launch year from 2014 to 2018. The biggest game of each year usually beats the average too, with famous exceptions like PUBG: Battlegrounds and Monster Hunter: World, a game whose later reviews climbed to 95%. These are picked by review count, which favours older games, so treat them as examples, not a test.

So games got better? Not so fast. Look at the gap in the middle of the chart. Something happened in late 2019.

In late 2019, Steam changed who writes reviews

On 30 October 2019, Steam launched its new Library, and it started prompting people to review games they’d been playing. Valve says daily reviews roughly quadrupled.

That changed who was writing them, including the engaged players this article counts. As reviews multiplied, the share from people with under two hours fell from about 21% to about 12%, so most of the new reviews came from people who’d played a while.

Why would that push ratings up? Probably because of who reviews without being asked. Unprompted reviewers tend to be people with strong feelings, often negative ones. A prompt reaches ordinary players, and most players still playing a game are happy with it. That’s a likely explanation, not one I could test: Steam doesn’t mark which reviews were prompted.

So games launched before and after 2019 were judged by different crowds, and the two sides can’t be compared directly. I can’t separate the prompts from anything else that changed at the time. It’s the same mechanism behind the Rotten Tomatoes story: change who’s voting, and the score changes with it.

Within each side, the picture is flat. Ratings for 2020 to 2024 launches barely moved, though that leans on the 2024 games, and for 2014 to 2018 the data can’t say either way.

Scoreboard 2: our own score says older games are better

Now a second scoreboard: QG Score, the rating we publish at QuantGamer. Games launched before late 2019 average about 8 points higher than newer ones. A clear lead for the old guard.

But QG Score gives more weight to games with more reviews. That’s deliberate: a score built on 50 reviews deserves less confidence than one built on 50,000. And older games have had years to collect reviews.

Give every game the same weight, and the lead vanishes: the two groups end up within half a point of each other. It isn’t that old games get a bonus. Games with lots of reviews do, old or new. Even PUBG: Battlegrounds drops from 83 to 66 at the catalogue’s typical weight. Older games are just more likely to have that many reviews.

So QG Score isn’t saying old games were better. It’s saying we have more evidence about them.

Scoreboard 3: the fairest test I could build

The first two scoreboards compare games judged by different crowds, or with different amounts of evidence. So I built a fairer one.

Use only reviews written since 2020, so every game faces the same crowd. And compare games at the same point in their lives: a 2017 game in its fourth year against a 2022 game in its fourth year.

Older games still edge ahead, by about three points in their fourth year. The eighth year looks similar; the sixth is unclear. One catch: only games still being reviewed at that age count.

Is that the proof? Two things say no.

Ratings slip a little every year

I followed a fixed set of 788 games launched from 2014 to 2017. From 2020 to 2025, their ratings slipped about 0.7 points a year. And every launch year was rated lower in 2025 than in 2021.

That matters for the fair test. A 2017 game reached its fourth year around 2020. A 2022 game reached it around 2025, five years further along that slide. Take the slide out, and the fourth- and sixth-year gaps disappear.

What I can’t tell is why ratings slip: whether games simply age, or reviewers are getting tougher.

The survivors

The old games still being reviewed years later are the ones people kept playing. The ones that dropped out had started with noticeably lower ratings. The fair test can only include the survivors, which flatters the older launch years.

What happens to a game’s rating over its life

Comparing games across eras is hard. Comparing a game with itself is easier, and it shows something many players wouldn’t guess: launch reviews are the kindest.

Engaged players recommend games most often in their first three months. By the second half of the first year, the share has dropped a couple of points, and that happens in both eras. Different people write those later reviews, but the pattern holds.

That makes the famous comebacks stand out. Monster Hunter: World was rated 28.6 points higher later in its first year than at launch, and Batman: Arkham Knight and No Man’s Sky climbed too. Others went the other way, from Civilization VI to PUBG: Battlegrounds. Starfield had the steepest fall of the 43 games I looked up: from about 65% of engaged players recommending it to about 41%. And some games barely move at all, like Hollow Knight, loved at launch and still loved years later.

Yearly sports games often run the other way: NBA 2K19’s later reviews were far kinder than its launch ones.

I picked these after seeing the data, as illustrations. They’re the exceptions, not the rule: on average, games are liked most at launch.

So, were old games more loved?

No. Put on equal terms, old and new games come out about level, and in both eras games were loved most when they were new.

Every gap people point to has something else behind it:

• Newer games’ higher first-year scores line up with a change in who writes reviews.
• Older games’ QG Score lead goes away when every game gets the same weight.
• Older games’ small lead at the same age is about the size of the yearly slide, and the survivors flatter them.

None of these gaps can be pinned on the games themselves. And I see no sign of movie-style inflation on Steam since 2020. First-year ratings for new games barely moved, and ratings of established games slipped a little each year.

How to read a game’s score

None of this means scores are useless. It means they need reading, not just glancing at.

Check when the reviews were written. A score built mostly before late 2019 came from a different crowd from one built since.

Don’t judge a game by its launch alone. Launch reviews are the kindest, and some games change a lot after: Monster Hunter: World climbed from 66.4% to 95.0% within its first year, while Starfield fell from 64.6% to 40.6%.

For older games, look at the recent reviews. Ratings slip a little every year, so recent reviews tell you more about how a game holds up now.

Treat a big review count as evidence, not quality. Lots of reviews means we know more about a game, not that it’s better.

The games we remember

Every year still produces games players love. Stardew Valley, Valheim, Elden Ring, Lethal Company and Black Myth: Wukong were each recommended by more than nine in ten engaged players in their first year.

The games we remember are the ones that lasted. The ones that didn’t, which had mostly started with lower ratings, faded from the reviews as well as from memory. Nostalgia isn’t wrong about the games we loved. It just forgets the rest.

So don’t judge a game by its launch week, or by the glow it leaves years later. Its real verdict is written slowly, by the players who are still there months and years on. Time reveals the difference between the games that last and the ones that blur. It also leaves evidence.

What this can’t tell us

• These are games in our data and people who wrote reviews, not all players or all releases.
• Deleted reviews and games removed from Steam are missing. Edited reviews show their current verdict.
• Several comparisons were designed after seeing earlier results, and the data has been examined before.
• Game age, calendar year and launch year can’t be fully separated.

Full results are available on request.

Resources

1. QG Score methodology: quantgamer.gg/methodology/qg-score
2. Previous articles: “A 78 on Metacritic Crashed a Stock by 30%. The Problem Isn’t the Number”, “What 18,000 PC Games Taught Me About Building Better Rating Systems” and “Nobody Outside Valve Knows How Long People Play Games”
3. Data source: Steam review data (playtime at review); Valve’s announcement of the review prompts
4. Dataset: 40.7 million Steam reviews across 26,538 PC games, through August 2026
5. Explore scores and playtime estimates at quantgamer.gg

Were Old Games More Loved? was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.

How long does it take to play Final Fantasy XIV?

I can give you three honest answers. The median player puts in 277 hours. The trimmed mean says 685. The raw mean says 2,960, because a small crowd of lifers has ten-thousand-hour profiles and pulls the average into another world.

All three numbers come from the same distribution. None is wrong. But publish any one of them without understanding why they disagree, and you mislead someone.

That disagreement is not an FFXIV quirk. It appears in over 40% of the PC catalogue. It is why most published playtime numbers deserve less trust than they get.

Why we refused the existing numbers

When we started building a scoring platform for PC games, playtime was non-negotiable. You cannot assess value per hour without an hour estimate. You cannot describe engagement without knowing how long people stay. When we went looking for a source we could defend, we found an industry running on a ghost.

For a decade, the default source has been SteamSpy. In 2018, Valve removed the API endpoint it scraped. SteamSpy kept publishing estimates. The industry kept citing them. Wrappers and analytics products kept repackaging them, often because there was no obvious replacement.

That left a hole where the method should have been. No visible uncertainty. No way to cross-validate. No honest account of where the numbers came from.

The alternatives were limited in different ways. HowLongToBeat covers roughly a tenth of the catalogue and measures story completion, not actual play. Public Steam profile sampling is rate-limited, mostly private, and skewed toward power users.

We could have wrapped one of those sources like everyone else. But building on numbers you cannot verify makes you a reseller of someone else’s guess. The guess kept justifying the caution. Partway through this project, we saw SteamSpy intermittently return zeros for games with millions of players.

So we built our own.

The raw material was already in our database. Every Steam review records the hours the reviewer had played when they wrote it. Our collector had accumulated 37.4 million of these across 99% of a catalogue of just over 20,000 games.

Review data has an obvious bias. People who bounce off a game in twenty minutes rarely write about it. But that is one bias, visible and modellable, versus a black box of unknown ones. I would rather model a known distortion than inherit an undocumented one. To be precise about what that means: what we publish is a modelled engaged-player estimate inferred from reviewer-hour distributions, not a claim that reviewers are a census of every owner.

The plan was simple to state and harder to execute: fit a Bayesian model per game and publish full distributions with honest uncertainty, not point guesses.

We went through three model families before one survived: a lognormal, a Student-t, and finally a mixture. The first two failed in useful ways. Both failures pointed at the same root cause the FFXIV numbers had already exposed.

Playtime is not one population.

The model that missed by 43%

We started with the obvious choice. Playtime is positive and right-skewed, so fit a lognormal to each game’s review hours and read the median from the posterior.

We tested it on two reference games chosen to be opposites. Vampire Survivors: 16,000 engaged reviews, a textbook right-skewed hump. Arx Fatalis: 1,800 engaged reviews, and a shape with two humps, one for completionists and one for tourists.

Vampire Survivors fit cleanly. Arx missed its empirical review-hours median by 43%. Not noise. Wrong by half.

My first diagnosis was heavy tails. A lognormal is thin-tailed on the log scale, so I swapped in a Student-t likelihood and let the fat tails absorb the second hump. Standard move. It did nothing. The fitted degrees-of-freedom parameter came back near 30, which is a Student-t politely telling you it wanted to be Normal all along.

I was wrong. Tail weight was never the problem.

The real problem was visible in the raw histogram the whole time. Arx’s review hours contained two populations answering two different questions. One cluster reviewed at around two hours: people who tried the game, formed an opinion, and left. Another reviewed at 20 to 200 hours: people who actually played it.

A single distribution fitted to both is a compromise that describes neither. The short-timers were dragging the estimated median toward the floor.

The fix was not a fancier distribution. It was a population definition. Steam refunds purchases under two hours of playtime, which gives the catalogue a natural behavioural boundary. Below 120 minutes, you are mostly sampling the decision to leave. Above it, you are mostly sampling the decision to stay.

So we split. Reviews under 120 minutes became their own signal, a per-game bounce rate modelled as a simple proportion. The playtime model fitted only the engaged population above the threshold.

Arx’s error dropped from 43% to 17.5%. Better, not solved.

That residual mattered more than I understood at the time. The two-hump structure had not gone away. It had moved above the threshold. Completionists and long-haul players are also two populations, and a single-mode model fitted to both was still splitting the difference.

The lesson stuck: when a point estimate misses by 40%, interrogate what population you are modelling before you upgrade how you are modelling it. Distribution-family fixes are seductive because they are one line of code. Population fixes require admitting the data means something different from what you assumed.

Scaling to 10,000 games

With the population defined, the production model was deliberately boring: a hierarchical Bayesian Normal model on the log playtime of engaged reviews. Games borrowed strength from their genre; genres borrowed strength from the catalogue. Variational inference throughout, because sampling-based methods at 10,000 games would take weeks, and this needed to run monthly on a schedule.

We also capped every game at 5,000 engaged reviews, downsampled uniformly. Past a few thousand observations, the posterior barely moves. The cap stops a 500,000-review game costing 100 times more compute than it returns in precision.

The bulk fit produced posteriors for 10,276 games. Point accuracy against held-out reviews came in at a log-scale mean absolute error of 0.064, a 74 to 84% improvement over a genre-average baseline. Rank correlation with HowLongToBeat, on the subset both cover, jumped from 0.19 for the baseline to 0.75.

For most of the catalogue, the model was doing exactly what we wanted.

Most is not all.

The discovery: two play modes are the norm

Validation kept flagging one cohort of games where the estimates ran hot or cold in a correlated way. We built a simple asymmetry metric, the distance between the mean and the median of log playtime, and ran it across the catalogue.

The split came back 56.6% symmetric, 43.4% asymmetric.

Nearly half the catalogue has the Arx shape. Not a handful of MMOs and roguelikes. Thousands of games where the engaged population itself contains two distinct modes: people who play the game to its natural end and stop, and people for whom the game becomes a place they live.

That reframed the problem. We had been treating bimodality as an edge case to engineer away. It was the terrain.

A story-driven RPG has finishers at 40 hours and modded-replay devotees at 400. A survival game has a base-building cohort and a thousand-hour server-community cohort. Once you see it, you cannot unsee it. No single-mode distribution will describe these games honestly.

We shipped v1 anyway, with the limitation documented rather than hidden: calibrated where the single-mode assumption holds, a stated projection where it does not. That documented limitation wrote the specification for the next version.

Which number do you publish?

Before the next model, we had to answer a question that was not really about inference. It was about honesty. When a distribution has two modes, which summary goes on the page?

We seriously considered the trimmed mean, the average of the middle 80% of the distribution. It is robust to tails and does not sit inside one mode of a bimodal shape. Statistically defensible.

Then we looked at what it would actually print for FFXIV: 685 hours as “typical playtime.”

Every reader who knows the game would compare that with their own experience and conclude the site was broken. The median, 277 hours, sits inside the largest cluster of real players. It is the number most people can locate their own experience near.

That settled a principle worth stating plainly: robustness to your model’s weaknesses is not a reason to publish a number. The summary you publish should answer the question a reader is actually asking. “How long do people typically play” is a median-shaped question.

So we publish the median as the headline. The trimmed mean and percentiles sit alongside it for anyone who wants the fuller shape. Bounce rate is published separately, so the leavers stay visible instead of being blended into a number they do not belong in.

Fitting the mixture

The proper fix for a two-mode population is embarrassingly direct: model two modes. Version 1.1 replaced the single lognormal with a two-component lognormal mixture, fitted per game, with the published median read from the full mixture distribution.

We ran a bake-off on a validation set: the v1 single-mode model against two mixture variants. On the asymmetric cohort, the games v1 could only project onto, median error fell from 20.3% to 7.7%.

That mattered. The hardest cohort landed at the same accuracy the single-mode model achieved on its easiest. Systematic bias collapsed to roughly zero.

The symmetric cohort improved too, from 7.6% to 5.6%, because a mixture can relax into one component when the data only has one mode.

The census the mixture enabled was its own result: among the games it examined, 72.4% show two materially separated play modes. That number is higher than the earlier asymmetry split because the mixture can detect separated modes even when their combined shape is not strongly skewed.

The finisher-and-devotee structure is not a quirk of live-service games. It is close to the default shape of how people play.

One more validation is worth reporting because it surprised me: playtime distributions are extraordinarily stable.

We tracked the empirical distributions daily for 37 days. Rounded to two decimals, day-over-day median movement was 0.00% across every volume tier. Once a game has accumulated a few hundred reviews, how long people play it is close to a physical constant of the game.

That stability is what makes a monthly refit schedule defensible rather than lazy.

What transfers beyond games

Playtime was the excuse. The lessons are general, and they are the ones I would carry into any domain with user-behaviour data.

When a point estimate misses badly, suspect your population definition before your distribution family. Our 43% miss was never a likelihood problem. It was two populations wearing one label, and the field recording it gave no warning.

Look for natural behavioural boundaries before inventing statistical ones. The 120-minute refund threshold was sitting in Steam’s policy the whole time, cleanly separating “decided to leave” from “decided to stay.” Most domains have one: the free-trial cliff, the first-week churn line, the single-visit patient.

Assume mixtures until proven otherwise. We found two material play modes in nearly three-quarters of the games we examined. If your users can relate to your product in more than one way, your engagement metric is probably averaging across populations that should never have been averaged.

Keep the modelling question separate from the publishing question. The best-fitting summary and the most honest summary for readers are not always the same number. FFXIV taught us that 685 hours can be statistically sound and editorially false at the same time.

The engine now refits monthly, covers 10,000+ games with full posterior distributions, bounce rates, and confidence tiers, and every number carries uncertainty you can inspect. The methodology is published in full, including what the model explicitly does not claim.

That last part is the habit I would defend hardest. Publishing the boundary of your model’s competence feels like weakness. It is the opposite. The documented limitation in v1 became the specification for v1.1, and the numbers people can trust are the ones that admit what they do not know.

Methodology and live estimates: quantgamer.gg/methodology/playtime-engine. Dataset: 37.4M Steam reviews across roughly 20,000 PC games; 10,000+ fitted posterior distributions.

Resources

1. Playtime engine methodology and live estimates: quantgamer.gg/methodology/playtime-engine
2. Previous articles:
3. Previous articles: “A 78 on Metacritic Crashed a Stock by 30%. The Problem Isn’t the Number” and “What 18,000 PC Games Taught Me About Building Better Rating Systems”
4. Data source: Steam review data (playtime_at_review); rank-correlation benchmark: HowLongToBeat
5. Dataset: 37.4M Steam reviews across roughly 20,000 PC games; 10,000+ fitted posterior playtime distributions.
6. Explore playtime estimates and scores at quantgamer.gg.

Before you go

• Please take a moment to like the post and follow the writer!
• Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute here

Nobody Outside Valve Knows How Long People Play Games. So We Modelled It. was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.

Update, July 2026: This article describes the scoring model as it stood at the time of writing (internally, v8). QuantGamer’s methodology has since moved on. The current model, v9.5, scores games on three different pillars: Reception (45%), Engagement (35%), and Durability (20%). A single continuous evidence multiplier replaced the old confidence handling, and a Bayesian playtime estimation engine replaced third-party playtime data. The design principles in this piece survived the rewrite intact. Exposing uncertainty, refusing unfair baselines, separating what players say from what they do. The machinery that implements them did not. The current methodology is documented in full at quantgamer.gg/methodology.

Most rating systems look objective because they reduce a messy market to a clean number.

A product is 4.4 stars. A game is 78 on an aggregate site. An app is “Mostly Positive.” Once a number becomes familiar enough, people stop asking how it was produced and start treating it as ground truth.

The problem is not that single-number scores are useless. It is that they compress away exactly the information that matters most: how much evidence exists, whether products are comparable in the first place, and whether the thing being measured has changed since the number was assigned. Earlier this year, a Metacritic score of 78 was widely blamed for a 30% stock drop at a Korean game studio. The number looked fine. The market disagreed. The limitations of simple aggregation are not theoretical anymore.

I ran into this problem while building a scoring system for PC games. On the surface, games seem like an unusually noisy domain: different storefronts, different prices, different genres, and products that can change dramatically after launch through patches and content updates. But that is also what makes them a useful stress test. If you can build a robust scoring system here, the lessons transfer surprisingly well to any domain with multi-source reviews, sparse long-tail data, and changing products.

Our dataset currently tracks roughly 20,000 PC games. About 18,000 have enough data to score. And one subset in particular exposes how broken naive rating logic can be: the roughly 10,000 scored games with at least 100 player reviews.In that group, 42% landed in the lowest grade band. Only 12% reached B or above. Just 22 games — 0.2% — qualified for the top grade.

The first failure mode: sparse data is not just “small data”

The cleanest way to see the issue is by review count. When I split scored PC games by how many reviews they had, the failure rates were extreme: titles with fewer than 20 reviews had a 100% F-rate. At 20–99 reviews, 95%. At 100–999, still 66%. It was not until the 1,000–9,999 tier that the rate dropped to 21%, and only at 10,000+ reviews did it fall to 3%.

That gradient should make any data scientist uncomfortable.

The easy interpretation is that obscure games are bad. There is some truth in that, the long tail of any digital marketplace contains a lot of low-effort supply. But review count is doing at least three jobs at once. It is a proxy for quality, because better products earn more attention. It is a proxy for discoverability, because products with strong distribution get more chances to accumulate evidence. And it is a proxy for confidence, because a score built from 15 reviews and a score built from 15,000 should not be treated as equally stable, even if the arithmetic mean is identical.

This is where many rating systems quietly fail. They present a point estimate without communicating how fragile it is. A title with 12 reviews can look precise because the interface only shows one number. But that precision is cosmetic.

For marketplaces, this matters because the long tail is where naive scoring becomes self-reinforcing. Low evidence produces noisy scores. Noisy scores reduce visibility. Reduced visibility suppresses future evidence. The score stops being a measurement and starts acting as a gate.

The second failure mode: price confounds everything

The next surprise came from pricing. If you ask consumers whether gaming is getting more expensive, most people picture £50–£70 releases. But when I grouped PC games by their original GBP price, the worst-performing segment was not premium pricing. It was budget pricing.

Free games led the dataset with an average score of 63.9 and just a 4.1% failure rate across 369 titles. Budget games — anything under £10 — were the worst segment on the platform: an average of 33.2 and an 81.3% failure rate across 10,950 titles. The mid-tier (£10–£30) averaged 46.8 with a 51.4% failure rate across 5,761 games. Standard-priced titles (£30–£50) averaged 54.4 with a 31.6% failure rate across 678 games, and premium titles (£50+) averaged 52.8 with 31.8% across 107 games — nearly identical to standard despite costing more.

The real lesson here is that cheap is not the same thing as value.

Budget PC games are full of low-quality releases, abandoned experiments, asset flips, and products that are inexpensive because they cannot sustain a higher price. Free games, meanwhile, are a completely different economic model — the best of them optimise relentlessly for engagement and retention, which makes them look strong on player outcome signals even when their monetisation is doing something entirely different from a paid title.

Price cannot be treated as a neutral attribute. It changes the expectation set, the kind of experience players tolerate, and what “good value” should mean. A £7 game that delivers two decent hours may be perfectly fine. A £60 game with the same outcome is not.

This is why raw cost-per-hour comparisons also fall apart. A 100-hour game is not automatically better than a 12-hour one. Length without quality is just time consumption. So instead of evaluating games by absolute price or absolute duration, we ask whether a game delivers value relative to products in a similar price neighbourhood — typically a ±30% band around its price point. The first version of the value model did not do this normalisation, and the result was basically a duration leaderboard. Long, cheap games dominated regardless of whether anyone actually enjoyed them.

The third failure mode: static scores age badly

The last failure mode is time.

Many products are not static after release, but games make this impossible to ignore. They ship with bugs, receive patches, add content, rebalance systems, and sometimes become materially different products months after launch.

A frozen aggregate struggles in that environment.

One recent example made it obvious. When Crimson Desert launched in March 2026, it sat at 78 on Metacritic. Player reviews on PC platforms told a different story. Our own daily data shows reviews starting at 74% positive in its first days and climbing steadily to 84% by day 26, with 132,000 reviews accumulated. The Metacritic score never moved.

If the thing being scored changes over time, a static average over all historical opinions is answering a different question from the one users think they are asking. It tells you the blended sentiment across the entire lifespan. It does not tell you what condition the product is in now. For products that evolve, those are different targets, and a scoring system should let you distinguish them.

Building the scoring system

The system we ended up with has three components. Quality carries the most weight at 40%, Value at 35%, and Behaviour at 25%. Those are the base weights — in practice, the effective balance shifts depending on how much data is available and how the components interact. For example, if Quality is very low, the Value component is dampened, because it does not make much sense to call a bad game a “good deal.”

The decomposition matters less than the design principle behind it: do not force fundamentally different signals to masquerade as one thing.

For Quality, we aggregate player reviews from multiple PC storefronts — primarily Steam, supplemented by GOG and Microsoft Store data for the 987 games where multi-source reviews are available. We use inverse variance weighting, where each source contributes based on both its sample size and the tightness of its consensus. In practice, this means a platform with 50,000 reviews and strong agreement will dominate, but a smaller platform with very tight consensus can still move the needle. The goal is not to override the big platforms, but to prevent them from drowning out meaningful signal from smaller ones.

For 502 games with twelve or more months of review history, we also apply a Kalman filter to smooth temporal fluctuations. The filter treats quality as a changing state rather than a fixed label — new evidence updates the estimate without overreacting to every short-term spike. This is the same family of techniques used in GPS tracking and financial signal processing: estimating a true position from noisy observations.

For Value, we use completion-time data from HowLongToBeat, available for roughly 2,778 scored games. But rather than ranking by raw cost-per-hour, we normalise against peer groups within similar price tiers. A £50 game is compared to other £50 games, not to a £5 indie. The question becomes not “how long is it?” but “does it deliver more or less than you would typically expect at this price?”

For Behaviour, we incorporate engagement signals: playtime distributions, achievement completion, and retention patterns. This is the reality-check layer. Reviews tell you what people said. Behaviour shows what they actually did. We have seen games with 90%+ positive reviews where median playtime is under two hours, and games with mixed reviews where the engaged players put in hundreds of hours. Without behavioural signals, both cases look identical in a review aggregate.

What we tried and rejected

Several approaches that looked promising in design failed in practice.

We initially tried a tau-squared approach for weighting reviews across storefronts. In medical meta-analysis, tau-squared is a standard way to quantify heterogeneity between studies. We hoped it would elegantly handle the variance between, say, Steam and GOG audiences. The problem was practical: because Steam’s sample size so massively dominates the others, the model consistently concluded that the smaller platforms added no statistically significant information. Technically correct, but it defeated the purpose. We wanted multi-source resilience, not a formal proof that Steam is big. Inverse variance weighting gave us a better balance between respecting volume differences and actually using the smaller sources.

We tested community calibration votes — letting users upvote or downvote the algorithmic score. This created a feedback loop almost immediately. Users overwhelmingly voted on games they already had strong feelings about, which meant the votes amplified existing sentiment rather than correcting for blind spots. We permanently excluded community voting from the composite.

We experimented with a sale bonus that boosted Value when a game was heavily discounted. The scores became unpredictable — a game would jump five points during a Steam sale and drop back the following week. A game does not become better when it goes on sale. It becomes a better deal. We removed the bonus and display deal information separately instead.

What the combined view reveals

Once you score products across three dimensions, interesting patterns emerge at the portfolio level.

Among publishers with at least 10 scored games, Valve leads with an average of 69.8, followed by Rockstar Games at 68.8 and Xbox Game Studios at 67.8. SEGA sits at 65.9, Devolver Digital at 64.2, Capcom at 62.1. At the other end, Square Enix averages 57.6 with a 27.4% F-rate, and Paradox Interactive averages 55.8 with 26.8%. All of these sit well above 50.7 — the platform average, which is dragged down by the thousands of low-quality titles flooding the long tail — but the spread across major publishers is significant.

The bigger lesson is methodological. Once you can decompose a score, you can ask why a publisher lands where it does. Is the issue product quality? Weak value relative to price? Strong launches with poor retention? A single number can rank publishers, but it cannot diagnose them.

What I would take from this outside gaming

PC games are a convenient stress test because the data is messy in all the familiar ways: multiple sources, uneven samples, changing products, confounded prices, and behavioural signals that do not align with stated opinions.

If you are building a marketplace score, recommendation signal, or quality ranking in another domain, there are a few questions worth importing:

Are you exposing uncertainty, or hiding it behind a clean-looking number?

Are you comparing products on a fair baseline, or letting price and visibility distort the result?

Are you measuring historical reputation or present condition?

And are you relying only on what users say, or also incorporating what they do?

What surprised me most in this project was not that PC games are noisy. It was how quickly a domain-specific problem turned into a general measurement lesson. The moment you move beyond a small set of highly visible products, simple averages start to break. The long tail becomes sparse and uncertain. Price stops being neutral. Time starts to matter.

A useful rating system is not the one with the cleanest number. It is the one that loses the least on the way to producing it.

Resources

1. Scoring methodology and live scores: quantgamer.gg/methodology
2. Previous article: “A 78 on Metacritic Crashed a Stock by 30%. The Problem Isn’t the Number”
3. Data sources: Steam, GOG, Microsoft Store, SteamSpy, HowLongToBeat, IGDB
4. Dataset: ~18,000 scored PC games; ~10,000 with 100+ player reviews scored across Quality, Value, and Behaviour dimensions.
5. Explore scores for 18,000+ PC games at quantgamer.gg.

What 18,000 PC Games Taught Me About Building Better Rating Systems was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.

In March 2026, Crimson Desert opened to a 78 on Metacritic. “Generally Favorable” is the platform’s term of art for that band. The market read it differently. Pearl Abyss lost roughly 30% of its value in a single session, with trading interruptions on the Korean exchange as the stock fell. Days later, commercial data began to arrive: the game sold more than three million copies, posted a Steam peak well above 250,000 concurrent players, and the stock recovered once investors had something better than critic averages to work with.

That sequence is not interesting because 78 was obviously wrong. A 78 is a perfectly defensible summary of how that critic pool felt in that review window. What’s interesting is that a single frozen number, produced by a system launched in January 2001 and still built around weighted critic aggregation, remains the market’s default shorthand for game quality.

QuantGamer scores Crimson Desert lower, at 73 (a B), but the decomposition is where the information lives. Quality at 76 says players broadly like it, with 83% positive sentiment across more than a hundred thousand Steam reviews. Value at 47 says £54.99 is steep for what the content delivers structurally. Behaviour at 70 says engagement is healthy so far, with strong early median playtime. The score is also flagged as provisional: the game is eighteen days old, and behavioural data is still accumulating. Full analysis takes two to four weeks. That flag is the point. Metacritic also scored Crimson Desert in its launch window. It just doesn’t tell you the number is preliminary. And once critics stop reviewing, the score stops moving. No mechanism exists to reflect what a game becomes after launch.

The problem is not that Metacritic exists. The problem is that a frozen number built from averaged critic opinion is still the best tool the industry has.

The Structural Problem

Metacritic’s public methodology is unusually blunt about what it is. A METASCORE is a weighted average of critic reviews converted onto a 0–100 scale, where some publications count more than others based on what Metacritic describes as their “stature.” The exact weights are not disclosed. When a Full Sail University researcher attempted to reverse-engineer them at GDC in 2013, Metacritic called the findings “wildly, wholly inaccurate” but offered no counter-disclosure.

For games specifically, Metacritic keeps the first review and first score it records from a publication and does not update that score later, even if the publication revises its own review. The stated rationale is that publications have been pressured to raise scores after initial publication. That policy exists for understandable reasons. It still means the system is structurally frozen at launch. A photograph of a building that has since been renovated, condemned, or demolished, still pinned to the wall as if nothing changed.

The consequences are not abstract. Metacritic scores have been contractually tied to developer compensation. In one well-documented case, a studio missed a royalty threshold by a single Metacritic point on a game that earned hundreds of millions in revenue. The studio conducted layoffs the following year. When a number carries that kind of weight, the architecture producing it should be transparent and defensible. Metacritic’s is neither fully transparent nor designed to reflect what a game becomes after launch.

That design made sense for boxed products sold into a relatively static market. It makes less sense for games that change materially after release. A game that launches broken and is repaired over two years carries the launch scar forever. A game that degrades through neglect, aggressive monetisation, or a collapsing multiplayer ecosystem keeps the benefit of its opening reviews. The number is stable because the system is stable, not because the underlying object is stable.

And it is one-dimensional. A £70 game and a £5 game are evaluated on the same axis. A tightly edited ten-hour campaign and a sprawling eighty-hour sandbox are evaluated on the same axis. A game people install and abandon after three hours and a game people actually inhabit for months are evaluated on the same axis. Approval is visible. Price fairness is invisible. Engagement is invisible.

OpenCritic, founded in 2015 as a more transparent alternative, improved some of the presentation problems. It surfaces a Top Critic Average, a Critics Recommend percentage, and a percentile rank. Its weighting is equal rather than hidden, which is cleaner. But it is still critic-only, still frozen around review cycles, and still one-dimensional in the way that matters here.

The critic pool itself is getting thinner. The global pool of games journalists has contracted significantly in recent years, with thousands of departures since late 2023. Any system that depends on that pool inherits its contraction risk. The aggregation layer becomes more central at exactly the moment the underlying observation layer becomes smaller.

Why Simple Fixes Fail

The obvious response is to move from critics to players. Steam reviews are better in several important ways: they are tied to ownership, they operate at far larger sample sizes, and they capture a population that actually spent time with the product. But they are still one-dimensional. A thumbs-up from a player with three hours and a refund is the same atomic signal as a thumbs-up from a player with two hundred hours and a completed run. The system knows whether someone approved. It does not know the depth of that approval.

Simple averages also ignore uncertainty. Ninety-five percent positive from a hundred reviews and ninety-five percent positive from a hundred thousand reviews are not the same thing. One is a single thermometer. The other is a thousand weather stations. Treating them identically is statistically careless.

And neither critic aggregates nor player aggregates solve a third problem. They say whether people liked the game. They do not say whether the asking price is fair for what the game actually delivers. A ten-hour game at £50 and a sixty-hour game at £10 receive identical treatment. A score that cannot distinguish between quality and value is not useless. It just answers a narrower question than most people think.

The Three-Component Architecture

The QuantGamer Score starts from a different premise. A game is not a verdict. It is an organism: growing, decaying, repriced, patched, abandoned, revived. The score should behave accordingly. That means treating it more like a financial analyst treats a stock, dispassionately and quantitatively, with explicit assumptions and reproducible calculations.

The proposition is divided into three components: Quality at 40%, Value at 35%, and Behaviour at 25%. These are measured independently, then combined into a composite score on a 0–100 scale mapped to a letter grade from S (87+, deliberately rare) through A, B, C, D, to F. The separation matters. Not everything belongs inside the number, but the things that do belong should not be collapsed into a single undifferentiated average.

Quality measures player sentiment across Steam, GOG, and Microsoft Store. The merge uses inverse variance weighting, where every platform contributes in proportion to the evidence it carries. There are no caps and no prestige adjustments. A review is a review. Steam dominates naturally because it has the largest volume; smaller sources still matter when they exist. Sample-size uncertainty is handled with Wilson confidence intervals before blending, so the system discounts fragile signals without pretending they are equal to large, stable ones. The full formula belongs on the methodology page, not in the body of an article. The principle is simple: sentiment is useful, confidence matters, and cross-platform evidence should be aggregated statistically rather than editorially.

Value measures price fairness, not discount opportunism. It is anchored to the original permanent price, not the current sale price. A game does not become better when it goes on sale. It becomes a better deal. For titles with HowLongToBeat coverage, the model incorporates cost per hour, normalised within price-tier peer groups: a £50 game competes against other £35–65 games, not against a £10 indie with completely different pricing logic. Without that step, any value system collapses into a tax on premium pricing.

Behaviour uses playtime data from SteamSpy, and it asks a different question from sentiment. How deep is median engagement? How close is median playtime to average playtime? Do most players get similar value, or do a few outliers inflate the average while the median sits near the floor? These are imperfect measurements, but they capture something critic aggregates and review percentages cannot. Actual play patterns are harder to manipulate than opinions.

One adjustment sits across the top of the architecture. When Quality exceeds 80, Value’s effective weight is reduced through a sigmoid function and the freed weight moves to Quality rather than Behaviour. This exists because short, exceptional games are structurally penalised by raw efficiency metrics. A three-hour masterpiece can be poor cost-per-hour and still be a sound proposition. Pure arithmetic does not understand that on its own.

Temporal Intelligence

The centre of the model is time.

Every signal is first classified by how fast it changes. Durable signals do not decay: base price, content hours, genre, achievement support. Slow-evolving signals change over months: review sentiment, playtime patterns, update cadence, DLC pricing patterns. Fast-transient signals change daily or weekly: current sale price, seven-day review spikes, short-term outages.

Only durable and slow-evolving signals enter the core score. Fast-transient signals appear on the game page as context and purchase guidance but do not contaminate the composite. This is what we call the Temporal Contract. It enforces a clean separation between “what this game is” and “what is happening around this game this week.” Without that separation, the score becomes a weather report.

Quality is where temporal intelligence matters most, because quality can genuinely change. The tool used here is a Kalman filter, a mathematical technique widely used in navigation, control systems, and finance for estimating the true current state of something from noisy observations over time. The same mathematics that guides a spacecraft through course corrections can guide a game’s score through noisy monthly review data.

Review data is noisy in ways that simple averages cannot handle cleanly. A single bad month can be a real warning or just a review bomb. A single good month can be a genuine turning point or just a viral spike. A cumulative average treats every past observation as equally alive forever. A Kalman filter does not. It gives recent months more influence, smooths one-off shocks, handles missing observations gracefully, and lets sustained improvement or decline flow through at a measured rate.

The result is a scoring system with aligned incentives. We call this the Lighthouse Principle. A studio improves its score by making a better game, pricing it fairly, and supporting it after launch. It cannot improve its score by running a sale, timing reviews carefully, or manufacturing praise on one platform for a weekend. That does not make the model immune to manipulation. It makes manipulation more expensive and less durable.

Cyberpunk 2077

Cyberpunk 2077 is the cleanest demonstration of why this matters.

The launch in December 2020 was chaotic. The PC version was in better shape than the console versions, but the broader reception was dominated by unmet expectations, technical issues, and a flood of negative sentiment. A frozen aggregate captures that moment accurately enough. It does not capture what happened next.

By 2026, Cyberpunk’s cumulative all-time player sentiment sits at 86%. A respectable number, but a compromised one, because it permanently carries every launch-month review at equal lifetime weight. The Kalman-filtered estimate, working month by month across the full review history, lands at 94.6%. That is not a marketing correction. It is an estimate of the game’s current quality after years of patches and the Phantom Liberty expansion.

After genre normalisation and multi-source sentiment merge, Cyberpunk’s Quality score comes out at 88. Value is lower at 63, because a £49.99 permanent price against a roughly twenty-six-hour main story is not exceptional on structural value alone. Behaviour scores 85, driven by deep engagement (roughly fifty hours of median playtime) and the fact that a very large player base kept playing rather than bouncing off. With quality-gated value dampening and a high-confidence multiplier, the final composite lands at 86, an A.

Metacritic’s frozen critic score for Cyberpunk also sits at 86. The similarity is superficial. Metacritic’s 86 has been frozen since December 2020. QuantGamer’s 86 was recalculated today from current data: current player sentiment, current price structure, current engagement patterns. Same integer. Different ontology.

Without the Kalman layer, the score would be lower. More importantly, it would be lower for the wrong reason, because the model would have confused historical memory with present condition. The point of temporal intelligence is not to forgive bad launches. It is to stop pretending that a game in 2026 is identical to the game it was in 2020.

S-tier is deliberately rare. Fewer than 1% of scored games reach it. The median sits in the low 60s. Cyberpunk’s 86/A places it well above the median, but below the handful of games that deliver exceptional quality, value, and engagement simultaneously.

Limits of the Model

No scoring system gets to claim final authority. This one does not.

Kalman coverage is still partial and expands only as enough monthly history accumulates. Playtime data includes idle time, which can overstate engagement for games that are often left running. Free-to-play titles have a structural Value advantage because zero price is a real fact even when monetisation design is doing other work off to the side. Review manipulation can still move the score if it is large and sustained enough. At that point it is no longer noise. It is part of the observed state.

The model is PC-first by construction. Steam provides the richest behavioural and review data surface in the market. GOG and Microsoft Store improve sentiment coverage but do not replace Steam’s centrality. There are genres where the benchmarks are still too coarse, sub-genres where peer groups should be finer, and classes of games where content hours remain a bad proxy for delivered value.

None of that is hidden. Transparency is the methodology’s trust mechanism. We publish these limitations because credibility depends on honesty about what the methodology can and cannot measure.

One person built this. AI agents explain how a solo founder can maintain a daily scoring pipeline across the Steam catalogue without pretending to be a studio. They do not explain the model. The methodology does. The useful part is not that the system exists. The useful part is that its assumptions are explicit.

The full formulas, data sources, and methodological notes are published at quantgamer.gg/methodology. The scores are deterministic, reproducible, and updated daily.

A 78 on Metacritic Crashed a Stock by 30%. The Problem Isn’t the Number. was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.