I Built an AI Review Advisor With Claude in 3 Days. It Won Best Use of Tiger Data at Hacktoberfest 2026.
The announcement
On October 23, DEV posted the winners of the Hacktoberfest Open-Source AI Challenge, Week 2: Side Hustle. I scrolled past the overall winner, past the featured categories, into the partner prizes. And there it was, under Best Use of Tiger Data: my project, Perforvisor 🤩.
The prize is $100 and an exclusive DEV badge. That’s not life-changing money. But it was the first time something I built from scratch, in three days, was picked out of a crowd of entries by people who judge this stuff for a living. And I built almost all of it by talking to an AI agent.
This is the story of how that happened.
Hacktoberfest is different this year
If you remember Hacktoberfest as “open four pull requests, get a T-shirt,” 2026 threw that out. Run by MLH and DEV with DigitalOcean as presenting partner, this year’s theme is “AI belongs to everyone.” Pull requests no longer count. Instead, you build brand-new projects with open-weight models and open-source AI, write about them on DEV and collect virtual stickers along the way.
DEV runs five challenges back to back through October: a weekend challenge, then a new themed week every Monday. I entered like everyone else,and then kept going. I took part in every DEV challenge and pretty much everything else Hacktoberfest offered.
Week 2 ran October 12 to 18. The theme was Side Hustle: build something with open-weight models that someone would actually pay for. There was $2,550 in prizes across 18 winners: one overall winner, six featured categories and eleven partner categories. Judges weighted the writing most, then theme fit, creativity and how well the thing actually worked.
The idea came from AI, then I bent it into shape
I’ll be honest: I didn’t walk in with a brilliant idea. I opened an empty project folder, pointed my AI agent at the challenge page and asked it for ideas.
It came back with a few: a local bookkeeping helper for freelancers, a contract checker, a listing writer for online sellers and a tool that drafts replies to Google reviews for small businesses. The review one caught my attention. But auto-replying felt thin. Anyone could copy it in a weekend. Also, my idea was to extend it to other platforms as well(like amazon, ebay, etc) and auto reply part would have become difficult.
What a small business owner actually needs isn’t a reply. It’s an answer to a harder question: what should I fix first? So I pushed the idea somewhere else. Instead of replying to reviews, read all of them, understand them and tell the owner what’s going on.
Then I trimmed it down to what could actually be built in three days. My first plan had Neo4j, Kubernetes manifests, a consumer app and a dozen cloud services. The version I shipped kept the core and dropped the rest. That cycle (ask AI, react to its suggestion, cut it down to reality) turned out to be the most useful part of the week.
What Perforvisor does
A cafe owner has hundreds of reviews and no time to read them. The star average says 4.1. It doesn’t say what to fix this week.
Perforvisor reads every review with a small open-weight model (Qwen3–4B) and gives the owner a short verdict, up to three concrete actions and the reviews behind each one. Along the way it:
- pulls out what each review talks about (food, service, wait time, cleanliness) and how the customer felt about each
- merges claims that say the same thing in different words into one knowledge graph
- tracks rating and sentiment over time and spots sudden bursts of reviews
- flags reviews you shouldn’t trust, like copy-pasted text or stars that contradict the words
- compares you with competitors and answers questions like “Is the wait long on weekends?” with the reviews cited
The moment I knew it worked came from a real cafe here in Indore. Perforvisor flagged a review that read ★★★★★ “Needs to be more clean.” Five stars, one complaint. A star-average dashboard counts that as a happy customer. Perforvisor read the words, not the stars and marked the rating as contradicting the text. That one review became the title of my submission.
The code is on GitHub. (The web app is currently offline because I removed the deployed AI to avoid the high cost of running it in production)
Built entirely with an AI agent
I built Perforvisor with Claude as my coding agent. It set up the monorepo, wrote the Fastify backend and React frontend, wired up LangGraph.js and ran the commands. My job was deciding what to build, reviewing what came back and steering when it drifted.
Get Kartik Agnihotri’s stories in your inbox
Join Medium for free to get updates from this writer.
That sounds like the AI did the work. It didn’t feel that way. The agent is fast at writing code and terrible at knowing which code matters. Every important decision in the project came from me measuring something and changing direction:
- I deleted the reranker. The agent’s first design used a small LLM to rerank search results. On my labelled questions it made results worse, scoring 0.63 against 0.86 for plain cosine similarity. It even scored “overheated on a flight” 2 out of 10 for the question “Does it overheat?” So it went.
- I let code decide the facts. Small models mixed up strengths and weaknesses when asked to list both. Now code pulls them straight from the classified claims and the model only writes the verdict and actions.
- I set the merge thresholds from data. Same-meaning claims scored 0.867 to 0.957 in similarity; different-meaning ones scored 0.772 to 0.912. The ranges overlap, so no single cut-off works. Above 0.93 merges automatically, 0.86 to 0.93 goes to the model as a yes/no question and below stays separate.
The agent made me ten times faster. It did not make me less responsible for the result.
The two problems that took the longest
I’d hosted a model on GCP Vertex AI before and, honestly, made a mess of it. This time I ran both models myself with vLLM on a single NVIDIA L4 GPU on GKE and I wanted to get it right.
The embedding model crashed on startup. EmbeddingGemma 2 kept killing the vLLM server. vLLM sizes some of its attention kernels for bigger GPUs like the A100 and H100 and the L4 has less shared memory per block. I patched the kernel file and mounted the fix into the container until it has fixed upstream. No AI agent guessed that one for me; it took reading the stack trace and the GPU specs.
Serverless functions have a time limit. The app runs on Vercel, where a function stops at 300 seconds. A first sync reads dozens of reviews through a model, which takes longer. So each run stops picking up new reviews at 240 seconds and queues a continuation. Every step skips work that’s already done, so the next run starts exactly where the last one stopped.
Why Tiger Data won it
- Search with pgvector. Every review gets one embedding and every claim gets one for finding merge candidates. Since retrieval is always scoped to one business, a simple index on the business plus an exact vector scan is fast and accurate at this scale.
- Trends with TimescaleDB. Every review is an event in a hypertable. Trend charts bucket them by day, week or month and detecting a suspicious burst of reviews is a single query.
- I tested and dropped hybrid search. Combining full-text and vector search scored 0.81 on my test set; cosine alone scored 0.89. Simpler won.
MongoDB Atlas held the records and the knowledge graph. Tiger Data answered the questions about when and how similar. Keeping that split clear made both easier to reason about.
What I learned
- An AI idea is a starting point, not a plan. The agent’s suggestion was a decent seed. The product only got interesting when I disagreed with it.
- Measure before you trust a design. The reranker sounded smart and made things worse. A tiny labelled test set (12 questions) was enough to catch it.
- Small open models are enough for narrow jobs. A 4B model pulling aspects and claims from one review at a time does the job and both models fit on one 24 GB GPU.
- Open models change the economics. Per-token pricing grows with every review read. A GPU I run is a flat cost every customer shares. At a proposed $9 a month, about 40 paying owners would cover an always-on GPU.
- Write it up properly. The judges weighted the writing most. Being concrete (real numbers, a real review, what I removed and why) mattered more than listing features.
- Keep showing up. I entered every challenge this month. Not every entry won, but each one made the next one better.
What’s next
Perforvisor is still small: one owner, one location, one question. Next on the list are a consumer side (“Is this place actually good at X?”, aware of which of the many restaurants called “Paradise” is near you), more sources like Amazon products and CSV imports, alerts when a burst of complaints starts and a fine-tuned extraction model.
Thanks to DEV, MLH and Tiger Data for running the challenge and congratulations to every winner and everyone who shipped something this week. If you run a small business and want to try Perforvisor on your own reviews, or want to contribute, check out the open-source code. The live demo is currently offline due to AI hosting costs.
And if you’re on the fence about entering the next challenge: enter. Three days, an AI agent and one stubborn idea can get you further than you’d expect.
Read the original submission on DEV: Perforvisor: a 5★ review said “Needs to be more clean.” Open models caught it. · See all winners: Week 2: Side Hustle winners
