Everyone’s got a quick question
And you shouldn’t have to answer them all.
There’s an interactive version of this story. Explore it →
I became the team’s search engine
Every team has a person everyone reaches out to. They know where things live, why a decision went the way it did, and who to ask when you’re stuck. If you’re on a team, you’ve probably messaged yours today. 😏
In Radical Candor, Kim Scott calls the steady, dependable people a team is built on its rock stars. Our team has a few too many sports fans, so we call ours the GOAT: the greatest of all time.
I lead a remote design team of fifteen, and a lot of the team’s questions find their way to me. Each seems small enough on its own, but the constant interruptions were starting to wear on me. 🫣
And apparently, there’s research behind that frustration: a 2008 CHI paper found that people often work faster to make up for interruptions, but feel more stressed and frustrated as a result.
I wondered how many of these interruptions were actually necessary. So I reviewed two and a half years of requests — 2,413 in total — and sorted them by what it took to answer. That led Bryan, my intern, and me to build a Slack agent to take some of them off my plate.
What was in my inbox
At my busiest, about nine housekeeping requests reached me every workday, close to fifty a week. Spread across a calendar, they filled almost every workday.
Some questions kept coming back
Plenty felt like déjà vu. 203 had been asked before, some more than a dozen times. The most repeated — how to access user study recordings — came up 32 times. My guess is that most repeats happened in DMs, where nobody else could see the answer. And even in public channels, how often does anyone dig through old Slack threads?
The wait by hour
Most of my designers are part-timers who work their own hours, often in the evening, while I keep a nine to five. During the day, I tried to get back to people as soon as I could, usually within about 15 minutes (allow me to feel proud of myself for a second 😎). Outside working hours, however, a question could wait 12 to 13 hours, until I was back the next morning.
Did these all need me?
Not really. After going through the cases carefully, I found that 53% didn’t need me at all. The answer already existed somewhere, in a doc, a Figma file or an old thread; people just couldn’t find it, or didn’t know it was there. Another 35% were mostly legwork, like digging up which spec was current or what a handoff thread had settled, before I signed off. The last 12% were the calls only I could make: someone’s role, their hours, who should help with what.
Most questions didn’t need my judgment. They just needed an answer
Could an agent answer them?
I’ll admit, I’d been quietly “cheating” on the easy questions: paste them into my own agent, check the answer, and pass it along in my own words. It worked often enough that I wondered, why not let the team ask directly? So Bryan and I built a quick Slack prototype. We called it le goat 🐐 (my fellow LeBron fans, iykyk).
Here’s what happened when I tried an easy one. A small server on Cloudflare picked up the mention and reacted with 👀, so whoever asked knew le goat was on it. It kept track of the thread and handed the question to Gemini, which looked things up in Notion, GitHub and the web, then wrote back in the same thread.
It worked, but a simple question could get a 639-word essay in return. I’d feel really guilty dropping that on someone in Slack. le goat needed work.
Making it a bot with manners
Before anyone else saw it, I taught it how to reply. Much of the interaction design was inspired by principles in Google’s People + AI Guidebook, especially around communicating uncertainty, handling errors, and keeping people in control. The challenge was making those principles feel natural in a Slack conversation.
- Show it’s working. A question used to meet four silent minutes, then the bot’s inner monologue. Now “Thinking” appears right away, with the steps folded under it and a Stop button.
- Get to the point. The answer used to hide in paragraph two. Now it comes first, in one line. A comparison becomes a small table, and counts become bars.
- Know what you don’t know. One of our developers loves quoting Socrates: real wisdom is knowing what you don’t know. Instead of a confident claim with nothing behind it, le goat now says what it couldn’t find and who would know, and its sources fold away under one tap.
- Leave the calls to people. Instead of editing our guidelines without asking, le goat drafts the change, like an update to a doc or a new card, and never makes it itself. The draft arrives on a card in Slack, and nothing changes until the right person approves it.
The MVP for everyday questions
With its manners in place, I scoped the MVP to the five kinds of questions people asked me most often: finding something (314 asks), catching up on what they missed (565), knowing who to ask (448), asking for design-system guidance (731), and turning a discussion into a card.
A tryout with the team
For the pilot, I invited the whole team to try le goat and ask it anything in Slack. And they kept asking: 228 questions in two weeks, averaging 31 a day in week two. That’s more than three times my busiest week.
They wanted it in the flow
The funny thing about giving people an agent they can ask anything: they start wanting it everywhere. Designers wanted le goat in Figma, where they spend their day, so we built that. Now they can @le goat in a Figma comment and get a reply right there.
From answering to asking
People were clearly using le goat. But I also wanted to know if I could trust its answers.
Competent, but not confident
So I checked 56 of its answers for hallucinations. Luckily, none were confidently wrong. But only 37 were confident and right. The other 19, about one in three, came back unsure. That’s a lot of uncertainty for an agent meant to unblock the team.
What puzzled me was that the answers seemed well grounded. The tools were connected, and the sources were cited correctly. So why was le goat so unsure?
Get Bill Guo’s stories in your inbox
Join Medium for free to get updates from this writer.
Here’s one of those 19 cases, recreated. Someone asked how big the field labels on our forms should be. le goat found three answers: the Forms PRD in Notion said TBD, Figma said 13px, and the code said 14px. It couldn’t tell which was current.
Finding the answers wasn’t the problem. Knowing which one to trust was.
Why the docs disagreed
To see how they drifted apart, I lined up the sources chronologically. A PM asked about the field-label size, and the PRD recorded TBD. Design later settled on 13px, and the code followed. But the PRD never caught up, and an unrelated refactor eventually changed the code to 14px. No wonder le goat was unsure.
This is partly a team coordination problem. Practices like Atlassian’s working agreements help clarify how teams work together and where decisions live. But decisions change, and documentation doesn’t always keep up.
A proactive teammate
And this wasn’t the only time our sources disagreed. Of the 2,413 asks, 204 (8.5%) were people checking which version of something was right. Those were some of the best questions I got. They helped catch discrepancies and kept the team aligned.
A real GOAT doesn’t just answer questions. They know which questions need asking.
So I wondered if le goat could do that too. That became the idea behind the next iteration: making it a proactive teammate that could speak up on its own, in three ways.
Shares the FYI. Decisions don’t always reach everyone who needs to know. So le goat sends a short morning update every day with what’s changed and where it was decided.
Flags the discrepancy. Every Monday, le goat checks for sources that don’t agree. Instead of just raising questions, it brings the right people the evidence and a proposed fix to review.
Follows through. Fixes can slip through the cracks. When one goes unreviewed, le goat checks whether it’s already been handled before sending the owner a reminder.
Every accepted fix leaves a cleaner source. And with each owner only reviewing their part, nobody has to maintain everything alone. A little divide & conquer.
A few tweaks under the hood
For anyone curious about the tech, here are a few nerdy improvements I made since the first prototype.
Classification. Not every message needs the same amount of thinking (or spending 🤑). Jev, a small, fast model, classifies messages first: simple ones get a prescribed reply, while real questions go to the LLM with an appropriate thinking level.
Connector expansion. We added Figma and our service knowledge base through Supabase, alongside Notion, GitHub and the web. That’s how we got le goat answering directly in Figma comments: a webhook picks up the mention, and the reply goes back into the same thread.
Permissions. We made le goat aware of where it’s speaking — public channels, private channels or DMs — to avoid oversharing sensitive information. People can opt in by linking their Slack account, letting le goat search their private conversations while keeping the answers visible only to them.
Triple-double for the team
I tracked three things to see if le goat helped: how many questions still reached me, how long people waited, and how often it was confident and right. All three improved. And the answers now live in shared threads, where the next person can find them.
Setting the GOAT free
Every team needs someone who knows where things live, why decisions were made, and who owns what. But their days shouldn’t go to answering the same questions.
le goat takes care of the everyday asks, freeing those people for the work that needs them: hard design calls, riffing on ideas, pushing boundaries, and raising the ceiling.
There’s a bigger benefit, too. As agents become part of everyday work, a reliable knowledge base matters even more. Fewer discrepancies mean fewer made-up answers and better information for people and agents alike.
Let le goat handle the busywork, so the real GOATs have time for the hard work.
What’s next: the agent farm
Now that le goat is out of the barn, I’ve been thinking about recruiting a few teammates. A sassy owl for critiques, a diligent mule for the dirty work, a quick horse for urgent requests. Every GOAT needs a team, right?
Further reading
A few resources that informed this work or offer useful perspectives on the challenges discussed:
- Radical Candor — Kim Scott. On building effective teams and recognizing the contributions of rock stars and superstars.
- The Cost of Interrupted Work: More Speed and Stress — Gloria Mark, Daniela Gudith & Ulrich Klocke (CHI 2008). On the hidden costs of workplace interruptions.
- People + AI Guidebook — Google PAIR. On designing AI interactions around uncertainty, trust, and human control.
- Working Agreements — Atlassian Team Playbook. On establishing shared expectations for how teams communicate and make decisions.
