Watch a release after merge: check observability (Datadog, Sentry…) for a few minutes and report back in the thread

[carmichaelt]

Problem

When an agent's PR merges and deploys, the thread treats the work as finished. The risky part comes next: the first few minutes in production.

Right now I have to open Datadog myself, find the right service, and compare errors and latency before and after the deploy. If something regresses, the thread that produced the change knows nothing about it, and I have to paste the context back in by hand.

Proposal

An optional release watch on a PR or thread:

• Trigger: the PR merges, or a deploy for that commit is detected.
• Watch window: a configurable length (default about 5 minutes) during which T3 compares the release against a baseline for the services it touched:
  • error rate and new error-tracking issues
  • latency (p95/p99)
  • monitors that start alerting
  • optionally, logs or RUM errors
• Report in the thread: a short summary when the window ends ("No regressions", or "Error rate on api up 4× since deploy, 2 new issues"), with links to the dashboards.
• On regression: the thread can wake the agent with the evidence to investigate and propose a fix or revert. It should not roll back by itself.
• Status in the PRs view: a badge on the PR ("Watching 3:12 left", "Healthy", "Regressed").
• Pluggable providers: Datadog first, then Sentry, Grafana and Vercel observability.

Why in T3

T3 already links PRs to threads and watches checks and reviews. Watching production after the merge is the next step in that same loop, and the thread already has the context needed to act on a regression.

What works today

With the Datadog MCP connector, an agent can query metrics, logs, error tracking and monitors. I can schedule a follow-up task to check about 5 minutes after merge. That proves the plumbing works, but it's manual, set up again for each PR, and invisible in the PRs view.

Suggested scope

1. First: a fixed window after merge, a fixed set of signals, and one summary message in the thread.
2. Later: deploy detection (instead of merge time), a regression badge in the PRs view, waking the agent automatically, more providers.

Open questions

• How does T3 map a repo or PR to the right services and environment? Per-project config, or inferred from tags like service/version?
• Is the baseline "the same window before the deploy" or "the same time yesterday"?
• Does this belong to the PR, or to the thread that made it?

Related

• Backlog board next to PRs (companion idea): Backlog board next to PRs: see planned work and start threads from issues (Linear, GitHub Issues, Jira…) #30981