Member-only story
I Keep Flagging That Our AI-Generated Code Is Getting Worse. I Keep Being Told I’m Missing the Point.
Our velocity metrics have never looked better. Our on-call incidents are getting harder to explain. I have been trying to connect those two things for eight months.
The post-mortem was the moment I stopped being able to let this go.
We had an incident on a Tuesday night. A race condition in our payment processing service that caused duplicate charges for a window of about forty minutes before our on-call engineer caught it and rolled back. About three hundred transactions were affected. The resolution took four hours. The customer support queue was still clearing two days later.
I had flagged the race condition in code review three weeks earlier.
Not in a vague, cover-your-bases way. Specifically. I had written a comment on the exact function where the issue lived, explained the scenario in which two concurrent requests would produce the duplicate, and suggested a locking mechanism that would prevent it. The comment had been marked resolved. The fix had not been made. The code had been merged and shipped.
