Sitemap
Level Up Coding

Coding tutorials and news. The developer homepage gitconnected.com && skilled.dev && levelup.dev

Member-only story

I Killed the Worker After the Charge. Nothing Retried.

11 min read5 days ago

--

Press enter or click to view image in full size
Rails Background Jobs: Idempotency and Retry Safety

I sent kill -9 to a Solid Queue worker a few seconds after its job had made its side effect. The job was marked failed and never ran again.

The same test on Sidekiq OSS left no job anywhere. The queue, the retry set and the dead set all read zero.

Rails background job retries cover a job that raises and a worker that shuts down cleanly. On the two queues I tested, they don’t cover a killed worker. A payment or email job that dies that way is either parked in a failed list or gone, and nobody gets paged for either.

The standard advice for Rails background job retries is to make the job idempotent, so a repeat run is harmless. I still follow it. But a retry only happens on some deaths, and I wanted to know which ones.

I ran each case once in development on macOS, Ruby 3.4.5, Rails 8.1.4, Solid Queue 1.4.0 on SQLite, and Sidekiq 8.1.7 open source against a local Redis. The job appends a line to a log file, which is the side effect, then sleeps 90 seconds.

I lowered Sidekiq’s shutdown timeout from its 25-second default to 5 seconds, so both…

--

--