Member-only story
Coding an Agent: Let There Be Vision
Teaching a text-only coding agent to read your screenshots
A coding agent that cannot see is a strange thing to use. You hit a compiler error inside a GUI, or a test harness prints a table that only makes sense laid out, or a stack trace scrolls past in a window that is not the one your agent is attached to. The obvious move is to screenshot it and paste. So you do, and the agent politely tells you it received a file called ef9cfcb8.png.
That is not a bug. The agent in question is plank, which runs DeepSeek V4 Flash locally through a hand-written Metal kernel, and the model is text-only. There is no vision tower to send pixels to, so the paste pipeline does the only honest thing available: it caches the image and hands over the path. The agent can pass that path to a tool. It cannot look at the picture.
You do not need to care about plank for any of this to apply to you. A text-only model behind a tool boundary is the common case, not the exotic one, and the fix turns out to be portable in a way the problem is not.
The gap is annoying rather than fundamental. A path is a perfectly good handle if something at the other end can open it. So the fix is not to bolt a vision tower onto an 81 GB text model. It is to write a small server that reads image files and hands back…
