Shortcuts + local AI — image to text
If you love AI but don’t like they way it uses up a whole heap of worldly resources 💦, this post will show you can to use your own macOS27 to easily convert an image to text with the click of a button.
In macos27, Goldern Gate, you can now describe your shortcut, “convert an image to text using local AI”. It produces a shortcut that works, but it uses a Cloud AI model. If you try to switch the “Cloud” to use an on device AI you get an error, so close 😭.
So lets fix that using Apples new local foundation model `fm`.
Techy bit — for those interested. In macOS27, +M1 chips, you now have easy access to the internal AI model, called the “Foundation Model” (fm). Open “Terminal” and enter something like the following and you get a description of the image.
fm respond --image ./filepath.jpg "explain the image in detail"So all we need to do is simply create a Shortcut that gets an images file path and call the command above. Thats it, super simple.
Get richard moult’s stories in your inbox
Join Medium for free to get updates from this writer.
So create a Shortcut…
- In the panel on the right, “shortcut details” select “Use as Quick Action”.
- On the left set “Receive” to only image types.
- In the right panel, “Action Library”, select “Get Details of Files” and drag that into the left panel. Set details to “File Path”.
- In the “Action Library”, select “Run Shell Script”, and drag that into the left panel. Add the command above.
- In “Action Library”, select “Show content” and drag that into the left panel.
You should now have a Shortcut very similar to….
Now — if this is the first time you have created a “Quick Action” you need one more step. Open “Finder” and find an image you want to convert to text. Double tap to open options menu. Near the bottom you’ll see “Quick Actions”, hover over that with the mouse and select “Customise”. You’ll been presented with a pop up, search for your Shortcut and switch it on.
Now go to the image in Finder, double tap to open the menu and select “Quick Actions”, you’ll now see your shortcut. Tap the Shortcut, and hey-presto, it gets converted to text, locally on your machine.
For more shortcut ideas check out this link.
