Member-only story
Think Fast, Answer Everything: LLM Sheaf Decoding to Get the Best of Both Worlds
We built a model that reads a document once and decodes all its answers in parallel. On French company extracts, it matched Gemini 3.6 Flash and ran about nine times faster. On messier documents, it needed one more trick ;)
“Sheaf decoding builds on several research papers that paved the way for shared prefixes and parallel answers!”
At LegalPlace, documents arrive all day: ID cards, passports, proofs of address, Kbis extracts, contracts, invoices. For each one we need two kinds of answers. Fast decisions: is this the right document, is it readable, is it still valid? And short extractions: a name, a date, a registration number, an amount.
That is our main challenge. Classic fast classifiers are quick but rigid: every new document type or question means new labels and a new training run. Large language models are flexible, but slow when asked to fill a long form. We wanted the speed of the first and the flexibility of the second, in one generic, modular approach, where a new document or a new question is a change to the prompt, not a new…
