Let’s Build a “Jev” From Scratch
I read the TypeSafe announcement, the Cactus docs, the community clones (OpenJev, LocalJev, Bespoke Nimble), and then actually built and trained a model. This is me explaining what I found and what I built
Before we proceed, let’s stay connected! Please consider following me on Medium, and don’t forget to connect with me on LinkedIn for a regular dose of data science and deep learning insights.” 🚀📊🤖
Now the fun part. I built a working, naive “Jev-style” model. This is my design for Jev’s real architecture. What I do deliberately copy from the proven interface:
- one
state, many typedquestions - answered in parallel (one encoder pass over the state)
- outputs are probabilities
- trained with a proper scoring rule (optimizing calibration)
- no text generation at all
Let me show you the whole thing, simplified, and then give you the real code.
The mental model
Think of it as: read a page of testimony once, then answer every question on a worksheet from memory of that same reading.
state ("the article") questions ("is urgent?", "which topic?", "score anger 0-1")
│ │
▼ ▼
┌───────────────┐ ┌─────────────────┐
│ Encoder │ (ONE pass) │ Encoder │
│ (transformer)│ │ (shared weights)│
└───────┬───────┘ └────────┬────────┘
│ pooled vector h_state │ h_question
└───────────────┬──────────────────┘
▼
fuse: concat → MLP
│
┌───────────────┼───────────────┐
▼ ▼ ▼
noul head choice head score head
(sigmoid) (softmax over K) (logistic → range)
│ │ │
P(true) distribution score in [lo,hi]
│ │ │
└───────────────┴───────────────┘
▼
all answers at once, from ONE state readThe key word is “ONE”. We encode the state exactly once. Every question reuses that pooled state vector that’s the parallelism, and it’s real, not a marketing trick.
The input and the output (concretely)
So you can see exactly what goes in and what comes out, here is a real request we ran on our trained model
INPUT one state, three typed questions:
{
"state": "I've been trying to connect my Stripe account for 3 days,
it keeps failing with a 403. I'm losing sales and my manager
is anxious. Please help ASAP.",
"questions": {
"is_urgent": { "type": "noul", "instructions": "The message conveys urgency or time-sensitivity." },
"department": { "type": "choice", "instructions": "Which team should handle this?",
"options": ["billing", "technical", "sales"] },
"frustration":{"type": "score", "instructions": "How frustrated does the customer appear? (0=civil, 1=angry)" }
}
}OUTPUT one typed, probabilistic answer per question (no text generation):
{
"is_urgent": { "type": "noul", "noul": 0.6742, "is_true": true },
"department": { "type": "choice", "distribution": {"billing": 0.0, "technical": 1.0, "sales": 0.0},
"argmax": "technical", "confidence": 1.0 },
"frustration": { "type": "score", "score": 0.518 }
}Every field is either a probability (noul), a probability distribution + confidence (choice), or a bounded number (score). No tokens are generated anywhere.
Get azhar’s stories in your inbox
Join Medium for free to get updates from this writer.
Screenshot of the request → response pair (the input on top, the model’s output below):
Three heads for three question types
Because a decision model reads the whole context, we use no causal mask every head can attend to the entire state. (A generative LLM can’t do this; it must read left to right. This is a real, structural advantage of “decision” models.)
Why is it fast?
- No autoregressive generation. No
for token in .... One forward pass, done.
Generation is where LLMs spend most of their latency.
- No output tokens to pay for. Every “answer” is a handful of numbers.
- One state pass for N questions. Adding questions doesn’t re-read the state.
- Small. Our toy is ~3M params; even real systems are far smaller than a frontier LLM.
Calibration the piece that makes it “honest”
A model that guesses “urgent = 99%” but is only right 60% of the time is useless for automation. So we train with a proper scoring rule (Brier score for noul, you’re penalized for confidence that doesn’t match reality), and then we do **temperature scaling**: search for a single scalar T that makes predicted probabilities match actual accuracy on a held-out set. That's our (small) implementation of "calibrated decisions."
the actual code
All code is in this repo. Here are the three files that matter, then the full source.
jev_toy/model.pythe architecture (encoder + 3 heads)jev_toy/data.pyturns real datasets into (state, question, type, target) pairsjev_toy/train.pytraining loop with parallel, per-head proper-scoring lossesjev_toy/eval.pyaccuracy, Brier, expected-calibration-error (ECE), temperature scalejev_toy/serve.pya mini "Jev API": one state, many questions, all answered in parallel
model.py (simplified)
class SystemOneModel(nn.Module):
def __init__(self, cfg):
self.encoder = _Encoder(cfg) # shared transformer encoder
self.noul_head = nn.Linear(cfg.d_model, 1)
self.choice_head = nn.Linear(cfg.d_model, cfg.num_choice_heads)
self.score_head = nn.Linear(cfg.d_model, 1)
def encode_state(self, s_ids, s_mask):
return self.encoder(s_ids, s_mask) # ONE pass over the state
def answer(self, h_state, q_ids, q_mask):
h_q = self.encoder(q_ids, q_mask) # encode each question
h = F.gelu(self.merge(torch.cat([h_state, h_q], -1)))
return {
"noul": self.noul_head(h),
"choice": self.choice_head(h),
"score": self.score_head(h),
}The data (real, public datasets)
We map the three question types onto real Hugging Face datasets:
(Toy scale: 1,500 examples each, subsampled so a laptop trains in minutes. The code scales to the full datasets and to GPU as-is.)
train.py Python
for batch in batches:
# ONE forward pass encodes all states in the batch
h_state = model.encode_state(state_ids, state_mask)
# every question (of any type) answered off that shared state
logits = model.answer(h_state, question_ids, question_mask)
loss = proper_scoring_loss(logits, types, targets) # Brier + CE + MSE
loss.backward(); opt.step()Results (ran on my machine, no GPU)
This section is honest numbers from a real run 8-core laptop CPU, 15 GB RAM, ~3.1M parameter model, 3 epochs, word-level vocab of ~20k.
Training loss: 1.55 → 1.27 → 1.00 across three epochs. Post-hoc temperature scaling picked T = 0.8 (ECE → 0.0254).
Accuracy is low that’s expected and fine: ~3M params,
a small slice of each dataset (1,500 of 120k AG News, 25% of BoolQ, 2% of SST-2), only 3 epochs, CPU-only.
The interesting result is the calibration: an ECE of ~0.027 means the model’s stated confidence genuinely matches how often it’s right, within ~3 percentage points.
That’s the whole architectural point of a “System One” model not raw smartness, but honest smartness a program can safely branch on.
And the serving demo returns the exact Jev response shape from one state + three parallel questions.
Does our architecture actually predict in parallel or is it just a fine-tuned transformer faking it? This is the honest question, so let’s answer it with a measurement, not an adjective. We ran the full answer path on one state while increasing the number of questions, on CPU:
If we were re-reading the state for every question (the naive fine-tuned-transformer approach), going from 1 → 12 questions would cost ~12× the time. Instead it costs ~3.7×.
The state encoder runs exactly once; the growth comes only from the cheap question-batch pass. That is the parallel property Jev claims, made measurable in our build. (This is a CPU number the structure is what matters, not the milliseconds.)
How it differs from “a transformer fine-tuned to act like Jev”: a normal fine-tuned LLM generates an answer token-by-token (slow, sequential, can drift off-schema). Ours has no generation loop at all the encoder runs, heads fire, done. That’s the structural difference, not just a speed trick.
The value of this build is architectural you can hold it all in your head, watch it train, and share one encoder across three question types. Full data + deeper encoder + GPU is the same code with bigger numbers.
