Jev is the future. Most Jev demos are still overengineering.

My feed spent last week routing support tickets with a $40 million model.

Jev came out of stealth on 15 September. TypeSafe AI, San Francisco, founded by Diogo Almeida, one of the people behind InstructGPT. Within three days I'd seen it sort inboxes, tag emails, detect languages, score sentiment and play Pong. Every demo ended with the same line. Look how fast, look how cheap, chat models can't do this.

Both halves are true. And I still think most of those demos picked the wrong opponent.

What Jev actually is

Jev doesn't write text. You send it a block of state and a list of typed questions, and it sends back decisions.

There are three question types. Choice picks one option from a list you supply, with a probability for each option. Score rates the state on a rubric you define. Noul answers a yes or no statement with a number between 0 and 1. All questions in one request get evaluated in parallel, in a single pass, and the output arrives as typed values your code can branch on. No JSON to parse, no "Sure! Here's the classification:" to strip.

The result is 70 to 500 milliseconds per call, at $0.042 per million input tokens, with output tokens free. A frontier model doing the same routing job takes seconds and charges you for every generated word.

I believe this shape is the future. We're heading toward a split. Very large, slow, expensive models like Fable or Opus for open-ended work, and next to them a family of narrow specialists: OCR, speech, image understanding, and now decisions. Jev is the first serious model in that last slot, and it's a clean idea.

So far, so good.

The comparison in every demo

Every Jev benchmark I've seen compares it to a chat model. TypeSafe's own headline is 193 times faster and 444 times cheaper, measured against the average of GPT-6 Astra and Fable 5.1 on four internal workflows. Ably's Pong demo puts Jev at 227 milliseconds per move against 2.5 to 3.5 seconds for Haiku, Gemini Flash and GPT-5.6 Sol.

Against a text generator, sure. Asking Fable to return the word "billing" is like hiring a barrister to read the room number off a door.

But classification was not invented in September. If you have a few thousand labelled examples, a fine-tuned BERT has been the standard answer since 2018. Small, boring, 110 million parameters, runs on a laptop CPU. Before that we had logistic regression on TF-IDF. Before that we had regex, and regex still handles more production routing than any of us admit in public.

Jev should be benchmarked against those. Almost nobody does it.

"Why does it matter what you compare it to, Jack?" you'll ask me. Because the number changes completely.

On a single request, BERT-base on a plain CPU answers in about 100 milliseconds unoptimized, a distilled version in about 60, and an optimized one under 10. Jev is 70 to 500 including the network. Same order of magnitude, sometimes faster, sometimes not. Batch the requests on a GPU and BERT drops well under a millisecond per item. That's where the hundred-times gap opens up, and it opens up in BERT's favour.

Cost goes the same way. $0.042 per million tokens is cheap until you notice the other side of the ledger. The BERT model already runs on the same box as your API, and it doesn't need a waitlist, a vendor, or a network connection.

One outside team has published a proper test. Parallel, who run rerankers and classifiers billions of times a day, put Jev up against their own fine-tuned models. On search reranking Jev matched one of their custom rerankers, NDCG@10 of 0.7, zero-shot, which is genuinely impressive. On topic classification and query freshness their internal models won. And Jev cost them more per document than the models they host themselves. Their own summary says the headline speed and cost figures are against autoregressive LLMs, not dedicated classifiers, and that specialised classifiers will often still win on both.

That's the whole argument, from people with more classifiers in production than I have.

Where Jev wins for real

Zero-shot. You have no labelled data, no time to collect it, and the question changes every week. Jev takes the question as a string at request time and answers it. A fine-tuned BERT can't do that. You'd need to gather examples, train, evaluate, deploy, and repeat when the labels move.

That covers a real set of jobs. Prototyping a routing layer before you know what the categories are. Research where the taxonomy is still moving. Non-technical people who need a classifier and will never train one. Agent guardrails where the question is different for every tool.

Parallel's conclusion was the sensible one: if you need a classifier and haven't trained one yet, start here, then check the quality on your own examples.

Where I'd stop is the moment the categories settle and the volume climbs. At that point you have the data Jev generated for you, and a fine-tuned small model will be cheaper, faster, and yours. Jev becomes the bootstrap, not the production path.

Scoring quality didn't move

There's a second confusion in the demos, and it's more expensive than the first.

Jev changes the output format. It doesn't change how well the model judges.

Take TypeSafe's own dashboard. Across their four workflows Jev lands at 67.8% agreement with the reference. Opus 5 in workflow mode lands at 74.1%. On invoice processing the gap is 61.8% against 79.1%. The company publishes this and says out loud that the multipliers are at the high end of real gains and that the evals were written by their own team. I respect that more than the marketing.

So what you get is cheap, fast, structured scoring at roughly Sonnet-level judgment. Good trade for most routing. Bad trade when the judgment itself is the hard part.

Jev is a language model trained on the same kind of data as the others. Swapping the text head for a probability head doesn't make it a better judge.

That's the point people miss when they say "Jev is great at scoring." It's great at returning a score. Whether the score is right is a separate question, and on that question it's a mid-tier model like any other.

The problem it does solve

The demos I've complained about all use Jev as a router. The use I care about is different, and I haven't seen anyone post it yet.

Put a reference next to the result and ask how far apart they are.

That's a new kind of question. A fine-tuned BERT can't answer it, because there's no dataset of "your spec versus your implementation" to train on. A chat model can answer it, slowly, in a paragraph you then have to parse, at Opus prices. Jev answers it as a score with a probability distribution, in under half a second, for a fraction of a cent.

We run MadBench, our eval rig, against every model release and against our own skills, and the validation tier is deliberately stupid: deterministic object diffs, OCR against golden screenshots, no language model anywhere in the pass or fail decision. I'm not changing that. Above it there's a scoring tier where a judge model rates how close a result came, and that tier is where Jev goes next.

The shape I want inside Magus: the agent ships a change, the harness hands Jev the spec and the diff, and asks a handful of typed questions. Does the implementation cover every requirement in the reference? Score the deviation on a rubric. Is there anything in the diff the spec never asked for? That last one is a hallucination detector with a probability attached, and it's the question I'd pay for.

Two constraints before anyone copies this. Jev only reads text and JSON, so a UI check means DOM or component code against the design tokens, or a vision model describes both screens first and Jev scores the descriptions. And the state plus the longest question has to fit in 32k tokens, so this works per component, not per repository.

I haven't measured it yet. I'm wiring it in now. When I do, the comparison will be Jev against Opus 5 as the judge on the same reference set, and I'll publish whichever way it goes.

What I'd say to the demo authors

Keep the demos. They're fun and the latency is real.

But when you post the benchmark, add one more column. Put a fine-tuned DistilBERT next to Jev, or a regex if the task is that simple. If Jev still wins, you've found a real use. If it doesn't, you've found out early, before the vendor lock-in and before your classifier depends on a waitlist.

I'm glad Jev exists. The idea that a model can return a decision instead of prose is the right one, and I expect a lot of models to look like this in two years. The mistake isn't using it. The mistake is measuring it against the wrong thing and then shipping a cloud call where a local model or three lines of regex would have done the job.

Classification was a solved problem. Jev solved the zero-shot version of it, and opened a second one, scoring a result against a reference without training anything. That second one is the one I'm going to test.