TypeSafe AI came out of stealth on September 15 with $40M and a model called Jev that does not generate text. You send it state and typed questions. It returns a choice from a set, a score on a rubric, or a probability that a yes/no holds. All the questions run in one parallel pass, answers arrive in 70 to 500 ms, and input costs $0.042 per million tokens. Output is free.
The launch claims need a closer read. The 193.6x and 444.6x figures come from TypeSafe’s own workflow evals, which score agreement with the average of GPT-6 Astra and Fable 5.1, not correctness. “Zero hallucination” means the output always matches your schema, which TypeSafe admits is “not empirical”. The Hacker News thread, 1,902 points and 498 comments, put it in one line: “Sure, it can’t emit an invalid type, but it can still emit a completely wrong valid value.” And the claim that matters most, that a 0.9 is right about 90% of the time, has no published curve or paper behind it.
I had a better test sitting in a database.
The Queue Nobody Was Going to Clear
Pricogni is my competitor pricing analysis product. Its matching engine pairs a retailer’s catalogue with scraped competitor listings and gives each pair a tier. Identifier or brand-plus-pack-plus-size agreement is high. Brand plus a strong name is medium. Brand plus a weak name is low, which means a human should look at it.
Nobody looks at 9,081 rows. The design doc from June specified a judge stage for exactly this: an LLM that adjudicates the low tier, advisory only, logged with model and prompt version. It was deferred on one line.
— docs/llm-enrichment-and-matching.md, Pricogni, June 2026Measure first: low-tier volume (judge workload) + per-match cost/latency of a judge tier at catalogue scale.
The schema already had a judge column with a verdict, a confidence, and a difference type. Nothing wrote it. Jev returns that shape natively, so the build took an evening.
The Build
The CEO’s own description of the three primitives, in the HN thread, is the clearest one I found.
— Diogo Almeida, TypeSafe CEO, on Hacker News“choice” maps to “match” statement, “score” maps to sorting, “noul” short for bernoulli maps to if-statements
About 150 lines. Per pair, the state is two named objects: the catalogue item with name, brand, manufacturer, pack count and sizes, and the competitor product with the same fields plus the first 1,500 characters of its scraped description. Two questions run on every pair.
same: {
type: 'noul',
instructions:
'Are `catalogue` and `competitor` listings for the same retail product: ' +
'same brand line, same variant (shade, flavour, scent, strength, form) ' +
'and same pack size? A field missing on one side is unknown, not a difference.',
criteria: {
true: 'Same product and same pack size',
false: 'A different product, variant, strength, form or pack size',
},
},
difference: {
type: 'choice',
instructions: 'What is the main difference between `catalogue` and `competitor`?',
criteria: {
none: 'No meaningful difference; the same product',
pack: 'Same product, different pack size or count',
shade: 'Same line, different shade, flavour, scent or colour',
formulation: 'Same line, different strength, form or formulation',
flanker: 'Different product within the same brand range',
line: 'Different product line or different brand',
},
},Code turns the probability into a verdict: 0.8 and above confirms, 0.2 and below refutes, anything between abstains and stays in the human queue. The model produces no rationale, so the code writes one from the numbers. Nothing downstream reads the verdict yet.
The Numbers
| Rows judged | 9,081 |
| Cost | $0.32 |
| Wall time, 6 concurrent | 13 min 22 s |
| Refute | 4,443 (49%) |
| Confirm | 1,952 (21%) |
| Abstain | 2,686 (30%) |
The probability distribution is bimodal: 3,557 rows under 0.1, 1,847 above 0.8, and a thin middle. That is the shape you want from a model that claims to know when it does not know. A shape is not a calibration curve.
The difference labels explain the low tier. 4,216 rows are flankers: same brand, same size, different product. A one-litre body wash in one scent against the same brand’s sensitive-skin variant. A shampoo against the conditioner from the same range in the same bottle. A knee support against the ankle support. The scorer’s “brand plus volume, weak name” rung, which I had read as a probable shade variant, refuted five to one. Same brand and same size is what a flanker looks like.
One row in the low tier had a human decision, and the judge disagreed with it: a reviewer had accepted a roll in one width as a match for the same brand’s listing in double the width. Jev put the probability of a match at 0.06 and called it a flanker. Different width, different price. I side with the model.
I read 50 verdicts by hand: 20 confirmations, 20 refutations, 10 abstentions. All 20 refutations held up. Two confirmations were arguable, both where the competitor name drops a variant suffix the catalogue keeps. Of the 10 abstentions, 8 deserved a human and 2 should have been refutations. 48 of 50 by my reading, and my reading is not ground truth.
What This Doesn’t Settle
- Calibration. Fifty hand-labelled rows cannot test whether 0.85 means 85%. Pricogni’s medium-tier audit will produce real labels over months. Until then the judge writes a column and nothing acts on it.
- The 193x and 444x. Nothing here validates them. My comparison point is a stage I never built because a frontier model at $15 per million output tokens made it not worth building.
- Whether this is more than logprobs. Sean Goedecke argues that you could “aggressively prefill a regular LLM and only produce one constrained token” and get most of the speed, and that no test-time compute “will likely cap this kind of model around the strength of non-reasoning LLMs”. mini-jev matches constrained JSON accuracy on a 4B model by reading logits. Alexis Gallagher’s read of the launch was that the reactions show “a lot of folks are not aware that encoder-only classifiers exist”. TypeSafe has published no architecture, reward function, or ablation.
- Prompt injection. The competitor description is scraped text. TypeSafe’s own jaggedness page says content written to steer the model can move the answer. Tolerable for a price-matching judge. Not for anything touching money or access.
- No arithmetic, no dates, no images. The pack and size contradictions that make hard rejects stay in code. The model reads names.
The Takeaway
- The price changed the architecture, not the accuracy. Screen-first, LLM-validate has been the published production pattern since 2024. The per-row cost of the validate step is what stopped me building it. At 32 cents per 9,000 rows it should run nightly.
- Free output tokens is the pricing story. A judge returns four numbers. Paying generation rates for four numbers made every LLM-as-judge design a budget line.
- Keep it advisory until you can measure it. The column exists, the review UI does not show it, and nothing publishes on it. Authority waits for the labels.
Related Posts
