Jev, the AI that decides without writing: I put it to the test against GPT

Ever since ChatGPT arrived, the models everyone talks about have been chatbots: you ask a question and they answer by writing. On September 15 the first of its kind that isn't one came out. It's called Jev, TypeSafe AI built it, and it can't write a single word: it makes decisions, and next to each one it tells you how sure it is. Its makers promise a workflow 193 times faster than a frontier LLM. A promise like that had to be checked, so I did: 105 customer emails for a business management app, six models, two languages, and a comparison only between models that make as many mistakes as Jev. Here's how it went, with the method, the tables and the Delphi code.

If you’ve put an LLM inside a business application, sooner or later you’ve used it for a small decision: which department gets this email, what kind of document just came in, whether the customer is angry. It works. But every decision costs a second or two plus output tokens, and at ten thousand emails a day the time and the bill become the problem.

On September 15 TypeSafe AI announced Jev, a model that can’t write a single word and only returns decisions. Their home page claims a workflow 193.6 times faster and 444.6 times cheaper than a frontier LLM. I talked about it in episode 7 of the “while true do;” podcast (in Italian), where I promised to publish the method, the tables and the Delphi code here.

What Jev is

You give it a context, which TypeSafe calls state (the text of an email, a record), and some typed questions. There are three types:

• choice: pick one of up to 255 options, each described in words;
• score: place the text on a scale of 2 to 10 levels;
• noul: a yes or no question, and the answer is the probability of yes.

You never get back text to interpret. You get the chosen option, the probability distribution over all the options and a confidence. All questions are computed in a single pass, so adding one barely changes the response time. List price is 0.042 dollars per million input tokens, and you don’t pay for output.

Since September 18 Jev is also on OpenRouter, at the POST /api/v1/systemone endpoint. That’s what I used: with a single key you call both Jev and the LLMs you compare it with, and every timing includes the same hop through OpenRouter.

Calling it from Delphi

The call is a POST with a JSON body, so THTTPClient and System.JSON are all you need, no SDK. This is the request from JevTriage_en, the demo program: three questions about an email, one per type.

// The three questions for Jev: department (choice), frustration (score), urgency (noul).
function BuildJevRequest(const AModel, AEmail: string): string;
begin
  var lRoot := TJSONObject.Create;
  try
    lRoot.AddPair('model', AModel);
    lRoot.AddPair('state', AEmail);
    var lQuestions := TJSONObject.Create;
    lRoot.AddPair('questions', lQuestions);   // from here on lRoot owns it

    var lDepartments := TJSONObject.Create;
    lDepartments.AddPair('administration', 'Invoices, payments, charges, refunds');
    lDepartments.AddPair('technical support', 'Program errors, freezes and malfunctions');
    lDepartments.AddPair('sales', 'Quotes, new modules, licenses, training');
    lDepartments.AddPair('other', 'None of the above');   // always include the fallback option
    lQuestions.AddPair('department', TJSONObject.Create
      .AddPair('type', 'choice')
      .AddPair('instructions', 'Which department should handle this email?')
      .AddPair('criteria', lDepartments));

    var lLevels := TJSONArray.Create;
    lLevels.Add('Calm, states the facts');
    lLevels.Add('Annoyed but polite');
    lLevels.Add('Very angry, harsh tone');
    lQuestions.AddPair('frustration', TJSONObject.Create
      .AddPair('type', 'score')
      .AddPair('instructions', 'How frustrated is the customer?')
      .AddPair('criteria', lLevels));

    lQuestions.AddPair('urgent', TJSONObject.Create
      .AddPair('type', 'noul')
      .AddPair('instructions', 'Is the request urgent or does it have a close deadline?'));

    Result := lRoot.ToJSON;
  finally
    lRoot.Free;
  end;
end;

This is the English version of the demo. The zip also has the Italian original, JevTriage, since Italian is the language these emails arrive in at an Italian software house. If you work with System.JSON and want a refresher on the fluent API and who owns what, there’s the complete guide to JSON in Delphi.

You read the response with GetValue<T> and a path: answers.department.choice, answers.department.confidence, answers.urgent.noul. Then the code, not the model, decides the routing by comparing the confidence against two thresholds:

// Routing is decided by the code, not by the model: confidence is just a number to compare.
function RouteByConfidence(const ADepartment: string; AConfidence: Double): string;
begin
  if (ADepartment = 'other') or (AConfidence < HUMAN_THRESHOLD) then
    Result := 'to an operator (a person decides)'
  else if AConfidence >= AUTO_THRESHOLD then
    Result := Format('automatic to "%s"', [ADepartment])
  else
    Result := Format('to "%s" with caution (needs confirmation)', [ADepartment]);
end;

AUTO_THRESHOLD is 0.85 and HUMAN_THRESHOLD is 0.5. Further down you’ll see that 0.85 is optimistic.

Four emails, one real run

JevTriage_en sends the same four emails to Jev and to google/gemini-2.5-flash-lite, which it asks to write a JSON with the same three fields. This is the output of a real run from today, in full:

Email 1: "Good morning, this month you charged the fee for the management software twice to our company card. Could you reverse the second charge? Thanks, Marta Rossi"
  Jev (typesafe/jev-1.13-20260917) in 455 ms
    department:   administration  (confidence 1.00)
                  administration=1.00  other=0.00  technical support=0.00  sales=0.00  
    frustration:  0.04 of 2  (confidence 0.95)
                  0=0.96  1=0.04  2=0.00  
    urgent:       no  (probability of yes 0.21)
    routing:      automatic to "administration"
  LLM (google/gemini-2.5-flash-lite) in 1101 ms
    department: administration, frustration: 1, urgent: true  (no probabilities, no confidence)

Email 2: "Since this morning, whenever I print an invoice the program closes with a memory access error. I have forty invoices to send out by tonight, I don't know what to do anymore!!!"
  Jev (typesafe/jev-1.13-20260917) in 373 ms
    department:   technical support  (confidence 0.99)
                  technical support=1.00  sales=0.00  administration=0.00  other=0.00  
    frustration:  1.75 of 2  (confidence 0.62)
                  0=0.00  1=0.25  2=0.75  
    urgent:       yes  (probability of yes 0.98)
    routing:      automatic to "technical support"
  LLM (google/gemini-2.5-flash-lite) in 715 ms
    department: technical support, frustration: 2, urgent: true  (no probabilities, no confidence)

Email 3: "Hello, we are a firm with twelve workstations and we would like a quote for the warehouse module and for staff training. No rush."
  Jev (typesafe/jev-1.13-20260917) in 365 ms
    department:   sales  (confidence 1.00)
                  sales=1.00  other=0.00  administration=0.00  technical support=0.00  
    frustration:  0.00 of 2  (confidence 1.00)
                  0=1.00  1=0.00  2=0.00  
    urgent:       no  (probability of yes 0.04)
    routing:      automatic to "sales"
  LLM (google/gemini-2.5-flash-lite) in 524 ms
    department: sales, frustration: 0, urgent: false  (no probabilities, no confidence)

Email 4: "Hi, I just wanted to tell you that yesterday's course was really useful. See you soon!"
  Jev (typesafe/jev-1.13-20260917) in 401 ms
    department:   other  (confidence 0.82)
                  other=0.87  administration=0.00  technical support=0.00  sales=0.13  
    frustration:  0.00 of 2  (confidence 1.00)
                  0=1.00  1=0.00  2=0.00  
    urgent:       no  (probability of yes 0.02)
    routing:      to an operator (a person decides)
  LLM (google/gemini-2.5-flash-lite) in 575 ms
    department: other, frustration: 0, urgent: false  (no probabilities, no confidence)

Email   Jev ms   LLM ms   Jev token in/out   LLM token in/out   Jev cost       LLM cost
    1      455     1101         460 / 84            113 / 18    $0.0000193     $0.0000185
    2      373      715         467 / 82            121 / 19    $0.0000196     $0.0000197
    3      365      524         459 / 81            110 / 18    $0.0000193     $0.0000182
    4      401      575         450 / 81            104 / 18    $0.0000189     $0.0000176
Total     1594     2915                                        $0.0000771     $0.0000740

Costs come from OpenRouter's usage.cost field (in credits, 1 credit = 1 USD).
Timings include the hop through OpenRouter.

The first email is a double charge, written calmly. According to Flash Lite the customer is annoyed and the request is urgent. Jev says calm and not urgent, and says it with the probabilities next to it.

The fourth email is a thank-you note, and here the two agree: other, calm, not urgent. Jev picks other with 0.82 confidence, and the code sends the email to an operator, as it does with every other whatever the confidence.

As for cost, on three questions Jev costs as much as Flash Lite, actually a hair more: the request to Jev counts about 460 input tokens, the one to the LLM about 110. Jev’s price advantage is the free output, and with three short answers there’s little output to save on. Four emails prove nothing, though, and Flash Lite is the cheapest model on the list: that’s why I wrote JevBench.

The benchmark: 105 emails, six models

The benchmark does the same job at scale, with these rules:

• 105 customer emails for a software house that makes business management software: double charges, invoices rejected by SDI (the Italian e-invoicing exchange system), payslips that won’t calculate, quote requests, cancellations, spam, phishing. 61 are hard on purpose: sarcasm, restrained frustration, negations (“it’s not urgent, but by tomorrow morning…”), a request buried in the PS of a newsletter;
• 25 questions per email (department, application module, frustration, urgency, refund request, threat to leave, sensitive data, and so on), asked in groups of 3, 10 and 25 per call;
• six models via OpenRouter: Jev 1.13, Gemini 2.5 Flash Lite, GPT-5.6 Luna, GPT-5.6 Terra, Gemini 3.1 Pro, DeepSeek V4 Pro. Same instructions for everyone, no prompt tuning;
• balanced accuracy as the main metric, with 95% bootstrap confidence intervals. Plain accuracy rewards a model that always answers “no”: here a model that always says the most frequent class would get 84% plain accuracy and 41% balanced;
• quality first, then timings: I compare speed and cost only between Jev and the cheapest LLM that makes as many mistakes as Jev, or fewer. A model ten times cheaper that gets one email in three wrong is no use as a stand-in for Jev.

There’s no Anthropic model, and not out of spite: the expected answers were written and reviewed by Claude Opus 5, and a model from the same family would start with an advantage on the borderline cases. I set the labeling rules and decided the most disputed case myself; the other disputed cases were decided by the same model on my behalf, and nobody re-checked every label by hand.

The main test is in English, because TypeSafe says Jev performs best in English and I didn’t want to put it at a disadvantage without saying so. Then I ran it again in Italian.

Who understands more

Each cell tells you how much a model understood across the 105 English emails, with that many questions per call. The columns aren’t three runs of the same test: “3 decisions” means the first three questions (department, frustration, urgency), “10” the first ten, “25” all of them. Each column adds new questions, some of them harder, which is why Jev drops from 97.2 to 94.8: at 25 the set includes the problem’s impact on the customer, where Jev stops at 68%.

The number is balanced accuracy, as a percentage. For each question, every possible answer is scored separately: for “urgent”, you take the share of urgent emails the model recognized as urgent and the share of non-urgent ones it recognized as non-urgent, and average the two. Then you average across all the questions. This way a rare answer counts as much as a common one. A model that always answers “no” to “urgent” gets 88 emails out of 105 right, but scores 50 here. Always giving the most frequent answer to every question scores 34.4 at 3 decisions, 39.4 at 10 and 41.4 at 25: that’s the floor to read the table against.

The square brackets hold the 95% confidence interval. I computed it by resampling the 105 emails at random 2,000 times (bootstrap): with another sample of similar emails, the result would almost always land inside that range. The wider the interval, the less precise the number, and with 3 questions it’s wide because every mistake weighs more. To decide whether an LLM is better or worse than Jev, though, overlapping intervals aren’t enough to go on: I compared each model with Jev email by email, on the same resamples, and only differences that stay clear of zero count.

With 3 and with 10 decisions per call no LLM does better than Jev by a margin you can tell apart from noise. With 25 only Gemini 3.1 Pro beats it, by 2.3 points [0.9; 3.7]. Flash Lite is worse at every level, so its slip on the four emails earlier wasn’t bad luck.

No model returned broken JSON in the benchmark’s 2,835 calls, 1,890 in English and 945 in Italian. With today’s models schema errors are rare, so you need other reasons to pick Jev.

How much faster, how much cheaper

Jev answers in about a third of a second and takes the same time with 3 questions or with 25: 320, 341 and 337 ms on average. The LLMs, on the other hand, have to write every answer, and they slow down: GPT-5.6 Terra goes from 1,397 to 2,338 ms, Gemini 3.1 Pro from 4,625 to 7,151.

This table, for the English run, compares Jev with the cheapest LLM whose accuracy matches Jev’s at each level:

I computed the cost ratio on today’s OpenRouter prices, so it changes as soon as a price does. The timings include the trip through OpenRouter, which from here costs between 20 and 30 ms round trip (median 20 ms in the English run, 28 in the Italian one).

If you compare Jev with a frontier model instead, the numbers grow. A thousand emails with 25 questions cost 0.08 dollars with Jev, 2.17 with GPT-5.6 Terra and 11.82 with Gemini 3.1 Pro, which also takes 21 times as long as Jev. Twenty-one times is still a long way from 193, but it shows where TypeSafe’s numbers come from: they compare Jev with frontier models.

In Italian

In Italian I ran the test again with Jev and the two OpenAI models:

Jev’s quality stays where it was in English. The only difference that holds up statistically is at 3 decisions, 1.9 points [0.6; 3.7]. The LLMs, however, slow down in Italian, because the same email takes more tokens: at 25 decisions GPT-5.6 Luna goes from 1,886 to 2,864 ms, while Jev goes from 337 to 348.

The English emails are an LLM translation of Italian originals written by an LLM. A difference between the languages could also come from the translation. The build_en.py script checks that every translated email keeps the expected answers of the original, but it can’t check the style.

Confidence: “it can’t hallucinate”?

TypeSafe says Jev can’t hallucinate. That holds for the form: it won’t return a field you didn’t ask for or an option that doesn’t exist. Errors of judgment are still there. If you ask it something the text doesn’t say, it can’t answer “I don’t know”: it spreads the probability over the options you gave it and picks one. That’s why the other option in the code above isn’t optional.

What protects you is the confidence. The table shows Jev’s choice and score answers, at 25 decisions, grouped by stated confidence:

(The number of answers and the mean confidence are from the English run.)

Above 0.95 the confidence holds up. In the middle Jev is optimistic: when it says 0.90 it’s right a little more than three times out of four. Confidence rises along with accuracy, so it’s useful, but you have to measure the threshold on your own data. With these numbers, the 0.85 in JevTriage_en would send quite a few wrong answers through automatically: in my data the sensible threshold for acting without confirmation is 0.95.

What it doesn’t prove

The test holds for one task: customer emails for an Italian business management app, same instructions for everyone. It doesn’t tell you how the models do in other domains, what the LLMs would get with a carefully crafted prompt or a few examples, or how much the result varies from one run to the next, because there’s only one run. The hard emails are deliberately overrepresented, so the absolute percentages are lower than what you’d see on real traffic. And the expected answers were written by an LLM, not by a team of people with coffee and a lot of patience.

There are two things to decide before you even try it. Today Jev is only a cloud API, in early access, with no published weights and no on-premise install: your customers’ emails leave your infrastructure and go to a startup that came out of stealth a week ago, after two years of work behind closed doors. And the configuration pins typesafe/jev-1.13, not the jev-latest alias, because the confidence thresholds are valid for one specific version.

Download it and try it

The code is here: jev-delphi-demo.zip. Inside you’ll find the demo in English (JevTriage_en) and in Italian (JevTriage), JevBench, the dataset in Italian and English, the full results with the CSVs and bench/METODOLOGIA.md, which explains every choice at much greater length than this article.

You need Delphi 13 (it should also work on 12 and 11, but I haven’t tried), the DelphiMVCFramework sources for MVCFramework.DotEnv, which reads the .env file, and an OpenRouter key with some credit on it. Copy .env.example to .env, add your key and build:

call "C:\Program Files (x86)\Embarcadero\Studio\37.0\bin\rsvars.bat"
msbuild JevTriage_en.dproj /p:Config=Debug /p:Platform=Win64
JevTriage_en.exe

JevTriage_en costs less than a cent. With six models JevBench makes 1,890 calls per language and takes a good hour. The English run cost about 3.5 dollars; the Italian one, with three models and 945 calls, about 0.9. Before launching it, try JevBench.exe --selftest, which runs all the reports on made-up results without calling any model. Any unknown argument stops the program, so a typo won’t kick off a paid run.

For the “here’s how I see it” part, where I’d put it in a business application and where I wouldn’t, listen to episode 7 of “while true do;” (in Italian). The Jevons paradox, which Jev is named after, is covered in episodes 4 and 5. Jev’s official documentation is at docs.typesafe.ai.

Frequently asked questions

What is TypeSafe AI’s Jev? Jev is a model TypeSafe AI calls “System One”: it does not generate text, it takes a context and some typed questions (choice, score, noul) and returns for each one the answer, the probability distribution over the options and a confidence. It was announced on September 15, 2026 and is used through an API, either directly from TypeSafe or via OpenRouter.

Is Jev really 200 times faster than an LLM? In Daniele Teti’s benchmark on 105 emails for a business management app, Jev was 4 to 8 times faster than the cheapest LLM with the same accuracy, and about 20 times faster than Gemini 3.1 Pro. The vendor’s 193x figures come from workflows chosen by TypeSafe and compare Jev with frontier models.

Does Jev make fewer mistakes than GPT and Gemini? With 3 and 10 decisions per call none of the LLMs tested beats Jev on balanced accuracy. With 25 decisions only Gemini 3.1 Pro is better, by about 2 points. Gemini 2.5 Flash Lite is worse at every level.

Is Jev’s confidence reliable? At the top, yes: above 0.95 Jev was right 96.5% of the time. In the middle band it is too optimistic: when it claims about 0.90 it is right 77.6% of the time. The threshold for handing a case to a person must therefore be tuned on your own data.

How do you call Jev from Delphi? With a JSON POST to OpenRouter’s /api/v1/systemone endpoint, using THTTPClient and System.JSON from the RTL. The full code, including the benchmark, can be downloaded from danieleteti.it.

Who wrote the benchmark’s expected answers? The expected answers were written and reviewed by Claude Opus 5. Daniele Teti set the rules and decided the most disputed case; the other disputed cases were decided by the same model on his behalf, and nobody re-checked every label by hand. That’s why there is no Anthropic model in the comparison.