With all the excitement around large language models over the last few years, it's easy to forget that machine learning has been around a lot longer than chat models, and that other models are built for very different jobs.
Today, most AI systems leave a lot of efficiency on the table by calling a full LLM just to get a simple true or false response or a basic classification.
Google recently released Gemini Decision 1, also known as a 'System 1' model. Instead of generating text token by token like a chat model, it reads your input in one pass and returns one of three structured statistical responses:
- True or false
- A score
- A category (classification)
Why decision models are fast and cheap
Because there are no generated text tokens, both the speed and the pricing look very different from a standard LLM.
You can ask up to 1,000 questions at once in a single request, and the extra latency per question is tiny. In my tests, asking 1 question took about 180ms, while asking 100 questions at once took about 250ms. That makes it practical to run a whole rubric of small checks together and combine them in your own application logic.
On pricing, Gemini Decision 1 costs $0.10 per million input tokens with $0 output token cost (plus up to 10,000 free requests per day on Google's free tier). If you're used to paying steep output-token rates on chat models, that's a big shift.
When I saw the release, I wanted to see how it actually held up in real workflows. I went through my old coding sessions and public datasets looking for places to either replace a slow LLM call or add a fast check that improves the overall system, ending up with 800 scenarios across 24 categories.
Experiment 1: model routing
One of the first places I tried it was model routing.
Instead of running your heaviest frontier model on every single step of an agent session, a router decides whether a specific request can go to a lighter, faster, cheaper model or needs larger reasoning capabilities.
The catch with LLM-based routers is that you're adding an extra LLM call (more latency and cost) in the hopes of saving more than you just spent. A fast decision model gives you a much cheaper way to grade the prompt up front.
First, I treated it like a normal LLM and asked one broad, subjective question: "Should this request go to a small model or a large model based on the contents of this prompt?"
The result was barely better than a coin flip at 0.72 out of 1. Not reliable enough to trust.
So I split that single broad prompt into four or five narrow, concrete questions in the same request: does this prompt touch authentication or billing? Is it just a styling tweak or a documentation update?
With that one change, accuracy jumped to 0.88 out of 1.
Five examples across 800 scenarios
That swing made me curious where else decision models shine and where they fall flat. Across the 24 categories I tested on 800 real sessions, here are five examples running from best to worst:
- Customer support team routing (~100% accurate): Triaging incoming customer queries to the right team was nearly 100% accurate, averaging 230ms per request at $0.34 per 10,000 requests. Basic classification like this is a great fit.
- Prompt injection detection (99.7% accurate): Checking incoming queries for prompt injection and text-based hijacking hit 99.7% accuracy.
- Deciding what to remember (88% accurate): Scanning recent conversation messages to spot a durable rule, instruction, or fact worth saving to memory hit 88% accuracy. Running a full LLM after every turn to check for memory updates gets expensive fast, so this is a useful filter.
- Splitting code files for retrieval (plain code won): I tried using the model to slice large code files into relevant chunks for retrieval, and it didn't perform well. Plain deterministic code logic was simpler and worked better.
- Predicting user pushback (46% accurate): This was the test I was most hopeful about. I wanted a persistent check that could spot when I was about to step in and correct my agent before it happened. Instead, it scored 46%, worse than a coin flip. That question was simply too broad and subjective for a single pass.
Where decision models fit
All of these tests ran on my own sessions and a public dataset, so they aren't exhaustive. Edge cases always exist, and testing on your own data is still the best way to learn where a model holds up.
The pattern across all 800 scenarios was consistent: break your check down into the narrowest, most specific, and least subjective questions you can, and a decision model like Gemini Decision 1 will be fast, cheap, and accurate. Save broad subjective judgment for full language models, and use plain deterministic code whenever simple code logic already solves the problem.
This is Part 1 of my exploration of decision models. I'll follow up with more results as I build these checks into real projects.
Top comments (1)
Patrick, your move from one broad question to four or five narrow ones matches something we measured from the other side. We tried to predict "will a cheap model be enough" from the prompt alone, using a prompt embedding on 539 coding tasks labelled by running the code. Held-out AUC was 0.594, and the best threshold scored the same as never escalating. Eleven crude surface features, like length and token counts, beat the embedding. A broad question about difficulty seems to leave little in the prompt to read, which may be the same effect as your 0.72 on the single broad routing question. The one thing that did better was showing the router the cheap model's draft (AUC 0.64), though it did not survive a multiple-comparison correction, so we call it promising. The write-up is here: dev.to/tom_jones_230c4659491adcd/w...
Our regular tests are on tool calls. They use 800 tasks from the public BFCL set (single calls, choosing between tools, parallel calls) plus trap prompts written to tempt a wrong call, with the same scorer for every model so the results compare. Alongside accuracy by task type we record speed, tokens and cost per right answer. Our October write-up covered 68 models, among them Claude Opus 5.5, Grok 4.7, Llama 3.3 70B and DeepSeek V4 Flash: dev.to/tom_jones_230c4659491adcd/t.... Current results are at tirtha.ai/bench.
We have also run a decision model as an approve-or-reject check on tool calls, measuring how many wrong calls it lets through: dev.to/tom_jones_230c4659491adcd/w...
Gemini Decision 1 in that role is the case I would most like to see run. If you want the 800 requests or our setup, ask and I will send them.