Your Chatbot Doesn't Need an LLM for Every Click

LLMs have made chatbots dramatically better.

But they have also created a strange architecture.

A user opens a chatbot and sees:

The user clicks “Track my order.”

And somewhere behind the scenes, we send context to a large language model and ask it to figure out what the user wants.

But… they literally clicked Track my order.

Why are we asking a large model to reason about something we already know?

The expensive path

A lot of modern AI applications effectively work like this:

User
  ↓
Chatbot
  ↓
LLM
  ↓
Understand intent
  ↓
Choose action
  ↓
Call API

This makes sense when the user's request is ambiguous:

“My package was supposed to arrive yesterday, but I'm travelling tomorrow. Can I get it delivered somewhere else?”

There is language to understand. There is context. There may be multiple possible actions. An LLM is useful.

But consider:

The intent is already known. Or:

These aren't language-understanding problems.

They're decision problems with a very small set of possible answers.

Put a tiny decision layer in front

This is where something like Jev becomes interesting.

Instead of sending every interaction directly to an LLM:

User
  ↓
Jev
  ↓
Decision
  ├── predefined action
  ├── API / tool
  ├── small model
  ├── large model
  └── human

Jev's job isn't to generate the response.

Its job is much smaller:

Given the current context and a constrained set of possible actions, what should happen next?

That changes the economics considerably.

Consider a customer-support chatbot

Imagine an e-commerce support bot. The first screen contains:

The customer clicks Track Order.

There is almost nothing to reason about. Jev can receive something conceptually as small as:

Context:
User clicked "Track Order"

Possible decisions:
- TRACK_ORDER
- ASK_FOR_ORDER
- ESCALATE

It returns:

TRACK_ORDER

The application calls the order API. Done.

• No long conversation needs to be generated.
• No huge context window needs to be sent.
• No expensive model needs to write three paragraphs explaining that it understands the customer wants to track an order.

Why this matters

The obvious benefit is cost.

If you're processing thousands or millions of chatbot interactions, repeatedly invoking a large model for deterministic or near-deterministic interactions is wasteful.

A constrained decision request can be tiny.

But arguably the more noticeable benefit is speed.

Consider the difference between:

Click
 → construct prompt
 → send conversation context
 → LLM inference
 → parse generated response
 → determine action
 → execute

and:

Click
 → Jev decision
 → execute

For predefined chatbot flows, the second path should naturally require far less work.

That can make the chatbot feel less like “AI is thinking…” and more like click → done.

The LLM doesn't disappear

This isn't an argument against LLMs.

It's an argument for using them where they provide value.

Suppose the customer ignores all the buttons and writes:

“I ordered two prayer mats but only one arrived, and I'm leaving town tomorrow. Can the second one be delivered to another address?”

Now we have a different problem.

There are multiple intents. There is context. The appropriate action isn't immediately obvious.

So the routing might become:

User message
      ↓
     Jev
      ↓
Complex / ambiguous
      ↓
     LLM
      ↓
Understand + reason
      ↓
Tool / API

The expensive model is still available. We simply don't wake it up for everything.

There are really three classes of chatbot interactions

A useful chatbot architecture can think about requests roughly like this:

Jev becomes the traffic controller

This is the part I find more interesting than simply calling Jev a cheaper model.

It doesn't necessarily have to compete with GPT, Claude, Gemini, or other LLMs. It can sit before them.

                    ┌─→ Direct action
                    │
                    ├─→ API
                    │
User → Chatbot → Jev ├─→ Small model
                    │
                    ├─→ Large model
                    │
                    └─→ Human

The question Jev answers isn't:

“What should I tell this user?”

It's:

“What should handle this interaction?”

That's a much smaller problem. And smaller problems often don't need giant models.

Imagine this at scale

Suppose a support chatbot processes 100,000 interactions. A large portion might be things like:

• Track order
• Check delivery
• View invoice
• Start return
• Change address
• Talk to agent

If 70,000 of those interactions can be resolved through deterministic or constrained decisions, why send all 100,000 through an expensive generative model?

Route the simple interactions through the cheapest appropriate path. Reserve the powerful model for the interactions where its intelligence actually matters.

The exact savings will depend on the models, prompts, caching, infrastructure, and conversation lengths involved.

But the architectural principle remains:

Model cost should scale with problem complexity, not simply with message count.

This isn't only about chatbots

Once you look at the problem this way, the same pattern appears everywhere:

Many of these decisions have a tiny number of valid outputs. Yet we're increasingly using enormous generative models to make them.

That feels backwards.

The future may be fewer unnecessary LLM calls

For the last few years, the default architecture has increasingly become:

Something happened
        ↓
      Ask LLM

A better architecture might be:

Something happened
        ↓
Can this be handled cheaply and deterministically?
        ↓
       Yes ─────→ Do it
        │
        No
        ↓
Does this require lightweight decision-making?
        ↓
       Yes ─────→ Jev
        │
        No
        ↓
       LLM

The goal isn't to avoid LLMs. The goal is to stop using a large model for decisions that never required one in the first place.

For a chatbot full of predefined actions, that can mean tiny decision requests, very little inference cost, and faster interactions.

And when something genuinely difficult appears? That's when you bring out the big model.

Don't use an LLM when you don't need one.

That's the idea behind Jev.
Jev — decisions, not generations.