Over the past couple of weeks, I've been seeing stories about enterprise companies facing skyrocketing AI token costs. One article cited Gartner research suggesting AI coding costs could eventually rival developer salaries. Other reports discussed companies facing monthly token bills in the tens of thousands of dollars and responding with spending caps and token budgets.
And tucked somewhere into many of those stories was a word that kept jumping out at me:
Unexpected.
To an extent, I sympathize.
When I first started building with AI, I didn't fully understand how much my prompts cost either.
Every prompt costs money.
Every instruction costs money.
Every piece of context costs money.
Every follow-up costs money.
For those of us who spent years writing freely without thinking about computational costs, it's a strange adjustment. You're accustomed to throwing words around without consequence. Then suddenly every character has a price tag attached to it.
But while I understand the surprise, I don't understand the response.
Because from where I'm sitting, token rationing looks a lot like treating the symptom while ignoring the disease.
Vibe Coding Was My Wake-Up Call
I understood a little about token economics. But when I started experimenting with vibecoding, that's when the meter became visible. I watched the AI read a file. Then reread the file. Then rewrite the file. Then explain the rewrite. Then review the rewrite. Then decide it could probably do a better rewrite. Then repeat the process.
Don't get me wrong. The results were often impressive.
But I finally understood why some people were getting blindsided by AI costs.
The thing about agentic systems is that they can look productive while quietly setting your budget on fire. As a bootstrapper operating on a budget that can best be described as "good luck," I learned a few things quickly.
I couldn't afford inefficiency.
I couldn't solve problems by throwing money I didn’t have at them.
I couldn't shrug off a surprise invoice and tell myself I’d figure it out next quarter.
If I wanted to build with AI, I had to build with intent and specificity because I simply couldn't afford not to. That mindset shaped how I approached agentic AI, and honestly, I think it's why I look at these enterprise token-cost stories differently.
The Cost Problem Isn't Always a Usage Problem
The narrative I'm seeing is straightforward.
Token costs are rising for enterprise companies.
Therefore employees must be using too many tokens.
Therefore token budgets (aka token rationing) will solve the problem.
Simple, right?
Maybe too simple.
When companies suddenly introduce weekly token allowances after receiving a frightening invoice, it's difficult for me not to see the move as reactive. Maybe I'm wrong. Maybe every one of these policies is the result of careful analysis, months of planning, and extensive architectural reviews.
But from where I'm sitting, it often looks like the corporate equivalent of slamming on the brakes after seeing a scary number.
And that's the problem.
A large bill doesn't automatically tell you why the bill is large. It just tells you that something is expensive. The real question is whether the expense comes from productive work or inefficient systems. Those are very different problems.
Choose the Right Model
One of the first things I learned was that not every task needs the newest, smartest, most expensive model.
The industry loves announcing the next breakthrough model. That's how these tools are marketed: Smarter. Faster. Better. More capable. And usually more expensive.
Does that mean you should automatically use the cheapest model available? Of course not.
But there are plenty of repetitive tasks that don't require the computational equivalent of a rocket scientist: Classification. Routing. Tagging. Basic summaries. Extraction. Formatting.
For those kinds of tasks, cheaper models often have more than enough brainpower to get the job done. Reserve the premium models for premium problems. That sounds obvious. Yet I suspect a lot of organizations are spending money where they don't need to.
Agentic AI Is Amazing. It's Also Expensive.
This is where a lot of companies run into trouble.
Agentic AI is incredible. It's one of the most exciting developments I've seen in years. But let's be honest about something:
Every loop costs money.
Every self-review costs money.
Every tool call costs money.
Every attempt to improve an answer costs money.
Every internal debate costs money.
When you have an LLM arguing with itself for ten rounds trying to squeeze out a marginally better answer, that conversation isn't free. Neither is the next one or the one after that.
One of the lessons emerging from recent research is that more tokens doesn’t lead to better results. Sometimes they produce more activity—and those aren’t the same things.
Without limits, an agent can spend an astonishing amount of time chasing tiny improvements while consuming enormous quantities of tokens. That's why guardrails matter. Building syntax like max_loops or maybe max_tokens into your workflow, along with stopping conditions, is a great way to declare boundaries.
Most AI providers offer ways to implement these controls. Use them. A machine doesn't know your budget, so you have to teach it.
What Happens When Developers Start Thinking Like Accountants?
This is the part that worries me.
Token rationing sounds reasonable until you think about how it changes behavior. Imagine you're a developer working under a strict token budget. You discover an issue. Should you investigate it further?
Maybe. But how many tokens will that cost?
Should you run another test? Should you let the model review the code again? Should you explore an alternative implementation?
Suddenly those aren't engineering questions anymore. They're budget questions. And once that shift happens, people start optimizing for token preservation instead of outcomes.
That's not because developers are lazy. When the goal becomes staying under budget, staying under budget becomes the priority. Innovation slows. Experimentation becomes harder to justify. And quality can suffer because people become more focused on conserving resources than solving problems.
Good engineering has always involved constraints. Memory constraints. Storage constraints. Processing constraints. Time constraints. But token quotas often feel like working with one foot on a banana peel. They don't necessarily remove inefficiency from the system. They simply distribute the anxiety.
What Actually Works
If the goal is reducing token costs, there are better options available.
Prompt Compression
Prompt compression is probably the easiest place to start. Many prompts are bloated with unnecessary explanations, repetitive instructions, jargon, and fluff.
Most of the time the model doesn't need a five-paragraph introduction. It needs a just-the-facts step. Cleaner prompts often produce cleaner results while consuming fewer tokens. Remember when GIGO was a thing?
Prompt Caching
Prompt caching is one of my favorite examples because it's so easy to understand—we’ve all been through the non-AI version of it.
Imagine calling customer support. You spend five minutes explaining your problem. The representative says, “I’m sorry for the inconvenience. Let me transfer you to Joe in Department. He can help you.”
Joe comes on and says, “How can I help you?” and now you spend another five minutes explaining the same problem. This isn’t Joe’s department but it is Beth’s. He transfers you to her and you take another five minutes explaining to Beth…
Most of us have lived through this bureaucratic nightmare. By the end of the call you're exhausted, annoyed, and wondering why nobody bothered taking notes.
That's life without prompt caching.
Now imagine every department has access to a shared notepad containing the details of your issue. You explain it once and everyone else reads the notes. The process moves forward and it only takes you 10 minutes instead of 30 to get the solution.
That's prompt caching. The model remembers background information instead of repeatedly processing the same context over and over again. Less repetition means fewer tokens. Fewer tokens mean lower costs. And unlike token rationing, nobody's productivity has to suffer in the process.
Traditional Code Still Matters
This is another place where my perspective may differ from some of the current hype.
I know no-code tools are popular and vibe coding is attracting a lot of people who don't want to touch traditional programming. That's fine. But there's no getting around the fact that code is still incredibly useful.
Python has saved me a lot of money. Not because it's magical. Because it's efficient.
Instead of sending mountains of raw data directly to an LLM, Python can filter, validate, transform, aggregate, and organize that data first. Every task handled by software is one less task consuming tokens. Businesses spent decades learning when to use databases, scripts, APIs, and automation tools. AI doesn't replace those lessons.
If anything, it makes them more important.
Using an LLM to perform work that a simple script could handle efficiently is a bit like using a bulldozer to plant flower seeds.
Sure, you can do it. But should you?
The Real Secret
I don't know the exact mechanisms behind these companies making headlines for massive token bills. They're not clients. I don't have access to their systems. I don't sit in their meetings.
What I do know is that token rationing feels like corporate theater.
It creates the appearance of action while pointing at what may be the wrong problem. And if there's one thing I've learned while building systems, it's that solutions built on incorrect assumptions make the problem worse.
The first phase of AI adoption was obsessed with capability. The next phase will be obsessed with efficiency.
The organizations that thrive won't necessarily be the ones spending the most on tokens. They'll be the ones getting the most value from every token they spend. They'll:
- Choose the right model for the job.
- Put guardrails around agentic systems.
- Leverage caching.
- Use traditional software where traditional software makes sense.
Most importantly, they'll recognize that many token-cost problems begin long before a developer ever submits a prompt.
Because when an engineering problem is treated like a purchasing problem, the invoice may get smaller for a while. The inefficiency doesn't. And no amount of token rationing can fix that.