Token efficiency: Building AI that spends intelligence wisely
Last year, my company decided to rebuild our enterprise AI platform, which our customers had been using for over a decade. But instead of simply updating it, we started over with a different ambition: to build an AI-native platform using AI.
Last year, my company decided to rebuild our enterprise AI platform, which our customers had been using for over a decade. But instead of simply updating it, we started over with a different ambition: to build an AI-native platform using AI.
Using AI to rebuild a platform of that size gave us first-hand experience with AI economics. We employed AI throughout the entire engineering process to write code, review changes, generate tests and accelerate development. It helped us reduce years of engineering to just a few months but also forced us to keep close watch over every single token we consumed and truly consider why we were consuming it.
This experience challenged one of our key assumptions: It turns out that token consumption is influenced as much by architectural decisions as by choice of model.
Indeed, we found that most of the token consumption waste wasn’t due to complex engineering tasks, but the architecture around them; this is something that is referred to as the architecture tax in enterprise AI.
Recursive agents kept spawning subtasks and subagents, and although they technically completed every task, each handoff introduced new prompts and context. Furthermore, frontier reasoning models were being wasted on routine classification tasks that deterministic code could have completed at a fraction of the cost. And, because we hadn’t clearly defined ‘success,’ retry loops continued firing and burning tokens to chase marginal gains, instead of improving business outcomes. Finally, we experienced context inflation, with workflows carrying information from earlier steps long after they had stopped contributing to the decision being made.
In context
- Topic: Estrategia y Gobierno de TI — Decisiones de portafolio, costo total, gobierno y marcos de referencia.
- Source: CIO
- Published: 15/10/2026
Continue reading at the original source →
Excerpt published automatically by the site radar. The full text belongs to its publisher and is linked above.
Why it matters
There is a wide gap between a company that has technology projects and one that has a technology strategy. The first accumulates initiatives; the second decides what it will not do. Moves like this are usually the moment somebody at the top finally wrote down the list of what gets left out.
My filter is always the same: does this change a decision, or only a tool? If at the end of the quarter the same people are approving the same things by the same criteria, there was no transformation, there was a purchase. The difference shows up in the committee minutes, not in the press release.
What usually goes wrong
What usually goes wrong is the part never discussed with the areas that will carry the work. It gets decided at the top, communicated downward, and the resistance shows up not as opposition but as slowness. Nobody says no; everything simply takes twice as long and nobody can explain why.
What to watch
- How the relationship with current vendors is left, which is usually where the dependency nobody measured is sitting.
- Who ends up owning the decision, and with what budget of their own rather than borrowed.
- Whether business metrics appear or only delivery metrics — cost per transaction beats percentage complete.
How I read this entry
If this landed on my desk, the first thing I would ask for is not a plan. It is an honest inventory of what is already running and what it costs to keep it alive. It almost always turns out that the budget for the new thing comes from switching off something old that nobody wants to name, and that conversation decides whether the project lives.
This entry is an excerpt from the original source, selected by the site radar. The commentary above is the site's own and does not belong to the cited publisher.
Comments
No comments yet. Yours can be the first.
Living through this in your own team?
Open the chat and tell me how you're handling it. I'm interested in comparing notes.
