AI's Economic Impact

Software used to be a simple business. You built a product, put it on a server, and sold it to as many people as possible. Each new user brought in cash without adding much to the bill. In the classic SaaS world, a company like Salesforce or Workday did not care if you logged in once or a thousand times. Their computers did the same amount of work. This flat cost made software a utility with no limit.

Artificial intelligence destroys this cozy arrangement. Every single time a person asks an AI a question, a computer somewhere works very hard. This work costs real money in electricity and chips. At Glean, the infrastructure team sees millions of these queries.

Some answers cost a tiny fraction of a penny, while others involve deep thinking and cost a whole dollar.

If a user asks a complex question to a high-end model like GPT-4o or a new Claude model, the price jumps by a hundred times.

It is like a restaurant offering an all-you-can-eat buffet but some guests are ordering five-star steaks while others only drink water.

Tokens are the new currency of the digital age. In this world, output tokens often cost ten times more than input tokens. When a model sits and "thinks" through a hard math problem or a legal brief, it burns through cash. High-power users can eat up all the profit from dozens of other customers.

In a group of a thousand users, just fifty people can create eighty percent of the total cost. This creates a massive hole in the pockets of software makers, who are stuck between charging too much for basic users or losing money on the heavy lifters.

With the arrival of massive chips like the Nvidia Blackwell B200, the speed of these models is rising, but the price of top-tier intelligence stays high. At places like the Microsoft data centers in Quincy, Washington, the power draw is staggering. Because of these costs, the era of "free" AI features inside every app is going to end. We are moving toward a world where you pay for what you use, just like gasoline or power.

The "free lunch" in software has been eaten by the very machines we built to serve us.

This fundamental shift in economics is already forcing companies to make difficult choices, leading to several unintended consequences.

Unintended consequences

Because AI costs so much to run, companies are now hiding their best features. They put the smartest models behind high paywalls or slow them down on purpose. This creates a "digital divide" where only the rich can afford the best reasoning. Another problem is the "accuracy tax." To save money, some apps route your questions to smaller, cheaper models that are more likely to lie. This sneaky cost-cutting leads to more mistakes in important work like medicine or law.

While some firms resort to cutting quality, others are finding more sophisticated technical solutions to manage these overheads.

The Secret To Smart Routing

The smartest engineers use a trick called model cascading. They do not send every question to the most powerful computer. Instead, they use a tiny, cheap model to guess how hard the question is. If you ask for a joke, a tiny model handles it for almost zero cost. If you ask for a code review, the system sends it to the expensive "frontier" model.

By doing this, Glean and others save millions of dollars.

This keeps the service fast and the company sustainable in an industry where the margin for error is as thin as a silicon wafer.

However, localized engineering solutions were not enough to stop the global supply chain from reaching its breaking point during the mid-2020s.

The Hidden Benefit Of High Prices

High costs are actually making the world smarter. Because every word costs money, developers are forced to write cleaner, better code. We are seeing a new wave of "lean" AI that does more with less energy. This push for efficiency is the best thing to happen to the environment since the move to LED bulbs.

It turns out that making intelligence expensive is the fastest way to make it efficient.

We are learning to respect the power of a single thought because we finally know exactly what that thought is worth in dollars and cents.