September 29, 2026
The AI industry spent the first phase of the boom obsessing over training.
Build the biggest model.
Use the largest cluster.
Buy more GPUs.
Train for longer.
That was where enormous amounts of capital flowed.
But as AI models become widely deployed, another part of the equation is becoming just as important.
Inference.
Inference is what happens after a model has already been trained.
Every time someone asks an AI system a question, generates an image, writes code with an AI assistant or gives an autonomous agent a task, inference infrastructure has to process the request.
And now investors are pouring billions into companies whose entire business is making that process faster, cheaper and more efficient.
One of the clearest examples is Modal Labs, an AI infrastructure company that is reportedly nearing a $750 million funding round at a $15.75 billion valuation, only months after its previous financing valued the company at $4.65 billion.
That kind of valuation jump tells us something important.
The AI infrastructure opportunity is no longer just about building models.
There is an enormous business forming underneath them.
Training is only the beginning
Imagine a company spends billions of dollars training an advanced AI model.
The model is finished.
The training cluster shuts down.
The model is released.
What happens next?
Millions of users start asking it questions.
Businesses connect it to applications.
Developers build software around it.
AI agents call it repeatedly.
Suddenly, the expensive part isn’t only training.
It is running the model.
Every request consumes computing resources.
And unlike training, inference happens continuously.
Twenty-four hours a day.
Seven days a week.
For potentially millions of users.
That makes inference economics fundamentally different.
A model that is expensive to train can still become extremely valuable.
But if it is also extremely expensive to run, the company needs infrastructure capable of delivering responses at acceptable speed and cost.
Why inference is becoming its own industry
The traditional cloud model wasn’t designed specifically for today’s AI workloads.
AI inference has unusual characteristics.
Requests can arrive in unpredictable bursts.
Models can be enormous.
Different customers may require different models.
Some workloads require extremely low latency.
Others can tolerate slower responses.
Some applications need thousands of simultaneous model calls.
Others may run AI workloads intermittently.
Infrastructure providers therefore need sophisticated systems for allocating GPUs and other accelerators dynamically.
That is the market companies such as Modal are targeting.
Instead of every AI company building its own infrastructure stack, specialised providers can manage the complexity underneath.
The economics are changing rapidly
TechCrunch reports that Modal’s new financing would more than triple its valuation from the $4.65 billion level reached just four months earlier.
The reason is straightforward.
Demand for inference infrastructure is growing rapidly, particularly as businesses increasingly deploy open-source models.
That creates an interesting shift.
The AI market is beginning to separate into two infrastructure problems.
Training
This is where a model learns.
It requires enormous clusters and often involves massive one-time or periodic computing campaigns.
Inference
This is where the model works.
It requires infrastructure capable of handling continuous demand efficiently.
The second category could ultimately become the larger long-term market because every successful AI application generates inference demand every time someone uses it.
Open-source AI makes the problem even bigger
One of the most interesting aspects of the inference boom is the growth of open-source and open-weight models.
A company does not necessarily need to rely on an AI provider’s own cloud API.
It can download a model, deploy it itself and customise the system for a particular application.
That creates enormous demand for specialised inference infrastructure.
A company might want to run an open model privately.
Another might need to serve millions of requests.
Another could require inference close to its customers for low latency.
Another may want to switch between several models depending on the task.
The infrastructure layer has to make all of this possible without requiring every company to become an expert in GPU scheduling.
The GPU is becoming only one part of the problem
For years, the AI infrastructure story was dominated by GPUs.
That remains extremely important.
But inference efficiency involves much more.
Networking matters.
Memory bandwidth matters.
Storage matters.
Software optimisation matters.
Scheduling matters.
Model compression matters.
Quantisation matters.
Caching matters.
And increasingly, specialised chips matter.
A company can potentially make an AI model dramatically cheaper to run without changing the model itself simply by improving the infrastructure underneath it.
That creates a huge economic opportunity.
If an infrastructure company can reduce the cost of running a model by 30%, 50% or more, the savings can become enormous at scale.
AI companies are under pressure to reduce token costs
There is another reason inference infrastructure is becoming strategically important.
AI companies are discovering that the cost of generating a response can become a major component of their economics.
The more capable the model, the more computation may be required.
Agents can be particularly demanding.
A normal chatbot might generate one response.
An autonomous agent could make dozens or hundreds of model calls while completing a complex task.
That means the rise of agents could multiply inference demand.
The industry is therefore entering an unusual feedback loop.
Better AI creates more useful applications.
More useful applications create more usage.
More usage creates more inference demand.
More inference demand creates more infrastructure investment.
And that infrastructure makes it cheaper to deploy even more AI.
The AI cloud is becoming a market of its own
This is why Modal’s funding story matters.
The company is not trying to compete with OpenAI or Anthropic by building the world’s smartest model.
It is building infrastructure underneath AI.
That makes it part of a much broader ecosystem.
Companies like Nvidia build the chips.
Cloud providers provide the computing infrastructure.
Specialised inference companies optimise how that infrastructure is used.
Model companies build the intelligence.
Application companies build products on top.
The result is beginning to resemble the software stack that developed around the internet.
And each layer can become a major industry.
There is a hidden battle over efficiency
The next major AI breakthrough may therefore not be a new benchmark record.
It may be a dramatic reduction in the cost of inference.
Imagine two models that provide roughly equivalent results.
One costs $1 to run.
The other costs $0.20.
The cheaper model has a huge commercial advantage.
It can be deployed more widely.
Companies can offer more generous usage limits.
Agents can perform more steps.
Developers can build more ambitious applications.
That means AI efficiency is becoming strategically important.
The companies that make inference cheaper could therefore influence how quickly AI spreads through the economy.
The industry is moving from model competition to infrastructure competition
The first AI race was largely about models.
OpenAI versus Anthropic.
Google versus Meta.
Closed models versus open models.
Now another competition is emerging beneath it.
Who can run those models most efficiently?
Who can provide the lowest latency?
Who can handle the largest workloads?
Who can make open models commercially viable?
Who can squeeze the most performance out of every GPU?
And who can build infrastructure that allows AI agents to run continuously without costs becoming impossible to manage?
Those questions may determine which AI applications actually survive commercially.
The quiet infrastructure boom
The public sees AI through chatbots.
It sees images, videos, coding assistants and agents.
But underneath all of that is a rapidly expanding infrastructure industry.
Modal’s reported $15.75 billion valuation is one indication of how much investors believe that infrastructure could be worth.
And the timing is significant.
The AI industry is moving from experimentation toward permanent deployment.
That means inference is becoming less like an optional technical feature and more like the electricity bill of the AI economy.
Every AI interaction consumes resources.
Every agent consumes more.
Every enterprise deployment creates another recurring workload.
And every successful AI application creates more demand underneath it.
The biggest AI companies may therefore not be the only winners.
A new generation of infrastructure companies is emerging around them.
And if the AI economy really does scale to the size its biggest companies are predicting, the business of running AI could eventually become just as important as the business of building it.
Laat een reactie achter