The era of shadow AI: How companies are losing control of models and API keys

A major shift is happening in enterprise AI routing. According to recent data, Chinese open-weight models like DeepSeek and GLM-5.2 now account for roughly 30% to 46% of enterprise API token usage.

Enterprises have largely stopped asking whether open-weight models are good enough. That question has resolved itself. But this rapid adoption has triggered a much deeper crisis — one of infrastructure, visibility, and control.

The enterprise AI stack is fragmenting faster than IT can secure it. And companies that try to solve this by forcing one rigid standard will be the ones left behind.

The new shadow IT is a credit card

Right now, a developer with a corporate credit card can access a top-tier AI model in under two minutes. There is no rotation policy, no spend ceiling, and no way to revoke it when they inevitably change teams. Companies spent ten years losing track of all the software they signed up for. Now the same thing is happening with APIs — but in months, not years. And this time, sensitive company data can also be leaked.

When your software talks directly to an AI provider, your data leaves with no one checking it. Every other exit has security, but this one has a “bearer token” — essentially, a master password that anyone holding it can use.

Nobody should be sharing a raw provider key. Virtual keys, per-team budgets, and one-click revocation are solved infrastructure problems in every other domain. AI got a temporary pass because it moved fast, but that pass is expiring.

The pattern we see repeatedly at nexos.ai is a company arriving convinced they are using only a few leading providers. Then they switch on visibility and discover models nobody on the platform team can account for and an unmanaged number of API keys already running in production. That isn’t negligence — it’s simply what happens when technology moves faster than governance.

The invisible cost of coding agents

When visibility fails, costs spiral. We at nexos.ai see this most clearly in how engineering teams have adopted AI coding agents like Claude Code, Codex, Cursor, OpenCode, and Zed. The more these tools are used, the higher the volume of AI requests, making it incredibly difficult to track where AI usage originates or how to control the costs.

Recently, we partnered with Nord Security, one of the world’s leading cybersecurity companies, to assess their production AI traffic. What we found is a textbook example of how quickly AI adoption outpaces a company’s ability to manage that usage efficiently. While analyzing their case, we discovered that while AI coding agents heavily reuse prompt context, they manage caching independently and inefficiently. Caching and smart routing both reduce AI costs, but through different mechanisms. Caching eliminates redundant tokens. Routing ensures the remaining tokens are processed by the most cost-efficient model that won’t sacrifice quality. The real win is that both work behind the scenes, allowing companies to scale without forcing developers to change a single line of their existing workflow.

Before optimization, Nord Security was seeing an initial cache hit rate of only 14%. Most of the repeated context was being sent directly to the LLM and processed entirely from scratch on every single call. The short-sighted solution would have been to force developers to change their tools. Instead, we implemented caching directly at the gateway, automatically reusing repeated context before requests even reached the LLM. An LLM gateway provides the control needed to ensure that these improvements are enforced across the entire company.

By processing only new information, Nord Security achieved a 77% prompt cache hit rate and a 46% reduction in total LLM costs compared with their production baseline. Most importantly, this required zero prompt re-engineering, zero model migrations, and zero workflow disruptions for their developers.

Betting on one model is a trap

To regain control over sprawling usage and rising costs, many procurement and IT teams instinctively try to standardize. They want to pick a single frontier provider and make it the company standard. This is the riskiest AI strategy a company can adopt today.

The LLM market is currently going through a brutal price war, made even more complicated by the rapid rise of open-source models that are now catching up to industry leaders in quality. In this environment, the cost of intelligence is dropping faster than most enterprise budgets can keep up with. The biggest losers in this new reality are the companies sitting on rigid, yearly contracts with heavy usage commitments. Because they lack a central gateway to keep their infrastructure flexible, they are effectively trapped — forced to pay “yesterday’s prices” for legacy models while more agile competitors instantly switch to cheaper, faster alternatives as they hit the market.

Another core issue of AI strategy is timing. Model leadership shifts in months, while enterprise architecture takes years to catch up. The teams that hard-coded a single provider’s SDK two years ago are paying a heavy migration tax today just to reach a cheaper, better alternative. Switching models should be a configuration change, not a sprint. If trying a new model costs engineering time, teams won’t even try — and will inevitably fall behind the ones that can adapt quickly.

We need to stop treating model selection as a one-size-fits-all policy and start treating it as a per-task routing decision. The strongest stacks nexos.ai teams see in production aren’t standardized — they are hybrid by design. They use frontier models where deep reasoning quality is the actual product, and they deploy open weights for classification, extraction, summarization, and anything running at massive volume.

In short, the smart thing is to pick the right model for each job, not one model for every job. Treating the one-model-for-all-jobs approach as a silver bullet and assuming that it can cover all needs is exactly how companies end up overpaying for the easy work and under-serving the hard work.

You can’t audit what you can’t see

Ultimately, the most pressing question for the enterprise isn’t which model is the smartest. It’s whether you can run one and legally prove where the data went. That is an infrastructure question.

If an AI-driven decision is challenged — especially under emerging frameworks like the EU AI Act — can you reproduce it? Do you know which model version was used, what data it saw, under which policy, and at what exact time? Most organizations find out the answer is “no” during the audit, rather than before it.

Regulatory readiness isn’t a document you write at the end of a project. It is a logging decision you make at the beginning. We don’t tell customers which models to trust — our job is making sure they can answer the auditor’s questions. Enterprises absolutely should have access to the whole frontier — even the parts their procurement team hasn’t written a policy for yet. The actual risk is access without evidence — andThe gateway is where that evidence gets created, because it’s the one place every request must pass through. Locking the whole company into a single AI model was never a realistic goal. The goal is knowing which models you have, who's calling them, and what they cost you.

How to win the AI adoption game

The shift to open-weight models, the explosion of coding agents, and the spread of ungoverned API access aren’t separate problems — they’re the same problem arriving from three directions at once. And the problem is that enterprise AI is scaling faster than the infrastructure meant to control it.

The good news is that the fix doesn’t require slowing anyone down. You don’t need to block providers or force a single model on your developers. You need a layer that sees every request, routes it to the right model, caches what’s repeated, and logs what an auditor will ask about later. The companies that win the next phase of AI adoption will be the ones who use and govern AI efficiently.