The System of Models Era: NVIDIA's Bet on Agentic Routing

The next frontier in AI is not a bigger model. It is a smarter way to route work between the models you already have. On August 11, NVIDIA made that argument explicit, shipping two releases that together amount to a blueprint for how production AI agents will actually run: a lightweight open model built for the repetitive, high-volume work that dominates agent workloads, and an open-source routing library that decides which model handles each step of a workflow.

What happened

NVIDIA expanded its Nemotron family with Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model that activates only 3 billion parameters per token. The design targets a specific and largely invisible problem: always-on agents spend most of their cycles on routine tasks like tool calls, result validation, and delegating work to sub-agents. Running a full frontier reasoning model for every one of those steps is expensive and slow.

The model delivers up to 4x faster output speed and roughly 30% faster agentic task completion than comparable models in its class, according to NVIDIA's own benchmarks. It is fully open: weights, training data, and recipes are published, alongside Nemotron-RL-Agentic-Terminal-Pivot, the reinforcement-learning dataset used to post-train it for coding-agent capabilities. It runs on everything from RTX PCs and DGX Spark to Jetson edge devices and data centers, and ships with NVFP4 and BF16 checkpoints.

Alongside the model, NVIDIA released NeMo Switchyard, an open-source routing library that sits inside agent tools and directs each request to the most capable and efficient model for that step, based on quality, latency, and cost priorities, without requiring an application rewrite.

Why it matters

The number builders should care most about is buried in NVIDIA's announcement: internal benchmarks show Switchyard maintaining frontier-level accuracy while cutting task completion cost to nearly one-third of running Anthropic's Opus 4.8 alone.

That is the economics of the "system of models" made concrete. NVIDIA's own framing is that modern agentic systems are not a single model but an ensemble: a frontier reasoning model plans and orchestrates, while smaller specialized models handle targeted tasks like code review, tool use, and security monitoring.

The partner results tell the same story from independent directions. LangChain routed only 7% of calls to a frontier model across 145 multi-turn Deep Agent tasks and cut cost by 74%, at a 6% accuracy tradeoff. Ramp matched a frontier model's performance while cutting cost by 58% and runtime by 33%. Cognition integrated Switchyard into Devin Desktop and reduced mean cost by 28%. Classmethod saw a 27% reduction in production testing, and Boomi sent 59% of traffic to a 5x-faster fine-tuned model while achieving 100% domain-routing accuracy.

Context

This is not a one-off product launch. It sits inside a broader realignment of how open models are being positioned for the agent economy. Meta released Muse Glimmer, another 30-billion-parameter open model aimed at local agents, days earlier. NVIDIA and Meta both belong to the Nemotron Coalition and the open-weight push that has been urging policymakers to avoid blanket restrictions on downloadable model weights. That fight is exactly what n8n Lab covered in The Great AI Schism, and the economics it produced in AI's Price Floor Just Collapsed.

The common thread across all of it is deployment flexibility and cost control. A model that can run locally, on-premises, or in the cloud, and a router that sends only the hard steps to the expensive model, is the same sovereignty-and-efficiency story the open-weight movement has been telling since the beginning.

Builder Impact

For automation engineers and agent builders, the implications are immediate and practical.

First, routing is becoming a commodity layer. NeMo Switchyard is open source and already plugging into LiteLLM, LangChain, Kong AI Gateway, and Nous Research's Hermes. The "which model handles this call" decision is moving out of bespoke integration code and into a reusable library you can drop into an existing stack without changing it.

Second, the cost curve is now something you can engineer. The difference between routing 100% of calls to a frontier model and routing only 7% of them is the difference between a project that pencils out and one that does not, at a surprisingly small accuracy cost.

Third, open weights plus routing changes the build-versus-buy calculus. You can fine-tune a small open model on your own domain data, route only the genuinely hard steps to a frontier model, and keep the whole thing on your own infrastructure. That architecture is available today on Hugging Face, ModelScope, and OpenRouter.

What's next

The open question is whether routing becomes a feature every agent framework absorbs natively, or a genuine moat for the vendors that control the routers. NeMo Switchyard is "coming to partner platforms soon," and the staged-router pattern Cognition shipped in Devin is likely to spread fast.

The more consequential question for builders is whether a system-of-models architecture becomes the default for agents. If it does, the frontier labs' lock on the economics of AI agents weakens, and the open-weight ecosystem gains real, measurable ground, one routed request at a time.