The takeaway
Orchestrator moves model choice out of application code and into configuration: provider failover, unified billing and per-turn routing behind one endpoint. The leverage shifts to whoever holds per-route quality and cost data.
Why it matters for builders
Agents can now switch providers without rewriting integrations or contracts, but the router does not decide which model deserves which task. Keep routing policy as versioned config next to your evals, measure cost per solved task rather than cost per token, and pin prompt_cache_key so a failover does not silently wipe your prompt cache.
Perplexity Puts Every Model Behind One Routing API
Perplexity shipped the Orchestrator API on November 17, turning the multi-model layer behind its own products into a public endpoint. One key and one base URL now reach models from OpenAI, Anthropic, Google and xAI alongside open-weight models from Z.ai, Moonshot and DeepSeek, with provider routing, automatic failover and a single bill attached. That sounds like plumbing. It is the most consequential layer in the AI stack right now.
What Perplexity Actually Shipped
The company says its internal model layer routes hundreds of millions of requests across more than 85 models and a dozen plus providers every day. Orchestrator exposes that same serving layer to outside developers. The API accepts OpenAI Chat Completions, OpenAI Responses and Anthropic Messages payloads, so an existing SDK keeps working after a base URL and API key change. That compatibility choice matters more than any model in the catalog: migration is measured in minutes, not sprints.
Provider routing is not a parameter you set, it is the default. You name a model and the platform picks a deployment based on observed error rates, capacity and latency. A model ID can map to several deployments, and traffic splits across them as health changes. When a deployment errors, rate-limits or times out, the request is retried on another deployment of the same model: a Claude request that fails on Anthropic can still succeed on AWS Bedrock.
Each attempt is bounded by a time-to-first-token and a total-duration limit, and exceeding either counts as a failure. If every deployment fails, the request returns HTTP 429 with Retry-After, and requests failing before producing output are not billed. That last detail is what operators care about, because retries that silently double cost are the classic failure mode of hand-rolled provider shims.
Billing is unified and provider-independent: each request counts against the answering model's published per-token rate no matter which deployment served it, so a failover does not change the invoice, and there is no separate platform fee. The catalog shows the spread that makes routing worthwhile. Per million tokens, Perplexity's own GLM 5.3 Flash lists at 0.15 in and 0.50 out, GPT-6.1 Sol at 2 and 10, Claude Opus 5.5 at 4 and 20, Kimi K3 at 3 and 15.

Automatic Routing Is the Interesting Part
The preview feature is where the strategy shows. Set the model to perplexity/auto and pass a routing object with a default model and up to five routes. Each route has a name, a plain-language description of the requests it handles, and a model. A decision model reads the conversation and matches it against those descriptions without seeing model names, then Orchestrator calls the assigned model. If nothing matches, or the decision model fails, the default takes over.
The demo case is the most honest one available: routing around refusals. Anthropic's own documentation notes that Claude Fable 5.2 safeguards can block benign cybersecurity and life-sciences work, so the company built profiles that keep Fable as the default and send cybersecurity, biology and post-refusal turns to GPT-6.1 Sol. On a 63-task subset of Terminal-Bench 4.1, Claude Code with Fable alone solved 38 tasks, both routed setups solved 41, at a higher estimated cost per task. A modest gain bought with real money, and Perplexity reported the trade-off.
Routing also happens inside a single conversation. A support thread can start on a cheap model, escalate a billing dispute to Opus 5.5, hand a broken integration to GPT-6.1 Sol and drop back to the cheap default on a thank-you message. Response headers report the selected route and whether the request was routed or fell back, which finally gives teams something to instrument.

Where the Abstraction Leaks
Three constraints matter.
First, media. The pre-flight compatibility check does not cover media, and requests carrying media go to default_model. Pick a text-only or expensive default and image or audio payloads will fail or quietly cost more than planned.
Second, cache and affinity. Each model keeps its own prompt cache, so a conversation hopping between three models leaves three partial caches behind. Orchestrator keeps related requests on the same deployment where possible and honours a stable x-session-id header or prompt_cache_key, but the docs are explicit that affinity is best effort and a failover can move the request elsewhere. On a long agent run, that means a failover saves the request and loses the cache hit, and the bill reflects it.
Third, error semantics shift. A 429 no longer means "you are going too fast", it can mean "every deployment we tried failed". Retry logic keyed on the old meaning will behave badly, and any custom fallback chain on top of the router has to separate client errors, which return 400 and are not retried, from exhausted deployments. Routing also inserts a decision step, so a routing mistake becomes a second failure mode whose fallback is your default model. Teams routing to save money should check what that default costs per month.

The Routing Layer Is Becoming the Market
Perplexity is not alone in this on the same day. DeepInfra put more than 100 of its hosted open models behind Opper's EU gateway, with standard, flex and priority tiers priced differently for the same model. Sarvam opened its Token Factory in India, serving eight models on B200 clusters through one OpenAI-compatible endpoint. Different companies, same conclusion: model choice is moving out of application code and into a configuration layer someone else operates.
For anyone building agents on top of n8n or a custom orchestrator, the durable asset is no longer the integration to one lab, it is the evaluation harness that says which route deserves the traffic. Providers converge on the same request formats so switching costs nothing, which also means the layer above can swap you out. Teams with their own per-route quality and cost data hold the leverage.
What Builders Should Do Now
Start by measuring cost per solved task rather than cost per token. The Terminal-Bench result is the template: 41 of 63 tasks at a higher unit cost can still be the better deal, or not, depending on what a failure costs downstream.
Keep routing policy in your own repository, expressed as data. A default model plus named routes with descriptions is a portable format whether it runs on Perplexity, Opper or your own gateway, and it survives a vendor change. Pin session affinity deliberately by sending a stable prompt_cache_key per conversation, and treat a failover as a cache miss in your cost model instead of being surprised by it.
Finally, watch the promised speed-aware routing. Over a measured 30-day window the fastest provider changed by the day: Anthropic led on 15 days for Claude Opus 5.5 output speed and AWS Bedrock led on the other 15, while Azure ran 27 percent faster than OpenAI on GPT-5.6 Sol. Once selection follows live speed, the router stops being a reliability feature and becomes a latency product, and the case for building your own shim collapses.
The honest summary: the abstraction is real, the failover is real, and the billing simplification is real. The intelligence about which model deserves which request is still yours to build.
The Automation Brief
Read 5 AI stories instead of 50.
The essential moves in AI agents, models, automation and infrastructure — filtered for builders and operators, with the part that actually matters.
No noise. Unsubscribe anytime.
Editorial notes
Stefan Trbojevic
n8n Lab Editorial
17 November 2026
17 November 2026
Sources
AI disclosure: AI assisted with research and drafting. Factual claims are reviewed by an editor.




