By May 2026, the cost calculus for production AI workloads has changed enough that the single-vendor default that worked in 2024 is now visibly leaving money on the table. Three pricing moves in the past 90 days set the new floor: xAI cut Grok 4.3 API input pricing roughly 40% on April 30 and shipped a 1M-token context tier for the same price as the prior 256K window; Anthropic's Sonnet 4.6 settled at $3 input / $15 output per million tokens with the 1M context tier in beta and a +2.4 SWE-bench gain over Sonnet 4.5; OpenAI's GPT-5.5 Instant became the new default tier underneath GPT-5.4, which retained frontier-leader positioning at premium pricing. The vendor-by-vendor read is interesting; the cross-vendor read is where the cost lever is. Workloads routed across these three by profile rather than locked to one vendor cut monthly LLM spend on the order of 40% in this desk's modeling, before tooling-layer optimizations land another 15-20%.
This piece is for operators running production LLM workloads with a monthly bill that has crossed five figures. The sub-five-figure regime tolerates single-vendor laziness; above it, routing pays for the engineering investment and then some.
The 2026 Routing Premise — One Model Does Not Fit All
The case for routing rests on a structural shift the 2026 frontier landscape made operative. In 2024 the capability gap between premium tiers and budget tiers across vendors was wide enough that high-stakes workloads ran on the premium tier of one vendor and lower-stakes workloads ran on whichever budget tier the developer happened to have wired in. The capability gap was the moat; the cost gap was the price of crossing it.
By May 2026, the within-vendor capability spread has compressed for many workload classes — Sonnet 4.6 is competitive with Opus 4 on a meaningful share of production tasks at one-fifth the cost; Grok 4.3 API at 1M context outperforms premium tiers from a year prior on long-document workloads; GPT-5.5 Instant handles general reasoning tasks that previously required GPT-4 or GPT-5.0 routing. Across vendors, the differentiation has migrated from raw capability to workload fit — coding versus general reasoning versus long-context analysis versus agentic execution versus multimodal. Each frontier vendor now has a workload class where it is genuinely the best price-performance choice and at least one workload class where it is paying premium for capability the workload does not need.
The routing premise is operational rather than philosophical: send each workload to the model where its profile matches the model's strength, and the bill drops by the sum of the mismatches the prior single-vendor stack was carrying.
Workload Categorization For Routing
Routing requires that workloads be classified accurately enough that the routing rules send the right traffic to the right model. The classification taxonomy that maps cleanly to the 2026 frontier landscape has roughly five categories.
Category 1 — Coding and code-adjacent work. Code generation, code review, code-aware refactoring, working in IDE-integrated agents, computer-use agentic tasks where the agent is operating a development environment. Volume in production AI stacks is typically heavy here.
Category 2 — Long-context analysis. Document review, multi-source research synthesis, large codebase analysis, transcript processing, contract or compliance review against extensive policy. The workload signature is "single prompt, very large input, structured output."
Category 3 — General reasoning and knowledge work. Customer-support response generation, internal Q&A, drafting, summarization, structured data extraction from unstructured sources. The workload signature is "moderate input, moderate output, broad domain coverage required."
Category 4 — Agentic execution. Multi-step agent workflows that combine reasoning, tool calls, and conditional branching. The workload signature is "many small calls within a single agent run, latency-sensitive, with tool-use compatibility a hard requirement."
Category 5 — Multimodal. Vision input, audio transcription paired with reasoning, document parsing where layout matters, video input processing. The 2026 multimodal landscape has hardened workload-specific differentiation that single-vendor routing tends to under-serve.
Grok 4.3 — Cheap Long-Context, Native Video, Agentic
The April 30 API release positioned Grok 4.3 as the price-performance leader for two specific workload classes. The 40% input price cut combined with the 1M-token context window made Grok 4.3 the cheapest meaningful long-context option among Western frontier models — substantially cheaper than Sonnet 4.6's premium 1M context beta tier, with comparable quality on the workloads where context size is the binding constraint rather than reasoning depth. Native video input on the API exposed multimodal video-reasoning at a price that previously did not exist for production deployment.
The fit profile: Category 2 (long-context analysis), Category 4 (agentic execution where Grok Computer integration is in play), Category 5 (video-input multimodal). The constraint: persistent memory between sessions is absent, which makes Grok 4.3 a poor fit for agent workflows that require state across turns rather than within turns. For Category 1 coding, Grok 4.3 underperforms Sonnet 4.6 on the standard SWE-bench cuts and is not the right router target for that traffic.
Sonnet 4.6 — Coding, Computer Use, Reliability
Sonnet 4.6 settled at $3 input / $15 output per million tokens with the 1M-context beta tier and shipped a +2.4 SWE-bench gain over Sonnet 4.5, retaining Anthropic's coding-task lead in the production tier. Computer use — the desktop-automation agent surface — got refinement in the 4.6 release that made it noticeably more reliable on multi-step desktop workflows than 4.5 was.
The fit profile: Category 1 (coding and code-adjacent work) is the strong fit; Category 4 (agentic execution where reliability and tool-use polish matter more than per-call price) is the secondary fit. Sonnet 4.6 over-pays for Category 3 (general reasoning, knowledge work) where GPT-5.5 Instant handles the workload competently at lower cost. For Category 2 (long-context) the 1M beta tier closes the context-size gap with Grok 4.3 but at a higher per-token rate; the routing rule favors Sonnet 4.6 for long-context where reasoning depth matters more than context size, and Grok 4.3 where context size is the binding constraint.
GPT-5.4 — General Reasoning, Knowledge Work, Default Fallback
GPT-5.4 retained frontier-leader positioning across a broad set of general reasoning benchmarks through May 2026, with the new GPT-5.5 Instant tier sitting underneath as the workhorse default for the cost-sensitive share of OpenAI traffic. The pricing structure rewards a two-tier internal routing decision before the cross-vendor router runs at all — premium-tier traffic to GPT-5.4 where the depth justifies it, default-tier traffic to GPT-5.5 Instant.
The fit profile: Category 3 (general reasoning, knowledge work, drafting) is the natural home, and GPT-5.5 Instant in particular handles the high-volume routine traffic that drives most of an enterprise's monthly token volume. GPT-5.4 retains a frontier-reasoning role for the harder analytical workloads where capability ceiling matters. The OpenAI tier is also the natural fallback target for traffic the primary routing rules cannot confidently classify — broad domain coverage and consistent tool-use behavior make it the safest "default if unsure" target.
The Routing Layer — Build vs Buy
The routing rules require an enforcement layer. Three serious options exist; the choice is a build-vs-buy decision sensitive to scale and engineering capacity.
LiteLLM (open source). A lightweight proxy that exposes a single OpenAI-compatible interface and routes to any backing model with provider-specific configuration. Self-hosted, no vendor in the data path, full control over routing logic. The cost is engineering effort to deploy, monitor, and maintain — non-trivial at production scale.
Portkey (managed). A managed gateway with routing, caching, observability, and governance baked in. Trades self-host control for substantially lower operational burden. Suitable for teams without dedicated platform engineering capacity.
OpenRouter (consumer-leaning, expanding to enterprise). Aggregates a wide model catalog under a single API and billing surface. Lower friction for experimentation and small-team production; enterprise features have matured through 2025 but still trail Portkey on governance.
The build-vs-buy break-point sits roughly where monthly LLM spend crosses $50K. Below that, managed gateways amortize quickly against the engineering effort; above it, the per-token pricing of managed gateways starts to matter and the self-host case strengthens. Either way, the routing layer is now production infrastructure on the same level as observability or feature flags.
Token Caching And Prompt Compression — The Other 20% Cost Cut
Beyond cross-vendor routing, two within-call optimizations carry the next layer of cost reduction. Token caching — reusing computed key-value cache state across requests that share a prefix — has matured into a first-class API surface across Anthropic, OpenAI, and Grok in 2026. For workloads with stable system prompts and large reusable context blocks, caching cuts effective input cost on cached tokens by 75-90% depending on provider. The optimization is essentially free engineering effort once the prompt structure is stable; the cost is restructuring the prompt so the cacheable prefix is fixed and the variable suffix is small.
Prompt compression — algorithmically reducing prompt length while preserving task-relevant information — produces a more modest but additive saving. Compression is workload-specific and most effective for retrieval-augmented workloads where retrieved context is high-volume and partially redundant. Production-grade compression libraries deliver 30-50% prompt-length reduction with negligible quality impact on most workloads when tuned carefully.
Batch APIs are the third lever. Anthropic, OpenAI, and now several open-weight inference platforms expose batch-tier pricing at roughly 50% the synchronous rate, with completion guaranteed within a 24-hour window. For asynchronous workloads — overnight content generation, scheduled research, periodic analysis — batch routing is straight cost reduction with no quality trade-off.
What This Desk Tracks Through Q2-Q3 2026
Three datapoints anchor ongoing tracking. First, the next-tier price moves from any of the three frontier vendors. The Grok 4.3 April cut and Sonnet 4.6 pricing settled the May floor; the next iteration of the cost war is overdue and will reshape routing rules. Second, the maturation of routing-platform observability — whether LiteLLM, Portkey, OpenRouter, and adjacent vendors close the gap on cross-model performance attribution that production deployments need to refine routing rules over time. Third, capability convergence on Category 3 (general reasoning, knowledge work). If GPT-5.5 Instant capability holds while Grok and Anthropic budget tiers close the gap, the routing rules for the highest-volume traffic class get rewritten by Q3.
Honest Limits
The 40% cost-reduction figure is this desk's modeled estimate based on representative workload mixes at the $10K-$50K monthly spend range. Actual savings depend on workload distribution, baseline vendor, routing rules, and whether caching and batch tiers are already in use. Workloads concentrated in a single category may see less benefit; workloads with significant agentic or long-context traffic may see more. The five-category taxonomy is a simplification; production workloads often cross categories within a single agent run. Pricing figures are accurate to late April / early May 2026 announcements and will age. Capability comparisons rely on benchmark cuts with known methodological limits — pilot testing against representative samples remains the only reliable way to validate routing decisions.