The Efficiency Counterthesis: Can Technology Outrun the Unwind?

Counterpoint-to: hormuz-to-ai-repricing-causal-chain — the pessimist case (macro-down: geopolitics, energy, finance, supply chains) Builds-on: ai-token-economics-and-open-source-competition Related: execution-plan-phase-0-1-2 Informs: Projects/edge-llm, Projects/sigil

A companion to The Full Stack — not a denial of the macro/geopolitical thesis, but an honest assessment of whether the compounding rate of efficiency gains in AI hardware, software, and architecture can outpace the economic and physical constraints closing in. Both docs arrive at the same structural conclusion from opposite directions: the middle disappears. Read them together. The question isn't whether the subsidy unwind is real. It's whether it matters if cost-per-useful-token is falling faster than prices need to rise. (Answer: the gains get captured as provider margin, so the user-facing economics barely change.)

TLDR

The pessimist case (in "The Full Stack") says: Hormuz → energy shock → supply chain crisis → subsidy unwind → token prices rise 60-160% → mid-market gets priced out.

The optimist case says: efficiency gains are compounding multiplicatively across hardware, software, architecture, and model design. The combined improvement rate may be 10-100x per generation (every 18-24 months). A 60-160% price increase against a 10-100x efficiency gain is a rounding error.

Both can be true simultaneously. The question is timing: do the gains arrive before the structures tear things down? And do they arrive in forms accessible to the people who need them — not just hyperscalers?

Part 1: The Compounding Efficiency Stack

The critical insight most analysis misses: AI efficiency improvements don't add — they multiply. Each layer of optimization compounds with every other layer.

The Four Multipliers

Combined: 24-180x improvement per generation when all four compound.

This isn't theoretical. The historical data confirms it: running a GPT-4-class model cost ~$20/M tokens in late 2022. In early 2026, equivalent performance costs $0.40/M tokens. That's a 50x reduction in 3 years — roughly a 1,000x reduction if you include the capability improvements at the same price point.

The question for the next 12-18 months: does this rate of compounding continue, accelerate, or hit a wall?

Part 2: Hardware — The Generational Leaps

Vera Rubin: The Platform Shift

Nvidia's Vera Rubin platform (announced CES January 2026, volume production H2 2026) isn't an incremental GPU upgrade. It's a full six-component stack designed to work as a unified system:

• Vera CPU + Rubin GPU + NVLink 6 + ConnectX-9 SuperNIC + Bluefield-4 DPU + Spectrum-X800 Ethernet
• 50 PFLOPS inference performance (NVFP4) — 5x Blackwell
• 10x lower cost per token than Blackwell
• 10x performance per watt improvement
• 288GB HBM4 per GPU, 22 TB/s memory bandwidth
• Optimized for 10 million token context windows
• 1/4 the GPUs needed to train MoE models vs. Blackwell

Rubin Ultra (2027): ~500 billion transistors, 384GB HBM4E, 32 TB/s bandwidth. Scales to NVL576 racks (576 GPUs) delivering 15 exaflops of FP4 compute.

What this means: If Blackwell delivered ~10x over Hopper, and Rubin delivers 5-10x over Blackwell, we're looking at 50-100x improvement in cost-per-token at the hardware layer alone in roughly two years (2024-2026). That's before any software, architecture, or quantization improvements.

The timing question: Rubin samples Q4 2026, volume production Q1 2027. This means the hardware efficiency gains arrive at almost exactly the moment the subsidy unwind hits. The full impact on token pricing won't be felt until H2 2027 — which is the same window where the overbuild-or-bust question resolves. These timelines are converging.

Beyond Nvidia: The Inference Chip Explosion

Nvidia isn't alone. The inference-optimized chip market is expected to hit $50B+ in 2026:

• Google TPUs — custom silicon, deeply integrated with Gemini
• Amazon Trainium/Inferentia — Anthropic's primary training platform
• AMD MI series — competing on price/performance, especially for inference
• Groq — LPU architecture optimized purely for inference speed
• Cerebras — wafer-scale chips for massive model training
• Custom ASICs from Meta, Microsoft, Apple — vertically integrated inference

Competition at the hardware layer is intense and accelerating. This matters because it means cost-per-token improvement isn't dependent on a single vendor's roadmap. If Nvidia stumbles, alternatives exist. If Nvidia delivers, competitors are forced to match.

Photonic Computing: The Wildcard (3-5 Year Horizon)

Not production-ready for LLM inference yet, but the physics are real:

• MIT: photonic processor performing matrix operations at the speed of light
• Tsinghua University: LightGen chip — millions of photonic neurons, 100x+ speed and energy efficiency over electronic chips for image generation
• OLIX Computing (UK): $220M raise, valued at $1B+, building inference-optimized photonic chips
• Nature publication: 16,000-component integrated photonic accelerator with ultralow latency

Photonic computing solves the energy constraint fundamentally — light doesn't generate heat the way electrons do. If this matures on a 3-5 year timeline, the energy/cooling/water bottlenecks identified in "The Full Stack" become solvable problems rather than permanent constraints.

Part 3: Software — The Optimization Avalanche

TurboQuant (Available Now)

Google's TurboQuant (March 2026): 6x KV cache memory reduction, 8x inference speedup on H100s, zero accuracy loss. Training-free, works on existing models. This alone allows:

• More concurrent users on the same hardware
• Longer context windows without additional memory
• Lower infrastructure cost for the same throughput

Speculative Decoding (Production Standard)

Originally 2-3x speedup, now built into vLLM, SGLang, TensorRT-LLM, and most serving frameworks. Draft models generate 3-12 candidate tokens per step, verified in a single parallel forward pass. 70-90% acceptance rate on domain-specific tasks.

Combined with other optimizations: FP8 quantization + Flash Attention 3 + continuous batching + speculative decoding on H100 delivers 5-8x better cost-efficiency than naive FP16 inference.

Agent Cost Optimization (60-80% Reduction)

The compound savings from prompt compression + model routing + semantic caching can deliver 60-80% total cost reduction without meaningful quality degradation. Specifically:

• Model routing: Use cheap models for easy tasks, expensive models for hard tasks. A 3B model handles 70% of requests; Opus handles the 30% that need it.
• Semantic caching: Reduces API call volume 30-50% for typical enterprise deployments by reusing responses to semantically similar queries.
• Prompt compression: Shorter prompts = fewer input tokens = lower cost. Modern compression loses <2% quality.

Real-world validation: Grupo Bimbo claims "tens of millions" saved after deploying thousands of low-code agents. Dow expects annual multimillion savings from invoice scanning agents.

Part 4: Model Architecture — The Densing Law

Small Models Getting Dramatically Better

A Nature Machine Intelligence paper identified the "densing law": capability density doubles approximately every 3.5 months. Equivalent model performance can be achieved with exponentially fewer parameters over time.

Evidence:

• Ministral 3 (14B): Matches Mistral Small 3.1 while being 40%+ smaller, trained on shorter horizon. Cascade distillation progressively transfers knowledge from large to small.
• SmolLM3 (3B): Outperforms Llama-3.2-3B and Qwen2.5-3B on 12 benchmarks. Competitive with 4B-class alternatives.
• Distilled 3B reasoning model: Outperforms current 11B and 6B models on GSM8K using multi-step reasoning distillation.
• Llama 3.2 1B/3B: State-of-the-art for their class, designed for on-device inference.
• Data distillation: 10x smaller models, 10x faster inference, approaching parity with teachers on targeted tasks.

What this means for the subsidy unwind: If a 3B model can do in 2027 what a 70B model did in 2025, the compute requirement per useful task drops 20x+ from architecture alone. This stacks on top of hardware and software gains.

Mixture of Experts (MoE): Only Use What You Need

MoE architectures (DeepSeek V3, Llama 4 Scout/Maverick) only activate a fraction of total parameters per token:

• Llama 4 Scout: 109B total, 17B active — enterprise-grade results at a fraction of the compute
• DeepSeek V3: 671B total, 37B active — matches frontier models on many benchmarks
• Vera Rubin specifically optimized for MoE: 1/4 the GPUs to train, 10x lower cost per token for inference

MoE is the architecture that makes the hardware gains real for users. Without MoE, Rubin's 10x improvement mostly benefits hyperscalers training massive dense models. With MoE, the gains flow directly to inference cost — which is what end users pay for.

Part 5: Edge and Browser — The Zero-Infrastructure Frontier

On-Device AI (Already Here)

Every flagship chip in 2026 includes a neural engine:

• Apple A18 Pro: 16-core Neural Engine, 35 TOPS
• Qualcomm Snapdragon 8 Elite: 75 TOPS across Hexagon NPU + Adreno GPU + Kryo CPU
• MediaTek, Samsung, Intel, AMD: All shipping NPU-equipped silicon

Apple Intelligence already runs a ~3B parameter model on-device for summarization, rewriting, and Smart Reply. ExecuTorch (Meta) hit 1.0 GA with a 50KB footprint running on everything from microcontrollers to phones with 12+ hardware backends.

What this means: The floor for "AI that runs without any server" is rising every product cycle. A ~3B model on a phone NPU in 2026 delivers utility that required a $20/month subscription to a cloud API in 2024. The subsidy unwind doesn't affect a model running on hardware you already own.

Browser Inference (Edge-LLM Territory)

WebGPU shipped across all major browsers (Chrome, Firefox, Edge, Safari) as of November 2025 — 82.7% global coverage.

Current performance:

• WebLLM: Llama 3.1 8B (4-bit quantized) at 41 tok/s on M3 Max. Phi 3.5 mini at 71 tok/s. Achieves 80% of native speed in a Chrome tab.
• Transformers.js v3: WebGPU support hits up to 100x faster than WASM for suitable workloads.
• LFM2-MoE (8.3B): Running entirely in-browser via WebGPU. A Mixture-of-Experts language model running in a tab.
• INT4 quantization: Reduces memory footprint 75%, enabling Llama-3.1-8B on average consumer hardware.

The trajectory: An 8B MoE model running at 40+ tok/s in a browser in 2026. By 2027, with the densing law and further WebGPU optimization, a browser-native model could approach GPT-4-class performance for common tasks. Zero API cost. Zero server. Zero subscription. Zero liability chain.

Part 6: The Compounding Math — Does It Outrun the Unwind?

The Pessimist Timeline (from "The Full Stack")

A developer paying $3/M tokens for Sonnet-class today would pay $5-8/M tokens post-subsidy.

The Optimist Timeline

The Net Effect

If post-subsidy prices rise 60-160% but efficiency gains deliver 10-50x improvement over the same period, the effective cost per useful task still falls dramatically.

Example at the Sonnet tier:

• Today: $3/M tokens, ~50K tokens per complex task = $0.15/task
• Post-subsidy (no efficiency gains): $7/M tokens, ~50K tokens = $0.35/task ← the pessimist case
• Post-subsidy WITH efficiency gains: $7/M tokens, but task requires 5-10K tokens (better models, compressed prompts, routing) = $0.035-0.07/task ← actually cheaper than today

The efficiency gains don't just offset the price increase — they potentially overwhelm it. But only for users sophisticated enough to adopt the optimizations (model routing, caching, compression, smaller models for simpler tasks).

The Access Gap Persists

Here's where the optimist case gets honest: these gains are not evenly distributed.

Who benefits:

• Hyperscalers (Rubin, custom ASICs, massive optimization engineering)
• Sophisticated enterprise teams (model routing, caching, multi-tier inference)
• Developers who can self-host and optimize (open-source MoE + quantization + edge deployment)
• Browser-native applications (edge-llm territory — zero marginal cost once built)

Who doesn't benefit:

• SMBs paying API prices (they get the price increase without the optimization)
• Non-technical users on consumer subscriptions (they get whatever the provider optimizes for them)
• Anyone who can't invest in optimization engineering

The efficiency counterthesis doesn't eliminate the stratification described in "The Full Stack." It accelerates it. The gap between "optimized" and "naive" usage grows wider, not narrower. A sophisticated user pays $0.04/task. A naive user pays $0.35/task. Same model, same capability, 9x cost difference.

Part 7: What Could Go Right — The Optimist Scenarios

Scenario O1: Efficiency Outpaces Unwind (40% probability)

Vera Rubin delivers on schedule (H2 2026). TurboQuant + speculative decoding + agent optimization compound to deliver 5-10x immediate improvement. The densing law holds — 3B models in 2027 match today's 30B performance. Open-source ecosystem thrives on efficient small models.

Result: Token prices rise nominally (30-50%) but effective cost per task falls. The subsidy unwind is real but invisible to well-optimized users. Enterprise AI adoption accelerates because ROI improves. Consumer access degrades on proprietary platforms but improves via browser/edge. The "ambient AI everywhere" prediction materializes for technical users by late 2027.

Scenario O2: Hardware Arrives Late, Software Saves It (30% probability)

Vera Rubin hits supply constraints (helium, HBM4, CoWoS — per "The Full Stack"). Hardware gains delayed 6-12 months. But software optimization (already available) plus model architecture improvements (densing law) deliver 3-5x improvement independent of new hardware.

Result: The 2026-2027 period is painful — prices rise, subsidies unwind, mid-market squeezed. But by 2028, the combination of delayed hardware + continuous software gains delivers the same outcome as O1, just slower. The bust scenario from "The Full Stack" is possible but the floor is higher than the pessimist case assumes because the software efficiency stack is hardware-independent.

Scenario O3: Photonic + Edge Convergence (15% probability, 3-5 year)

Photonic computing matures faster than expected. Edge NPUs reach 100+ TOPS. Browser inference runs 30B+ effective-parameter models. The entire inference layer moves off the cloud and onto devices and edge networks.

Result: The hyperscaler infrastructure becomes less relevant for inference (still needed for training). Token pricing becomes irrelevant for most consumer and SMB use cases because they're running locally. The "captive capability" scenario from "The Full Stack" is averted because the capability escapes into hardware people already own.

Scenario O4: Oversupply Becomes a Gift (15% probability)

The bust scenario from "The Full Stack" plays out — hyperscaler capex gets cut, GPU oversupply, memory normalizes. But from the optimist lens: cheap GPUs + cheap memory + efficient models + mature optimization stack = the most accessible AI environment ever. The bust is the best thing that could happen for democratized AI.

Result: Token prices crash (not just normalize — crash) because oversupply meets efficiency gains. Self-hosting becomes dirt cheap. Open-source models running on surplus hardware deliver capabilities that were enterprise-exclusive 18 months earlier. The "internet after the fiber crash" parallel — the infrastructure boom enables the usage boom, just with different economic winners.

Part 8: What This Means for Building

The Window

The efficiency gains don't eliminate the macro/geopolitical risks. They create a race condition. If you're building something that benefits from cheap AI inference:

• Next 6-12 months: Build on today's efficient tools (quantized open-source, speculative decoding, agent optimization). Don't depend on Rubin arriving on time.
• 12-24 months: Rubin + densing law + continued optimization delivers 10-50x. If you've already built the architecture that can route between tiers, you capture these gains automatically.
• 24-36 months: Either the soft landing or the bust has resolved. In both cases, AI is cheaper and more accessible than today. In the bust case, dramatically so.

The Strategic Bet

The pessimist case says: don't build in the middle, everything dies. The optimist case says: build at the efficiency frontier, because the frontier is moving faster than the headwinds.

Specifically:

Edge-LLM: WebGPU at 82.7% coverage. 8B MoE models in browser at 40+ tok/s. Densing law means 3B browser models in 2027 match today's 8B. This project is BETTER positioned by the efficiency counterthesis than by the pessimist case — every efficiency gain that makes models smaller and faster is a direct tailwind for browser-native inference.

Sigil (human-in-the-loop): The efficiency gains make AI more capable, which means more AI output flowing into production, which means MORE need for human review checkpoints. Faster, cheaper AI doesn't reduce the need for governance — it increases it because the volume of AI-generated work scales with cost reduction.

The friend's education idea: The optimist reframe — don't teach people to use ChatGPT (a product that's changing). Teach people to optimize. Model routing, prompt compression, semantic caching, edge deployment. The 9x cost gap between naive and optimized usage is the educational opportunity. The curriculum is "how to get 10x more from 50% less spend." That's valuable in every scenario.

Part 9: The Reality Check — Efficiency Gains Aren't For You

This section synthesizes the pessimist case (hormuz-to-ai-repricing-causal-chain) and the optimist case (above) into what probably actually happens.

The Pattern That Always Holds

Efficiency gains in technology don't flow to consumers. They flow to margins. This has been true for every prior cycle:

• CPUs got 1000x faster → software got 1000x more bloated → user experience stayed roughly the same
• Internet bandwidth went 100x → websites went from 50KB to 5MB → page load times barely changed
• Cloud computing dropped server costs 10x → SaaS companies captured the margin → customers pay $20/month forever
• Smartphone processors got 10x faster → apps got 10x heavier → battery life stayed at one day

The pattern: efficiency gains get absorbed by the provider layer, not passed to the end user. The provider uses the headroom to either increase margins or increase capability (which justifies maintaining the same price).

Applied to AI

Vera Rubin delivers 10x cost per token reduction. Anthropic doesn't cut prices 10x. They use the headroom to:

1. Improve margins — gross margin trajectory from 40% to 77% by 2028. That's the efficiency gains becoming profit. That's how they reach breakeven in 2027-2028 without raising prices.
2. Add capability at the same price — longer context, better reasoning, agentic features, Claude Code, Cowork plugins. You get more for $3/M tokens, not the same for less.
3. Maintain competitive pricing against OpenAI without losing money. The prisoner's dilemma from The Full Stack, Part 3 resolves not through price changes but through margin improvement. Neither lab raises prices. Neither lab lowers them. Both become profitable through efficiency, not pricing power.

The user still pays $3/M tokens for Sonnet-class. They just get a better Sonnet. The $20/month consumer subscription persists. What you get for $20 keeps improving. But the economic structure — who pays what — doesn't fundamentally change.

Routing Becomes a Commodity Feature

The "teach people to optimize with model routing" play has a shelf life of about 12 months.

Claude Code already routes between Haiku and Sonnet/Opus internally. Cursor does this. Every serious AI-integrated tool is building routing. The libraries already exist: LiteLLM, Martian, Unify, RouteLLM. Within 12 months, smart routing will be a checkbox feature, not a competitive advantage or a curriculum topic.

The optimization techniques that represent a 9x cost advantage today become built-in features tomorrow. That's how platforms work — they absorb the optimization layer into the product, eliminating the need for external expertise. The platform always eats the tooling layer eventually.

What Actually Changes (The Synthesis)

If efficiency gains become profit and routing becomes commodity, the real picture is:

The subsidy unwind and the efficiency gains roughly cancel each other out at the price level. The status quo price point persists, but the underlying economics restructure entirely underneath. This is the boring middle path — and it's the most likely outcome.

The Middle Still Disappears — From Both Sides

This is the core finding that both docs agree on from opposite directions (see The Full Stack for the pessimist path to the same conclusion):

From the pessimist side: Prices rise → middle can't afford it → squeezed out.

From the optimist side: Efficiency rises → middle can't capture it → squeezed out.

From the reality-check side: Prices stay flat, but the capability ceiling keeps rising and the minimum viable AI integration keeps getting more sophisticated. Keeping up requires continuous technical investment that the middle can't sustain. The platforms absorb the optimization layer, so you can't differentiate on tooling. The enterprise tier gets better deals and dedicated support. The bottom tier goes to zero cost via edge/browser. The middle pays list price for a product that's always slightly behind what the top tier gets.

The middle doesn't disappear because AI fails. It disappears because AI succeeds unevenly. The technology is available to everyone. The ability to extract maximum value from it is not. And the platform providers are incentivized to capture efficiency gains as profit rather than pass them through as savings — because that's how they reach profitability after years of subsidized losses.

What Actually Survives (Revised Assessment)

Given that prices likely stay flat, routing gets commoditized, and efficiency gains become provider profit:

Sigil — STILL VALID, possibly stronger. Human-in-the-loop governance is NOT something labs will build in and give away. It's orthogonal to their interests — they want MORE automated usage, not more human checkpoints. Every efficiency gain that increases AI throughput increases the volume of AI output flowing into production, which increases the need for governance. Liability pressure (see The Full Stack, Part 7) creates demand the labs won't serve. The platforms eat the tooling layer but they don't eat the accountability layer.

Edge-LLM — STILL VALID, reframed. Not "escape the price increase" (prices probably stay flat) but "escape the dependency." Privacy, offline access, zero-latency, no TOS changes, no account required, no data flowing to a third party. The value proposition shifts from economic to sovereignty. Every densing law improvement (3.5-month doubling of capability density) is a direct tailwind — smaller, better models run in browsers more naturally every cycle.

The friend's education idea — WEAKER than previously assessed. If prices stay flat, there's no urgency for cost optimization education. If routing is built into platforms, there's no optimization to teach. The wrapper kill zone is still lethal because platforms keep absorbing wrapper-layer features. The education play would need to be something the platforms CAN'T absorb — domain-specific workflow design, regulatory compliance, organizational change management. That's management consulting, not AI education.

The friend's debate/comparison system — STILL DEAD. Nothing in the reality check changes this assessment. It's a wrapper in every scenario.

The One-Liner (Revised)

"The Full Stack" says the war didn't create the problem, it gave the problem a deadline. "The Efficiency Counterthesis" says the gains are in a footrace with the deadline. The reality check says: the gains and the unwind cancel each other out, the providers capture the difference, and the middle still disappears. The war, the efficiency, the subsidies — they're all noise around a structural truth: the technology succeeds unevenly, and the gap is permanent.

How These Docs Connect

These two research docs are a single analysis viewed from two altitudes:

Both arrive at the same structural outcome from opposite directions. The middle disappears not from one force but from two forces moving in the same direction — the ceiling falling (price pressure) and the floor dropping away (optimization gap the middle can't reach).

The things that survive in BOTH analyses: deep enterprise integration (Anthropic's moat), human-in-the-loop governance (Sigil's positioning), zero-infrastructure edge inference (edge-llm's territory), and the platform providers themselves (who capture efficiency as margin).

The things that die in BOTH analyses: thin wrappers, naive API usage, mid-market services without a compliance or governance moat, and any business plan that depends on token prices staying artificially low.

Sources

Vera Rubin Platform

• Tom's Hardware: Vera Rubin NVL72 — 5x inference, 10x lower cost per token
• CNBC: First look at Vera Rubin and how it beats Blackwell
• Nvidia: Rubin platform official page
• Nvidia Newsroom: Six new chips, one AI supercomputer
• Next Platform: Vera Rubin obsoletes current AI iron

Hardware Efficiency and Inference

• GPUnex: AI inference economics — 1,000x cost collapse
• Nvidia Blog: Leading providers cut costs 10x with open-source on Blackwell
• Deloitte: Why AI's next phase demands more power, not less
• VAST Data: 2026 — The year of AI inference

Photonic Computing

• MIT: Photonic processor for ultrafast AI computation
• Science: LightGen — all-optical chip for vision generation
• Nature: Integrated large-scale photonic accelerator
• SiliconAngle: OLIX photonic AI chip startup raises $220M

Software Optimization

• Google Research: TurboQuant — redefining AI efficiency
• Prem AI: Speculative decoding — 2-3x faster inference
• Moltbook: AI agent cost optimization — reduce spend 60-80%
• Zylos Research: Token economics and FinOps in production

Model Architecture and Distillation

• Nature Machine Intelligence: Densing law of LLMs
• Mistral: Ministral 3 — cascade distillation
• BentoML: Best open-source small language models 2026
• Prem AI: Data distillation — 10x smaller, 10x faster

Edge and Browser Inference

• Vikas Chandra (Meta): On-device LLMs state of the union 2026
• Calmops: Edge AI and on-device AI 2026 complete guide
• WebLLM: High-performance in-browser LLM inference engine
• Mozilla AI: 3W for in-browser AI — WebLLM + WASM + WebWorkers
• SitePoint: WebGPU vs WebASM browser inference benchmarks
• WebGPU Community: LFM2-MoE 8.3B running in browser