The AI Cost Paradox: Cheaper Models, Bigger AI Bills

Five costs that should not be confused

Most arguments about AI economics mix numbers from different layers. Separating them prevents a cheap API request from being compared with a billion-dollar data-centre project.

1. Price per token. The developer's published API charge for model input and output.
2. Provider inference cost. The chips, memory, electricity, cooling, networking, spare capacity and serving software used to generate a response. Providers rarely disclose this figure in full.
3. Cost per completed task. Every model call, search, tool, retry, verification step and human check needed to finish useful work.
4. Training cost. The compute, data, researchers, evaluation and experimentation required to create or materially improve a model.
5. Infrastructure cost. Land, buildings, accelerators, power connections, cooling and backup systems that may support many models over several years.

The third figure is the one most business cases miss. Customers do not buy tokens for their own sake. They buy an answer or an action they can trust.

What Indian businesses should measure first

The cheapest token price is rarely enough to choose an architecture. Salary structure, error risk, customer expectations, language, exchange rates, tax and integration effort all influence the completed-task cost. Start with the operating decision, then inspect the model bill.

1. Price the successful outcome

Track cost per support case resolved, invoice processed, accurate product description, appointment booked or qualified sales lead. Include tool charges, retries, human correction and the cost of errors. Do not count an answer as successful merely because a model returned text.

Productivity evidence belongs beside that outcome. Stanford's 2026 AI Index cites gains around 14% to 15% in customer support and 26% in software development, with weaker results where work was hard to monitor or errors needed heavy review. [21] Treat those figures as context, then measure the task in your own queue.

2. Route work instead of using one model everywhere

A small model can classify and draft, a mid-tier model can handle normal work, and a frontier model can be reserved for difficult cases. High-risk decisions still need a person who is accountable for the result.

A fictional 800-conversation example

Illustrative only—not a GreyBath client result. Imagine an operations team receives 800 support conversations a month. A small model classifies every message, a mid-tier model drafts routine replies, and 8% of cases go to a frontier model and a named human reviewer. Using the 18 August price snapshot later in this article, assume 700 input and 80 output tokens for classification; 1,800 input and 350 output tokens for 736 routine drafts; and 3,000 input and 700 output tokens for 64 complex cases. The model-only total is about US$7.72.

At an illustrative exchange rate of ₹84 per US dollar, that is about ₹648. If the 64 escalated cases each need eight minutes of review at an illustrative loaded rate of ₹450 an hour, review adds ₹3,840. If 760 conversations are verified as resolved, the combined model-and-review figure is about ₹5.91 per verified resolution before tool APIs, cloud services, retries, GST and payment charges. Replace every token count, rate, percentage and outcome with observed pilot data. Compare the result with your actual staff time and error cost; this example does not prove a saving.

3. Put boundaries around agents

Set maximum steps, tokens, retries, searches, tool calls, time and spend per task. Retrieve only the documents needed for the current job; sending the whole history or document library on every run wastes money and can increase privacy risk.

4. Test Indian operating conditions

Hindi, Marathi, Gujarati, Tamil, Bengali and other languages can tokenise differently across models. Test real customer conversations rather than extrapolating from English. Add USD-INR movement, GST treatment, cloud charges and payment costs to the budget.

5. Keep the workflow portable

Separate business logic from the provider, cache repeated instructions, batch work that is not urgent and keep interfaces replaceable. Compare the automated workflow with the full human process: turnaround time, missed follow-ups, service quality and capacity matter alongside salary.

Why fixed-capability inference keeps getting cheaper

Stanford's 2025 AI Index estimated that the cost of reaching GPT-3.5-level performance on the MMLU benchmark fell from about $20 per million tokens in November 2022 to $0.07 by October 2024. That is a reduction of more than 280 times for a comparable capability level. [6]

Epoch AI offers a deliberately rough planning estimate: fixed-capability inference has recently become roughly five to ten times cheaper per year, with considerable uncertainty and variation by task. [7]

No single breakthrough produced that curve. New accelerators process more tokens per watt; smaller models now handle work that once required a frontier model; quantisation reduces memory; distillation transfers useful behaviour; and batching, caching, routing and better serving software keep expensive hardware busier.

The August 2026 price ladder

The dated list prices still show how widely model tiers can differ. They are retained as an editorial snapshot so that the worked arithmetic remains reproducible; they are not a current buying guide.

If you cannot name the task, the model route, the cap and the rupee cost of a failure, you are buying tokens rather than managing an outcome.

OpenAI, Anthropic and Google apply different tokenisers, reasoning settings, context limits and cache charges. [1] [2] [3] A lower-priced model that needs three attempts may cost more than a stronger model that succeeds once.

A cheap query does not reveal provider profitability

Public information is not sufficient to claim that every paid API request loses money. A well-utilised service can earn a positive margin on an individual request while the company remains unprofitable after free usage, research, training, unused capacity, depreciation, safety work, support and financing.

List prices also reflect competition. OpenAI reduced Luna and Terra prices on 30 July 2026, Anthropic made Sonnet 5's introductory price permanent, and Google publishes dated price changes for Gemini. [1] [2] [3] Those decisions are commercial strategy as well as engineering arithmetic.

Why agents can cost more even when tokens get cheaper

A chatbot normally receives one request and returns one answer. An agent may plan, search, read company documents, call an API, inspect the response, retry a failed step and ask a stronger model to verify the final result. Some of that activity is never visible to the user.

Gartner forecasts provider inference cost per agentic workflow will increase more than fivefold through 2028. It calls this the inference paradox: better unit economics can still raise total operating cost when workflows use more steps, tools and verification. Goldman Sachs describes scenarios in which one user request expands into 10, 20 or even 50 model operations and forecasts a 24-fold rise in total token consumption from 2026 to 2030. [8] [9]

A calculation using the listed OpenAI prices

Consider a simple chat using 2,000 input and 1,000 output tokens, then an agent using 100,000 total input and 20,000 output tokens across its full run.

Search APIs, databases, orchestration, monitoring and human review sit outside those model totals. The finished Sol workflow costs about 27.5 times the simple Sol request even though both use the same price card.

Cheaper supply also creates more demand. Teams generate more drafts, add AI to more products, process larger documents and automate work that was previously uneconomic. Economists recognise this as the Jevons effect: efficiency can increase total consumption rather than reduce it. Make the practical response explicit—cap monthly volume as well as cost per call, require approval when a pilot expands to a new workflow, and review whether extra drafts or retries improve the verified outcome.

Training, data centres and power move on a different curve

Serving yesterday's capability is getting cheaper while building the frontier remains capital intensive. Stanford estimated GPT-4 training compute near $78 million and Gemini Ultra near $191 million. A 2024 Epoch AI extrapolation suggested the largest training runs could exceed $1 billion by 2027 if historical scaling continued; that is a scenario, not a disclosed invoice. [10] [11]

Outside estimates remain uncertain because leading models disclose less about training code, dataset size and parameter counts. The physical stack also serves more than one run: Epoch AI's estimate for a typical one-gigawatt AI data centre is roughly $38 billion upfront and $0.9 billion a year to operate, but such a facility may train, test and serve many models for years. [12] [14]

The IEA's 2026 outlook puts global data-centre electricity use near 485 terawatt-hours in 2025 and about 950 terawatt-hours in 2030, with AI-focused facilities potentially tripling their demand. It also estimates that roughly 20% of planned projects could be delayed without action on grid constraints. [22] [17]

What one query uses

A 2026 Microsoft Research study estimated a median 0.31 watt-hours per optimised frontier-scale query under modelled production conditions; its long-reasoning scenario, with roughly 5,000 output tokens, used about thirteen times as much. [18] A normal prompt is small; billions of prompts and persistent agents are not. Workload shape matters more than a single viral estimate.

Bubble risk or structural boom?

AI is a stack of businesses rather than one investment. Chips, data centres, model providers, API platforms and workflow applications have different margins and failure modes.

Nvidia reported $75.2 billion of data-centre revenue in the first quarter of its 2027 financial year, up 92% year over year, with gross margin near 75%. [19] Gartner expects inference infrastructure spending of $23.3 billion in 2026 to exceed training infrastructure spending of $19 billion. [5] Production demand and supplier revenue are measurable.

Goldman Sachs' scenario-based baseline—not a demand forecast—uses about $765 billion of annual AI capital expenditure in 2026, rising towards $1.6 trillion in 2031. [4] Returns will not be evenly distributed. Capacity can arrive before customers, grid delays can leave hardware idle, and subsidised model or application pricing may not survive competition.

A workflow should still be judged by the verified value it creates. A low-cost run that produces unreliable work is not a sound business case.

AI is a structural technology boom with speculative pockets, not a single yes-or-no bubble.

What pricing may look like by 2030

Forecasts should be treated as planning ranges. Four directions have stronger support than a detailed prediction for every market segment.

1. Fixed-capability AI becomes cheaper. Gartner expects provider inference cost for a one-trillion-parameter model to be more than 90% lower in 2030 than in 2025. [20] Providers may retain part of that saving for reliability, security and margin.
2. Frontier intelligence retains a premium. The industry will keep creating a new top tier with longer context, more reasoning, better tools and reserved capacity, even as older capability moves down the price curve.
3. Agent workflows remain expensive in the near term. Longer jobs, premium decisions, tools and verification can outweigh reductions in token price.
4. Business pricing shifts towards outcomes. Tokens remain useful for infrastructure accounting, while customers increasingly pay for cases resolved, documents processed, leads qualified, time saved or service guarantees.

Some prices will move against the broad trend. Google, for example, published a scheduled increase for Gemini 3.7 Flash from January 2027. [3] Budgets need room for provider changes as well as efficiency gains.

Appendix: cost three real tasks in 30 minutes

This is a first-pass operating check, not a procurement model. Use three recurring tasks from your own queue and keep the evidence beside the calculation.

1. Minutes 0–5: define success. Name the completed outcome, the monthly volume and the failure that would make the result unusable.
2. Minutes 5–12: set the route and limits. Record the model route, maximum steps, retries, tool calls and the person who owns an error.
3. Minutes 12–20: sample actual runs. Capture input and output tokens, tool charges, retries and human review minutes. Do not estimate these from a single demo.
4. Minutes 20–26: calculate cost per success. Add model, tool, cloud, review and correction cost; convert foreign-currency charges using an agreed finance rate; divide by verified successful outcomes.
5. Minutes 26–30: make a controlled decision. Set a cost cap, name the owner and decide whether to stop, extend the pilot or test a different route.