First Section Title - intro
2. Section with Data - needs a
| Provider / Model | Input ($/1M tokens) | Output ($/1M tokens) | Context Window | Best Use Case |
|---|---|---|---|---|
| OpenAI GPT-4o | $2.50 | $10.00 | 128K | Complex reasoning, multimodal |
| OpenAI GPT-4o mini | $0.15 | $0.60 | 128K | Classification, extraction, short replies |
| OpenAI o1 | $15.00 | $60.00 | 200K | Math, code, multi-step planning |
| OpenAI o1-mini | $3.00 | $12.00 | 128K | Cheaper reasoning, coding helpers |
| Anthropic Claude 3.5 Sonnet | $3.00 | $15.00 | 200K | Long context, code, agentic loops |
| Anthropic Claude 3.5 Haiku | $0.80 | $4.00 | 200K | Fast chat, summarization |
| Anthropic Claude 3 Opus | $15.00 | $75.00 | 200K | Legacy premium tasks |
| Google Gemini 1.5 Pro | $1.25 | $5.00 | 2M | Huge context, document QA |
| Google Gemini 1.5 Flash | $0.075 | $0.30 | 1M | High-volume, low-latency |
| Mistral Large 2 | $2.00 | $6.00 | 128K | European compliance, multilingual |
| Mistral Small | $0.20 | $0.60 | 32K | Budget routing, simple instructions |
| Meta Llama 3.1 405B (hosted) | $3.50 | $3.50 | 128K | Open-weights parity, fine-tunes |
| DeepSeek V3 | $0.27 | $1.10 | 64K | Coding, math, surprisingly capable |
Before you scroll past the table thinking "yeah, I already know the headline prices," look at the spread. The cheapest input token in this list is Gemini 1.5 Flash at $0.075 per million. The most expensive is Claude 3 Opus at $15 per million. That's a 200x difference on the input side alone, and the output gap is even wider when you compare Opus to Flash. If your application is sending everything through Opus because "it's the best one," you're paying for the privilege of using a sledgehammer to hang a picture frame.
Now apply that spread to a realistic workload. Say you run a customer support copilot that processes roughly 50 million input tokens and 12 million output tokens per month, which is actually quite modest for a mid-sized SaaS product. Sent through Claude 3.5 Sonnet, that's $174 per month on input and $180 on output, for a total of $354. Route the same traffic through Gemini 1.5 Flash and you're at $3.75 on input and $3.60 on output, totaling $7.35. Even if Flash only handled 60% of the queries because the other 40% genuinely need Sonnet's reasoning depth, your blended bill drops from $354 to roughly $147. That's $207/month saved on a single application, and the math scales linearly as you grow.
The other thing the table makes obvious is how aggressively the providers have been pricing smaller models. GPT-4o mini at $0.15 input / $0.60 output is genuinely good at structured extraction, JSON formatting, sentiment classification, and short conversational replies. Mistral Small at $0.20/$0.60 is in the same neighborhood. These are not "downgrades." They're specialized tools for jobs that don't require a 200-billion-parameter model to solve. Using GPT-4o to summarize a five-sentence customer email is the AI equivalent of renting a cargo van to move a single book.
The Hidden Tax: Context, Caching, and Output Bloat
Sticker price is only half the story. The other half lives in three places that almost nobody audits: system prompt bloat, redundant context, and runaway output tokens. Each one of these can quietly inflate your bill by 2x to 10x without changing the visible behavior of your application.
System prompt bloat is the easiest to diagnose. Open your codebase, find the prompt you're sending on every request, and count the tokens. I guarantee it's larger than you think. I've audited startups where the system prompt was 4,000 tokens long because somebody added "helpful guardrails" six months ago and nobody removed them. At $2.50 per million input tokens with GPT-4o, those 4,000 tokens cost a full penny per request. Across 100,000 requests per month, you've spent $1,000 just on the part of the prompt that says "you are a helpful assistant who refuses to discuss politics." Compress that prompt to 800 tokens and you save $800/month. The model will not suddenly start endorsing candidates.
Redundant context is sneakier. If you're building a RAG pipeline, every request usually ships the user's question plus the top-k retrieved chunks plus the conversation history plus the system prompt. The conversation history grows with every turn. By turn 20, you're spending more on shipping the chat log back to the model than on the actual answer. Most providers now offer prompt caching, which discounts repeated input tokens by 50% to 90% depending on the provider. Anthropic's cache reads cost $0.30 per million tokens for Sonnet, a 90% discount on the standard $3.00 input price. OpenAI's automatic caching on GPT-4o gives you 50% off cached input. If you're not using these caches, you're burning money on tokens the provider has already processed for free.
Output bloat is the most painful of the three because output tokens cost 3x to 8x more than input tokens across the board. Every time your prompt says "explain in detail" or your code requests "verbose output" or your function-calling schema has fifteen optional fields, you're paying premium rates for words the user might never read. A common pattern I see in production is a chain-of-thought prompt that asks the model to "think step by step before answering." That reasoning is great for accuracy, but it's also billed as output tokens. If the final answer is 200 tokens but the reasoning is 1,500 tokens, you've paid for 1,500 tokens of scratchpad work at $15 per million. Across volume, this is the single largest source of AI overspending I've found.
Smart Routing: One Endpoint, Many Models, Right Model Every Time
The single most effective cost optimization I've shipped in the last year is also the simplest conceptually: don't send every request to the same model. Classify the request first, then route it to the cheapest model that can handle the task. The classification step itself uses a tiny cheap model, so the overhead is negligible. The savings are not.
Below is a working Python example that demonstrates this pattern using the unified global-apis.com/v1 endpoint. The key insight is that this single endpoint speaks the OpenAI SDK format, so you can swap it in for your existing OpenAI client with literally one line of code. You send the same chat completion payload, but you can change the model field to any of the 184+ supported models without managing separate API keys, separate SDKs, or separate billing relationships.
# smart_router.py
# A lightweight router that picks the cheapest viable model per request.
# Uses global-apis.com/v1 as a single OpenAI-compatible endpoint.
import os
import json
from openai import OpenAI
client = OpenAI(
api_key=os.environ["GLOBAL_APIS_KEY"],
base_url="https://global-apis.com/v1",
)
# Tier definitions: (model, max_input_tokens, cost_per_m_input, cost_per_m_output)
TIERS = {
"nano": ("gemini-1.5-flash", 1_000_000, 0.075, 0.30),
"mini": ("gpt-4o-mini", 128_000, 0.15, 0.60),
"mid": ("claude-3-5-haiku", 200_000, 0.80, 4.00),
"pro": ("claude-3-5-sonnet", 200_000, 3.00, 15.00),
"ultra": ("gpt-4o", 128_000, 2.50, 10.00),
}
CLASSIFY_PROMPT = """You are a routing classifier. Read the user's request and the
conversation so far, then respond with a single JSON object: {"tier": "nano" | "mini"
| "mid" | "pro" | "ultra", "reason": "<12 words>"}. Tiers: nano=tiny task,
mini=structured extraction, mid=short chat/summary, pro=multi-step reasoning,
ultra=complex multimodal or long planning. Reply with JSON only."""
def classify(messages):
"""Use the cheapest model to decide which tier this request needs."""
resp = client.chat.completions.create(
model=TIERS["nano"][0],
messages=[{"role": "system", "content": CLASSIFY_PROMPT}] + messages,
temperature=0.0,
max_tokens=60,
response_format={"type": "json_object"},
)
try:
parsed = json.loads(resp.choices[0].message.content)
return parsed.get("tier", "mid")
except Exception:
return "mid"
def estimate_cost(model_name, in_tokens, out_tokens):
for tier, (m, _, in_cost, out_cost) in TIERS.items():
if m == model_name:
return (in_tokens / 1_000_000) * in_cost + (out_tokens / 1_000_000) * out_cost
return 0.0
def chat(user_messages):
"""Route to the cheapest tier that fits the task."""
tier = classify(user_messages)
model_name, max_ctx, _, _ = TIERS[tier]
resp = client.chat.completions.create(
model=model_name,
messages=user_messages,
temperature=0.3,
)
usage = resp.usage
cost = estimate_cost(model_name, usage.prompt_tokens, usage.completion_tokens)
return {
"reply": resp.choices[0].message.content,
"tier": tier,
"model": model_name,
"input_tokens": usage.prompt_tokens,
"output_tokens": usage.completion_tokens,
"cost_usd": round(cost, 6),
}
if __name__ == "__main__":
# Example: a short request lands on nano, a complex one would land on pro.
print(chat([{"role": "user", "content": "Extract the email addresses from: 'hi bob@acme.com and alice@example.org wrote in'"}]))
print(chat([{"role": "user", "content": "Plan a 7-day itinerary for a family of four visiting Kyoto in April, balancing temples, food, and a rest day."}]))What's happening above is worth pausing on. Every request hits the same endpoint, authenticates with the same key, and returns a response in the same shape your existing code already handles. But internally, the cheap classification step decides whether this particular message needs Gemini Flash at fractions of a cent per call or Sonnet at a few pennies per call. For a customer support inbox where 70% of messages are "where is my order" or "can you reset my password," you can realistically route 70% of traffic to Flash and bring your inference bill down by 5x or more without touching latency or quality in any way your users would notice.
The second piece worth highlighting is the cost accounting. Every call returns not just the reply but a precise dollar figure, broken down by input and output tokens. This is the kind of telemetry that turns "we think AI is expensive" into "AI cost us $0.000342 on this request, $412 total this month, projected $4,950 next month." When you can see the cost of each feature, you can make informed decisions about which ones to keep, which to optimize,