AI System Design for Engineers • Module A: Architecture PatternsLesson 1: Model Selection — Capability, Cost & Latency Trade-offs
Picking a model is not a one-time decision — it is an architecture decision that affects latency, cost, quality, and maintainability for the lifetime of your system. This lesson gives you a repeatable framework instead of guesswork.
Every model selection involves trading off three axes. You can optimize for two, but rarely all three:
Capability (quality)
▲
│
claude-sonnet ──┤── claude-opus
│
claude-haiku ──────┤
│
gpt-4o-mini ────┘
◄────────────────►
Cost & Latency (lower = better)
Rule: Pick the cheapest/fastest model that meets your quality bar.
Test if the cheaper model meets quality — don't assume the expensive one is necessary.Select your task type to see the recommended model and a code snippet:
response = client.messages.create(
model="claude-haiku-4-5-20251001",
max_tokens=50,
messages=[{
"role": "user",
"content": f"Classify this query as: billing, technical, general\n\n{query}"
}],
)
category = response.content[0].text.strip()import anthropic
client = anthropic.Anthropic()
MODELS_TO_TEST = [
"claude-haiku-4-5-20251001", # Test cheapest first
"claude-sonnet-4-6",
# "claude-opus-4-8", # Only if sonnet fails
]
def evaluate_model_for_task(model: str, test_cases: list[dict]) -> float:
"""Score a model on your specific task. Pick the cheapest that passes."""
scores = []
for case in test_cases:
response = client.messages.create(
model=model,
max_tokens=500,
messages=[{"role": "user", "content": case["prompt"]}],
)
score = judge_output(response.content[0].text, case["expected"])
scores.append(score)
return sum(scores) / len(scores)
QUALITY_THRESHOLD = 0.90 # 90% of test cases must pass
for model in MODELS_TO_TEST:
score = evaluate_model_for_task(model, test_cases)
print(f"{model}: {score:.1%}")
if score >= QUALITY_THRESHOLD:
print(f"→ USE THIS MODEL: {model}")
break # Stop — cheapest model that meets quality bar
# → claude-haiku-4-5-20251001: 93.2%
# → USE THIS MODEL: claude-haiku-4-5-20251001# Approximate pricing (verify at anthropic.com/pricing — changes frequently)
PRICING = {
"claude-haiku-4-5-20251001": {"input": 0.00025, "output": 0.00125}, # per 1K tokens
"claude-sonnet-4-6": {"input": 0.003, "output": 0.015},
"claude-opus-4-8": {"input": 0.015, "output": 0.075},
}
def estimate_monthly_cost(
model: str,
avg_input_tokens: int,
avg_output_tokens: int,
requests_per_day: int,
) -> float:
p = PRICING[model]
cost_per_request = (
avg_input_tokens / 1000 * p["input"]
+ avg_output_tokens / 1000 * p["output"]
)
return cost_per_request * requests_per_day * 30
# Example: RAG app, 10K requests/day
for model in PRICING:
monthly = estimate_monthly_cost(model, 2000, 300, 10_000)
print(f"{model}: ${monthly:,.0f}/month")
# → claude-haiku-4-5-20251001: $637/month
# → claude-sonnet-4-6: $9,900/month
# → claude-opus-4-8: $49,500/monthAlmost every AI system has tasks of different complexity. Classify them: use haiku for routing and extraction, sonnet for generation, and reserve opus (or a domain fine-tune) for the small fraction of genuinely hard cases. The cost difference between tiers is 10–60× — the quality difference is usually much smaller.
In the next lesson, we implement this tiering mechanically as a model cascade — a system that routes each request to the cheapest model likely to handle it well, and automatically escalates when confidence is low.