Even with perfect schema design, LLMs fail sometimes: they return the wrong type, hallucinate a field value, or the API returns a 429. Retry and fallback patterns make your structured output pipeline resilient to these failures.
| Error Type | Cause | Recovery |
|---|---|---|
| ValidationError | Wrong field type, out-of-range value, missing required field | Retry with error in prompt |
| JSONDecodeError | Model returned markdown or prose instead of JSON | Extract JSON with regex, then retry |
| RateLimitError (429) | Too many requests per minute/day | Exponential backoff + honor Retry-After header |
| APIConnectionError | Network issue, provider outage | Exponential backoff, circuit breaker |
| All retries exhausted | Model consistently fails this input | Rule-based fallback or return safe default |
Best for: Pydantic validation fails (wrong type, out-of-range value, missing field). Retry with the validation error in the prompt so the model can self-correct.
from pydantic import BaseModel, ValidationError, Field
import anthropic
import instructor
client = instructor.from_anthropic(anthropic.Anthropic())
class ExtractedData(BaseModel):
product_name: str
price: float = Field(ge=0, description="Price in USD, must be non-negative")
quantity: int = Field(ge=0, le=10000)
category: str
def extract_with_retry(text: str, max_retries: int = 3) -> ExtractedData | None:
last_error = None
for attempt in range(max_retries):
try:
# instructor handles validation retry internally
# but here's the manual version for full control:
result = client.messages.create(
model="claude-haiku-4-5-20251001",
max_tokens=300,
response_model=ExtractedData,
max_retries=max_retries, # instructor retries on ValidationError
messages=[{
"role": "user",
"content": f"Extract product data from: {text}"
+ (f"\n\nPrevious attempt failed: {last_error}" if last_error else ""),
}],
)
return result
except ValidationError as e:
last_error = str(e)
print(f"Attempt {attempt + 1} failed: {e}")
print("All retries exhausted")
return NoneInclude the validation error in the retry prompt.“You returned price=-5 but price must be non-negative. Try again.” gives the model the specific fix it needs. Retrying without context just reproduces the same error.
Add jitter to backoff delays. Without jitter, all clients that hit a rate limit at the same time will retry at the same time — creating a thundering herd. delay += random.uniform(0, 1) is enough to spread the load.
Lesson 4 covers rate limiting on the producer side: how to control how fast your application sends requests so you never hit API rate limits in the first place.