Lesson 4: Unit Testing LLM Applications

You can write deterministic unit tests for non-deterministic LLM functions. The trick is to test properties of the output — not exact text — and to use a judge LLM when the property requires semantic reasoning.

Three Categories of LLM Unit Tests

Assert that the output is structurally correct: valid JSON, correct fields, expected types. These run in milliseconds and should be in every CI pipeline.

import pytest
import json
import anthropic

client = anthropic.Anthropic()

def extract_entities(text: str) -> dict:
    """LLM function under test."""
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=300,
        messages=[{
            "role": "user",
            "content": f"Extract entities from this text as JSON with keys 'people', 'places', 'orgs'. Text: {text}",
        }],
    )
    return json.loads(response.content[0].text)


def test_output_schema():
    result = extract_entities("Apple CEO Tim Cook announced new products in Cupertino.")
    assert isinstance(result, dict), "Output must be a dict"
    assert "people" in result, "Must have 'people' key"
    assert "places" in result, "Must have 'places' key"
    assert "orgs" in result, "Must have 'orgs' key"
    assert isinstance(result["people"], list), "'people' must be a list"


def test_empty_input_does_not_crash():
    result = extract_entities("")
    assert isinstance(result, dict)


def test_no_entities_returns_empty_lists():
    result = extract_entities("The sky is blue.")
    assert result.get("people", []) == []
    assert result.get("orgs", []) == []

Running LLM Tests in CI

# pytest.ini — configure for LLM tests
[pytest]
markers =
    llm: marks tests that call the LLM API (slow, costs money)
    guardrail: marks safety behavior tests (run on every PR)
    format: marks format/schema tests (fast, no LLM judge)

# CI strategy:
# - Always run: format tests (fast, free)
# - On every PR: guardrail tests (medium, ~$0.01/run)
# - On release: full eval suite (slow, ~$1/run)

# pyproject.toml
[tool.pytest.ini_options]
addopts = "-m format"  # Default: only format tests
# Override: pytest -m "format or guardrail"

# GitHub Actions — .github/workflows/eval.yml
jobs:
  format-tests:
    steps:
      - run: pytest -m format --tb=short

  guardrail-tests:
    if: github.event_name == 'pull_request'
    steps:
      - run: pytest -m guardrail --tb=short
    env:
      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Managing Test Flakiness

LLM tests have inherent flakiness because the model's output varies. Three strategies reduce it:

Temperature=0 for both the tested function and judge calls — reduces output variance dramatically
Run N times, take majority — for critical guardrail tests, call the model 3 times and require 2/3 pass
Test properties, not text — never assert on specific words; assert on presence of concepts or structure

Next, we look at prompt regression testing — how to detect when a prompt change silently degrades model quality across your entire test suite.

import pytest import json import anthropic client = anthropic.Anthropic() def extract_entities(text: str) -> dict: """LLM function under test.""" response = client.messages.create( model="claude-haiku-4-5-20251001", max_tokens=300, messages=[{ "role": "user", "content": f"Extract entities from this text as JSON with keys 'people', 'places', 'orgs'. Text: {text}", }], ) return json.loads(response.content[0].text) def test_output_schema(): result = extract_entities("Apple CEO Tim Cook announced new products in Cupertino.") assert isinstance(result, dict), "Output must be a dict" assert "people" in result, "Must have 'people' key" assert "places" in result, "Must have 'places' key" assert "orgs" in result, "Must have 'orgs' key" assert isinstance(result["people"], list), "'people' must be a list" def test_empty_input_does_not_crash(): result = extract_entities("") assert isinstance(result, dict) def test_no_entities_returns_empty_lists(): result = extract_entities("The sky is blue.") assert result.get("people", []) == [] assert result.get("orgs", []) == []

# pytest.ini — configure for LLM tests [pytest] markers = llm: marks tests that call the LLM API (slow, costs money) guardrail: marks safety behavior tests (run on every PR) format: marks format/schema tests (fast, no LLM judge) # CI strategy: # - Always run: format tests (fast, free) # - On every PR: guardrail tests (medium, ~$0.01/run) # - On release: full eval suite (slow, ~$1/run) # pyproject.toml [tool.pytest.ini_options] addopts = "-m format" # Default: only format tests # Override: pytest -m "format or guardrail" # GitHub Actions — .github/workflows/eval.yml jobs: format-tests: steps: - run: pytest -m format --tb=short guardrail-tests: if: github.event_name == 'pull_request' steps: - run: pytest -m guardrail --tb=short env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}