An agent without memory starts every conversation from scratch. Memory architectures determine what the agent “knows” across turns, across sessions, and across users — and the choice profoundly affects both capability and cost.
Select a memory type to see its scope, storage mechanism, and implementation.
Use when: Short conversations, simple Q&A agents, prototyping. Zero infrastructure required — every message is passed in the messages array. Cost scales with conversation length.
# In-context memory: just pass all messages every call
from anthropic import Anthropic
client = Anthropic()
conversation_history = []
def chat(user_message: str) -> str:
conversation_history.append({
"role": "user",
"content": user_message,
})
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
system="You are a helpful assistant.",
messages=conversation_history, # Full history every call
)
assistant_message = response.content[0].text
conversation_history.append({
"role": "assistant",
"content": assistant_message,
})
return assistant_message
# Turn 1
print(chat("My name is Alice"))
# Turn 2 — remembers because full history is in context
print(chat("What is my name?")) # → "Your name is Alice."def summarize_history_node(state: AgentState) -> dict:
"""Summarize old messages to keep context window manageable."""
messages = state["messages"]
KEEP_RECENT = 10 # Keep last 10 messages verbatim
if len(messages) <= KEEP_RECENT:
return {} # No action needed
old_messages = messages[:-KEEP_RECENT]
recent_messages = messages[-KEEP_RECENT:]
summary_response = model.invoke([
HumanMessage(content=f"Summarize this conversation history in 3-5 bullet points: {old_messages}"),
])
summary = summary_response.content
# Replace old messages with summary
return {
"messages": [SystemMessage(content=f"[Conversation summary: {summary}]")] + recent_messages,
}With in-context memory every call re-sends the whole history. If each turn (one user message plus one reply) adds about t tokens, then call number k sends roughly k × t input tokens, and the total input over N turns is t × (1 + 2 + ... + N) = t × N(N + 1) / 2. This grows with the square of the conversation length. For t = 300 and N = 20:
t, N = 300, 20
total = t * N * (N + 1) // 2 # 300 * 210 = 63,000 input tokens across the conversation
last_call = t * N # 6,000 tokens on the final call alone
# With a rolling window of the last 5 turns (10 messages) plus a ~200-token summary:
window = 5 * t + 200 # 1,700 tokens per call once the window is full
capped = t * (1 + 2 + 3 + 4 + 5) + (N - 5) * window # 4,500 + 15 * 1,700 = 30,000Capping the context roughly halves the input tokens in this 20-turn example, and the gap widens as conversations get longer, because the capped cost grows linearly instead of quadratically. The price is lost detail: anything that was only in the discarded messages and not captured in the summary is gone. All numbers here are illustrative token counts, not measurements.
Long-term memory means storing facts outside the conversation and retrieving the relevant ones when needed. Four decisions determine whether it helps or hurts. What to write:store durable facts and preferences ("prefers metric units"), not every sentence. When to write: after a turn, with a separate step that extracts candidate memories, rather than letting the main model save anything it likes. How to retrieve: by semantic similarity to the current message, limited to a few entries. When to forget: by expiry time, by explicit user request, or when a newer fact contradicts the old one.
import numpy as np
import time
class MemoryStore:
def __init__(self, embed):
self.embed = embed # function: str -> np.ndarray, supplied by you
self.items = [] # in production: a database or vector store
def add(self, user_id: str, fact: str, ttl_days: int = 180):
self.items.append({
"user_id": user_id, # always namespace by user
"fact": fact,
"vec": self.embed(fact),
"expires": time.time() + ttl_days * 86400,
})
def search(self, user_id: str, query: str, k: int = 3, min_sim: float = 0.3):
q = self.embed(query)
now = time.time()
scored = []
for m in self.items:
if m["user_id"] != user_id or m["expires"] < now:
continue # other users' and expired memories never match
sim = float(q @ m["vec"] / (np.linalg.norm(q) * np.linalg.norm(m["vec"])))
if sim >= min_sim:
scored.append((sim, m["fact"]))
return [fact for _, fact in sorted(scored, reverse=True)[:k]]Retrieved facts are then inserted into the prompt, for example in a clearly labeled block of the system message. Because they come from earlier conversations, treat them as data that may be wrong or outdated, not as instructions.
add_messages reducer, a node that returns a new message list appends to the existing messages instead of replacing them. To trim history you need to return RemoveMessage entries for the old ones. Verify with a test that the state shrinks after summarization.In-context memory is exact but its cost grows quadratically with conversation length. Windowing and summarization cap that cost at the price of detail. Long-term memory adds persistence across sessions and needs deliberate rules for what to write, how to retrieve, when to forget, and how to keep users separated.
Next, we scale from single agents to multi-agent systems using CrewAI — where specialized agents collaborate, delegate tasks, and share information.