A vector database is purpose-built to store, index, and search high-dimensional vectors at scale. Where a relational database asks "find rows where id = 42", a vector database asks"find the 10 vectors most similar to this query vector" — in milliseconds across millions of entries.
You could store vectors in a Postgres FLOAT[] column. For 1,000 documents, a brute-force scan works fine. At 1 million documents, computing cosine similarity against every row takes seconds. At 100 million, it is completely infeasible.
Vector databases solve this with Approximate Nearest Neighbor (ANN)indices — data structures that find the ~10 closest vectors without checking every single one. The trade-off is a small probability of missing the absolute closest match (hence "approximate"), but in practice recall stays above 95% while latency drops from seconds to milliseconds.
| Capability | Postgres | Vector DB |
|---|---|---|
| Exact row lookup by id | ✅ | ✅ |
| Filter by metadata (date, category) | ✅ | ✅ |
| Nearest-neighbor vector search | ❌ (slow) | ✅ (fast) |
| Billion-scale ANN index | ❌ | ✅ |
| ACID transactions | ✅ | ⚠️ varies |
| Complex JOIN queries | ✅ | ❌ |
Note: pgvector adds ANN search to Postgres, making it a hybrid option — excellent for moderate scale (<10M vectors).
ChromaDB runs in-process (no server needed), persists to disk, and has a clean Python API. It is the fastest way to get started and the right choice for prototypes and small production systems (<1M vectors).
import chromadb
from openai import OpenAI
# --- Setup ---
chroma = chromadb.PersistentClient(path="./my_vectordb")
openai = OpenAI()
def embed(text: str) -> list[float]:
return openai.embeddings.create(
model="text-embedding-3-small",
input=text,
).data[0].embedding
# --- Create a collection (like a table) ---
collection = chroma.get_or_create_collection(
name="my_docs",
metadata={"hnsw:space": "cosine"}, # Use cosine distance
)
# --- Add documents ---
documents = [
"Neural networks learn by adjusting weights using backpropagation.",
"Python is the most popular language for machine learning.",
"The Eiffel Tower is located in Paris, France.",
]
collection.add(
ids=["doc1", "doc2", "doc3"],
embeddings=[embed(doc) for doc in documents],
documents=documents,
metadatas=[{"source": "textbook"}, {"source": "blog"}, {"source": "wiki"}],
)
print(f"Collection size: {collection.count()}") # → 3
# --- Query ---
results = collection.query(
query_embeddings=[embed("How do neural networks train?")],
n_results=2,
)
for doc, distance in zip(results["documents"][0], results["distances"][0]):
print(f"[{distance:.3f}] {doc[:60]}...")Every vector database exposes the same four primitives:
upsert(id, vector, metadata)Add or update a document. Always include metadata for filtering.
query(vector, n_results, where)Find the n closest vectors to the query. where filters by metadata before the ANN search.
get(ids)Retrieve specific documents by their IDs — useful for hydration.
delete(ids)Remove documents. Most DBs require you to re-embed if you update the content.
The five documents below represent a small collection in a vector database. Select a query to see which documents get retrieved (highlighted) vs. which are pushed down (dimmed).
Real collections mix documents from different sources, dates, or categories. Use metadata filters to scope retrieval before the ANN search runs:
# Only search documents from "textbook" source
results = collection.query(
query_embeddings=[embed("How do neural networks train?")],
n_results=3,
where={"source": "textbook"}, # Pre-filter by metadata
)
# Filter by date range (using ChromaDB's $gte / $lte operators)
results = collection.query(
query_embeddings=[embed("recent AI developments")],
n_results=5,
where={"year": {"$gte": 2024}},
)In Lesson 5, we go deeper into how the similarity calculation works — and why choosing between cosine similarity, dot product, and Euclidean distance is not arbitrary.