Keeping Embeddings in Sync

Treat the embedding as derived data with a contract. Store, alongside the vector: the source row id, a hash of the exact text embedded, the model name and version, the chunking parameters, and a timestamp. That metadata is what makes every subsequent operation decidable — the content hash tells you whether a row needs re-embedding, and the model name tells you which vectors belong to the current index.

Triggering re-embeds. The reliable pattern is the same as any dual write: when content changes, mark the row (or write an outbox/job row) in the same transaction as the content update, and let a worker embed asynchronously. Doing the embedding call inline in the request holds a transaction open across a slow API — the failure mode from the transactions topic — and makes writes fail whenever the provider does. CDC on the source table is the same idea without application involvement.

Model migrations are the interesting case. Changing embedding model invalidates every vector, and you cannot mix models in one index — distances across models are meaningless. The safe procedure is a new column or table for the new model, backfill in batches while continuing to serve from the old one, verify retrieval quality with your eval set, then switch reads and drop the old vectors. This is expand-migrate-contract again, applied to derived data, and it is worth pointing out that pinning the embedding model version is therefore a real operational commitment.

Deletions and permissions are the correctness risk. A deleted or unshared document whose chunks remain in the index will be retrieved and quoted back to a user who should not see it. Deletion must cascade to chunks (a foreign key with ON DELETE CASCADE if they live in Postgres), and permission filters must be applied at query time against live data rather than baked into the vector metadata at index time.

Finally, make the backfill resumable and rate-limit-aware: batch, checkpoint, retry with backoff, and expect the provider to fail partway through a million-row job.

Most RAG systems degrade not because retrieval is badly designed but because the index quietly drifts from the source — stale chunks, deleted documents, a half-finished backfill after a model change. Being able to describe the sync pipeline is what separates "I built a demo" from "I ran this in production" in an AI-forward interview.

• Store a content hash, model name and chunk parameters with every vector, or you cannot tell what is stale.
• Never call an embedding API inside the transaction that writes the content; enqueue the work instead.
• Marking work in the same transaction as the content change is the outbox pattern applied to embeddings.
• Vectors from different models are not comparable, so a model change is a full re-index, not an incremental update.
• Migrate models with a parallel column, batched backfill, eval-based verification, then a read switch.
• Deleted or newly restricted documents must be removed from the index, or retrieval leaks them into answers.
• Apply permission filters at query time against live data, not against metadata frozen at indexing time.