Postgres is all you need (until 10M rows)
A pragmatic look at when to add a queue, a cache, a vector store — and the surprising amount of infrastructure you can defer by keeping Postgres in the middle.
A pragmatic look at when to add a queue, a cache, a vector store — and the surprising amount of infrastructure you can defer by keeping Postgres in the middle.
Every architecture decision we have made in the last three years has started with the same question: "Can Postgres do this?" More often than not, the answer is yes — or yes, with a caveat that turns out to be tolerable.
Queues. Not at Twitter scale, but at the scale of most applications? Absolutely. SELECT ... FOR UPDATE SKIP LOCKED is a reliable, transactional job queue with no additional infrastructure. We ran our notification pipeline this way for two years and 40 million jobs before we hit a ceiling.
Full-text search. For most applications, tsvector and GIN indexes are completely sufficient. We ran our search on Postgres for 18 months. The reason we eventually moved to a dedicated search service was not performance — it was the need for faceted search and relevance tuning, not throughput.
The most dangerous infrastructure decision is adding a service you do not need yet. Every service you add is a deployment, a failure mode, an ops burden, and an on-call page at 3am.
Rate limiting, session storage, and pub/sub are the only three cases where we consistently recommend Redis over Postgres. The reason is not performance — Postgres is fast enough for all three — it is the data model. Redis's sorted sets and atomic operations map more naturally to these patterns than Postgres does.
Our empirical finding, across a dozen products: Postgres handles most workloads comfortably to 10 million rows in the hot tables, assuming sensible indexing. Beyond that, you hit a ceiling that depends more on your query patterns than on Postgres itself. Partitioning, read replicas, and connection pooling extend the runway significantly — but they also add complexity.