Read replicas will not solve the problems you think they solve

By Emeka Chibuike · 21 July 2026163 views
Read replicas will not solve the problems you think they solve

A database admin is asked to fix slow page load times on a user-facing dashboard. The primary Postgres database is at 80% CPU. The solution, everyone agrees, is a read replica. The replica is provisioned in an afternoon. Traffic is routed to it. CPU on the primary drops to 40%. Two weeks later, the page loads are still slow. The queries that were slow on the primary are slow on the replica. The CPU problem is solved. The performance problem is not.

This is the most common read replica misconception. A read replica reduces the load on the primary by handling queries that would otherwise run there. It does not make those queries faster. A 3-second query on the primary is a 3-second query on the replica.

Understanding what read replicas actually solve — and what they do not — prevents a category of architectural decisions that spend infrastructure budget without fixing the actual problem.

What read replicas actually do

A read replica is a continuously synchronized copy of the primary database. Writes go to the primary and are replicated to replicas asynchronously. Reads can be distributed across replicas, reducing the number of queries the primary must handle.

The benefit is throughput, not latency. A primary that can handle 1,000 queries per second can be supplemented with a replica that handles another 1,000 queries per second, for a combined read throughput of 2,000 queries per second. Neither the primary nor the replica handles individual queries faster than before.

The benefit is most applicable when:

  • The database is CPU-bound due to query volume, not query duration
  • Reads are significantly more frequent than writes
  • The reads can tolerate replication lag (stale data for a bounded window)

If the database is CPU-bound due to a few slow queries that run frequently, the slow queries must be fixed. Adding a replica runs the slow queries on the replica instead of the primary, removing the CPU load from the primary, but users still wait for the slow query to complete.

What actually fixes slow queries

A slow query on a read replica is the same slow query that was slow on the primary. The tools for fixing it are the same regardless of where the query runs.

Missing indexes. The most common cause of slow queries. A query that scans a large table without an index takes seconds. The same query with an appropriate index takes milliseconds.

-- Find slow queries on Postgres (requires pg_stat_statements extension)
SELECT
    query,
    calls,
    total_exec_time / calls AS avg_ms,
    rows / calls AS avg_rows,
    100.0 * shared_blks_hit / NULLIF(shared_blks_hit + shared_blks_read, 0) AS cache_hit_pct
FROM pg_stat_statements
WHERE calls > 100
ORDER BY avg_ms DESC
LIMIT 20;

-- EXPLAIN ANALYZE to see the query plan for a specific slow query
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
SELECT u.id, u.email, o.total, o.created_at
FROM orders o
JOIN users u ON u.id = o.user_id
WHERE o.created_at > NOW() - INTERVAL '30 days'
ORDER BY o.created_at DESC;

-- If this shows "Seq Scan on orders" rather than "Index Scan", the query needs an index
CREATE INDEX CONCURRENTLY idx_orders_created_at ON orders (created_at DESC);

After adding the index, the query plan changes from a sequential scan to an index scan. The query that took 2.3 seconds takes 18 milliseconds.

N+1 queries. An application that fetches a list of orders and then fetches each order's user in a separate query executes N+1 queries for a list of N orders. On a list of 100 orders, this is 101 queries. Adding a replica does not reduce the 101 queries to 1.

# N+1 pattern — 101 queries for 100 orders
def get_order_dashboard(user_id: str) -> list:
    orders = db.query(Order).filter_by(user_id=user_id).limit(100).all()
    return [
        {
            "order_id": str(o.id),
            "status": o.status,
            "customer_name": db.query(User).get(o.customer_id).name  # N+1 here
        }
        for o in orders
    ]

# Fixed — 1 query with a join
def get_order_dashboard(user_id: str) -> list:
    rows = db.execute(
        """
        SELECT o.id, o.status, u.name AS customer_name
        FROM orders o
        JOIN users u ON u.id = o.customer_id
        WHERE o.user_id = %s
        ORDER BY o.created_at DESC
        LIMIT 100
        """,
        (user_id,)
    ).all()
    return [
        {"order_id": str(r["id"]), "status": r["status"], "customer_name": r["customer_name"]}
        for r in rows
    ]

The fix is 1 query. A replica would have run 101 queries on the replica instead of the primary. The latency would be unchanged.

Missing pagination. A query that fetches all rows from a large table to display the first 20 to a user processes the entire table. Adding an index helps but does not fix the fundamental problem of fetching more data than is needed.

Unoptimized query structure. A query that uses subqueries where a join would be more efficient, or that applies functions to indexed columns (preventing index use), is slower than it needs to be regardless of which server runs it.

The correctness problems read replicas introduce

When read replicas are added without understanding the replication lag implications, applications develop correctness bugs that are difficult to trace.

Read-after-write consistency. A user creates an account. The write goes to the primary. The confirmation redirect reads from the replica. The replica has not yet received the write. The application shows "User not found." The user, confused, tries again. The second read may hit the primary (or a different replica that has caught up) and succeed.

# This pattern is unsafe with a read replica
def register_user(email: str, password: str) -> Response:
    user = User.create(email=email, password_hash=hash(password))
    primary_db.commit()

    # Unsafe: reading from replica immediately after write
    # The replica may not have the new user yet
    created_user = replica_db.query(User).filter_by(email=email).first()
    if not created_user:
        return Response(status=500, body="User creation failed")  # False negative

    return Response(status=201, body={"user_id": str(created_user.id)})

# Safe pattern: return data from the write, not from a subsequent read
def register_user(email: str, password: str) -> Response:
    user = User.create(email=email, password_hash=hash(password))
    primary_db.commit()

    # Return the created user's data directly — no replica read needed
    return Response(status=201, body={"user_id": str(user.id), "email": user.email})

Stale data in user-visible contexts. A user updates their shipping address. The write goes to the primary. The order page reads the shipping address from the replica. The replica shows the old address. The user sees the old address and edits it again. They submit. The write goes to the primary. The primary now has two conflicting updates and the most recent one wins — which may not be the one the user intended.

Payment and access control errors. A user upgrades their subscription. The write goes to the primary. The feature access check reads from the replica. The replica shows the old subscription level. The user is denied access to a feature they just paid for. This is the most damaging form of stale read — it directly affects the user's perceived value of the product.

These bugs are intermittent. They are most common under load (when replication lag increases) and in the milliseconds to seconds immediately after a write. They are almost impossible to reproduce in a development environment. They appear as low-rate user complaints that are hard to correlate with a specific technical cause.

When read replicas are the right answer

Read replicas are the right answer when the primary database's CPU is genuinely bound by query volume — too many queries, not too slow queries — and when the reads can tolerate replication lag.

Specific cases where replicas unambiguously help:

  • Analytics and reporting queries that run against current data but tolerate 30-60 seconds of staleness
  • Background jobs that do not depend on very recent writes (nightly report generation, data export, ETL)
  • Read-heavy features where the data changes infrequently (product catalog reads, static content)
  • Load testing and read benchmarking without affecting the production primary

The decision to add a replica should follow a diagnosis of the actual performance problem, not precede it. If the diagnosis shows that the primary is handling 5,000 slow queries per second — 5,000 queries that each take 2 seconds — the 5,000 slow queries need to be fixed. A replica will run them on separate hardware, but the users waiting for those queries to complete will still wait 2 seconds per query.

Common mistakes when adding read replicas

Routing all reads to the replica without auditing which reads need consistency. A blanket configuration that sends every read to the replica will eventually produce a stale-read bug in a path that should have used the primary. Access control checks, subscription status checks, fraud detection, and payment verification should always read from the primary. The migration to replica reads should be explicit and opt-in, not blanket and opt-out.

Using the replica for writes validation. After a write completes, reading from the replica to validate the write result is a race condition. The replica may not have received the write yet. The correct approach is to validate from the data returned by the write transaction itself, not from a subsequent read:

# Wrong: validates by reading from replica — race condition
def update_user_email(user_id: str, new_email: str) -> User:
    primary_db.execute("UPDATE users SET email = %s WHERE id = %s", (new_email, user_id))
    # Replica may not have this update yet
    return replica_db.query(User).filter_by(id=user_id).first()

# Right: returns data from the write operation
def update_user_email(user_id: str, new_email: str) -> dict:
    primary_db.execute("UPDATE users SET email = %s WHERE id = %s", (new_email, user_id))
    return {"user_id": user_id, "email": new_email}  # Known from the write

Not monitoring replication lag. The only way to know how stale the replica's data is at any moment is to monitor replication lag continuously. A replica that is 500ms behind under normal conditions may be 30 seconds behind during a write spike. Applications that assume the replica is "close enough" without monitoring actual lag will encounter stale-read bugs during exactly the periods when the system is under most stress — when writes are highest and lag is largest.

-- Monitor replication lag on Postgres
-- Run on the primary to see lag per replica
SELECT
    client_addr AS replica_address,
    state,
    sent_lsn,
    write_lsn,
    flush_lsn,
    replay_lsn,
    write_lag,
    flush_lag,
    replay_lag
FROM pg_stat_replication;

-- Run on the replica to check its own lag from the primary
SELECT
    NOW() - pg_last_xact_replay_timestamp() AS replication_lag;

Alert when replication lag exceeds the threshold at which stale reads would affect correctness. For most applications, a lag alert at 5 seconds is appropriate. For applications with write-then-read patterns in user-facing flows, a lag alert at 500ms is more appropriate.

Treating the replica as a solution to connection pool exhaustion. When the primary's connection pool is exhausted, adding a replica and routing reads to it reduces the number of read connections on the primary. But if the pool exhaustion was caused by slow queries, the slow queries now exhaust the replica's connection pool instead. The connection pool problem requires either fixing slow queries (reducing query duration), increasing the pool size (if the server has capacity), or using a connection pooler like PgBouncer that multiplexes many application connections over fewer database connections.

Not testing with real replication lag. Development and staging environments typically have replicas with negligible replication lag — the write volume is low and the servers are co-located. Bugs caused by replication lag do not appear in these environments. A staging environment configured to introduce artificial replication lag (or using a replica that replicates with a configured delay) catches stale-read bugs before they reach production.

The diagnostic sequence that should precede any replica decision

Before provisioning a read replica, complete this diagnostic sequence:

  1. Run pg_stat_statements to identify the top 20 queries by total execution time
  2. Run EXPLAIN ANALYZE on each of the top queries
  3. Identify whether each slow query has a sequential scan that an index would prevent
  4. Add missing indexes and re-measure query latency
  5. Identify N+1 query patterns using query logs grouped by request ID
  6. Fix N+1 patterns with appropriate joins or batch fetches
  7. Re-measure primary CPU and latency

In most cases, steps 3–6 reduce primary CPU by 40–70% and user-facing latency by a similar margin. If primary CPU remains high after fixing queries — because there are simply many well-optimized queries running concurrently — a replica is the right next step. The replica is now being used for its actual purpose: distributing query volume, not hiding slow queries.

The engineer who treats "add a read replica" as a first response to database performance problems will spend infrastructure budget without fixing the performance problem, and will introduce replication lag bugs that appear intermittently in production for months before they are correctly diagnosed. The database performance problem requires query analysis first. A replica is sometimes the right answer to what query analysis reveals. It is rarely the right starting point.

What good replica routing looks like in practice

When read replicas are justified — because the primary is handling high query volume of well-optimized queries — the routing logic must be explicit about which queries go where.

class DatabaseRouter:
    """
    Explicit routing layer for read replica decisions.
    Forces every read to be categorized rather than defaulting everything to replica.
    """
    def __init__(self, primary, replica):
        self._primary = primary
        self._replica = replica

    @property
    def primary(self):
        """
        Use for: writes, access control checks, subscription status,
        fraud detection, any read immediately following a write.
        """
        return self._primary

    @property
    def replica(self):
        """
        Use ONLY for: analytics queries, reporting, background jobs,
        read-heavy features where 1-2s staleness is documented and acceptable.
        NOT for: access control, payment verification, reads after writes.
        """
        return self._replica

    def for_access_control(self):
        """Access control must always use the primary."""
        return self._primary

    def for_analytics(self):
        """Analytics can tolerate replica lag."""
        return self._replica

    def for_user_facing_data(self, session_recently_wrote: bool = False):
        """
        User-facing reads go to replica unless the session recently wrote.
        Pass session_recently_wrote=True if a write occurred in this session
        in the last 10 seconds.
        """
        return self._primary if session_recently_wrote else self._replica

Making the routing decision explicit at the call site — rather than configuring it globally as "all reads go to replica" — ensures that every read has been thought about. An access control check that accidentally passes for_analytics() is a bug that is visible in code review. A global replica configuration that silently routes all reads is a bug that is visible only in production incidents.

When to reconsider the replica

A replica that was provisioned to address high write volume on the primary should be reconsidered when the write volume drops. Teams that provision replicas during a traffic spike and then reduce write volume through query optimization sometimes continue running the replica indefinitely — paying for infrastructure that is no longer solving the original problem.

Monitor the primary CPU and query volume continuously. If the primary CPU drops below 40% after query optimization work, the replica may be providing more replication lag risk than throughput benefit. The calculation is: replica cost + replication lag debugging overhead vs. primary headroom. At low enough primary utilization, keeping reads on the primary is simpler and more correct.

The direction of database architecture in most growing products is: start on a single primary, add indexes before adding infrastructure, add a replica when the primary is genuinely volume-bound, and consider sharding or read-specialized databases (search indexes, columnar stores) only when replicas are saturated. Each step should be validated by measurement before the next is considered. The teams that follow this progression spend their infrastructure budget on problems that are actually present, not problems they expect to have in the future.

Comments

No comments yet. Be the first!

Sign in to leave a comment.