Connection pool size: how to calculate it with Little's Law

The right connection pool size is the number of queries you need in flight at the busiest moment, and Little's Law tells you what that is: concurrency equals throughput multiplied by the time each connection is held. At 1,200 database transactions a second averaging 5 ms each, only about 6 connections are busy on average, so a pool of hundreds adds contention, not speed. Size the database side from its CPU cores, size the app side with Little's Law, leave headroom for bursts, and confirm both with a load test.

Little's Law in one line

L = λ × W: the average number of items inside a stable system (L) equals the average rate at which they arrive (λ) multiplied by the average time each one spends inside (W). John Little published the proof in 1961 in the journal Operations Research, and it holds whatever the pattern of arrivals, as long as the system is stable over the period you measure. Three readings matter for pool sizing:

• Web tier: requests in flight = requests per second × average response time.
  
  
• Database pool: busy connections = transactions per second × average time a connection is held per transaction.
  
  
• Any queue: items waiting = arrival rate × average wait.
  
  

Keep units consistent, with seconds on both sides, and remember that the law works on averages. Requests don't arrive evenly within a second, so real pools need headroom above the average, as the example below shows.

Why bigger pools make databases slower

It feels as if more connections should mean more work done. A database server, though, has a fixed number of CPU cores and a fixed amount of memory and disk bandwidth. The PostgreSQL wiki describes the curve: throughput climbs as connections rise until resources are saturated, then reaches a knee and falls, because extra connections only compete for locks, memory, CPU caches and context switches. Its advice is striking: PostgreSQL will usually finish the same batch of transactions sooner by running a few at a time than by running hundreds at once.

The team behind HikariCP, a widely used Java connection pool, turns this into a design rule: "You want a small pool, saturated with threads waiting for connections." Waiting in the pool's queue is cheap. Waiting inside the database, while holding locks and memory and sharing a CPU core with dozens of other queries, is not.

Sizing the database side

1. Count the database server's physical cores, not hyperthreads.
   
   
2. Start from the PostgreSQL wiki's formula for active connections: (core_count × 2) + effective_spindle_count. The spindle count is zero when your working data fits in memory, so a 16-core server starts at about 32. The wiki notes the formula predates SSDs, so treat it as a starting point for testing, not an answer.
   
   
3. Split that budget across workloads, such as web requests, background workers and reports, using separate pools in PgBouncer.
   
   
4. Set max_connections a little above the total of all pools plus admin, monitoring and replication connections. PostgreSQL also holds three slots for superusers by default.
   
   
5. Load test and adjust in both directions.
   
   

Sizing the app side: workers, threads and pools

The app side is where Little's Law earns its keep. Every request in flight needs a worker to run on, and every worker that is talking to the database needs a connection.

• Worker slots. Gunicorn's design notes suggest starting with (2 × CPU cores) + 1 worker processes per server and say that 4 to 12 workers usually handle heavy traffic. With the gthread worker class, each process also runs several threads, so slots per server = workers × threads.
  
  
• Servers or pods. Divide the requests in flight by the slots per server, and leave room: running slots at around 60% keeps queues short when traffic jumps.
  
  
• App-side pool per process. Give each process roughly as many connections as it has threads, so threads don't wait on each other. These connections go to PgBouncer and are cheap, but they still add up: pods × workers × pool size.
  
  
• Server-side pool. This is the number that matters to the database, and it comes from the database formula above, not from the app's thread count.
  
  

The two sides differ because they hold connections for different lengths of time. An app-side connection is typically held from the first query of a request until the request ends. With PgBouncer in transaction mode, a server connection is held only while a transaction runs, which for short autocommit queries is a few milliseconds. That gap is what lets a few dozen server connections serve hundreds of app threads; the guide to connection pooling explains the two-tier design.

A worked example

Take an illustrative exam platform at the peak of a mock test, with a 16-core database behind PgBouncer:

Now see how sensitive the result is to W. Suppose a release drops an index and one of the three queries goes from 2 ms to 40 ms. The average hold time becomes about 14.7 ms, and busy connections jump to 3,600 × 0.0147, or about 53. That is more than the pool of 24, so requests start queueing at PgBouncer and response times climb. Doubling the pool wouldn't fix it; it would put around 50 slow queries on 16 cores at once. The fix is the index, which is why database indexing is a capacity tool as much as a speed tool.

The same arithmetic explains why autoscaling alone doesn't save a slow database. More pods raise the number of client connections, not the database's ability to finish queries. Our guide to handling 100,000 concurrent users covers the other layers that keep W small.

Load testing the result

1. Replay a realistic journey with a load-testing tool such as k6 or Locust: log in, start a test, autosave answers, submit. Run it at the forecast peak and at 1.5 times it.
   
   
2. Try three server-pool sizes, for example 16, 24 and 48, and compare throughput and 95th-percentile latency for each.
   
   
3. Watch the queue and the database together: cl_waiting and maxwait in PgBouncer's SHOW POOLS, active sessions in pg_stat_activity, database CPU, and busy app workers.
   
   
4. Pick the smallest pool that keeps latency within target with almost no clients waiting at the forecast peak.
   
   
5. Test again after big releases, because new queries change W, and W changes everything.
   
   

Key takeaways

• Little's Law: connections needed = transactions per second × time each connection is held.
  
  
• Past saturation, more connections make a database slower; queue in the pool, not in the database.
  
  
• Start the server-side pool near twice the database's cores, split across workloads, and load test.
  
  
• Size app workers and pods from requests in flight; app-side connections to a pooler are cheap.
  
  
• Slow queries raise W, and with it the connections you need; fix the query rather than growing the pool.
  
  

Frequently asked questions

What is Little's law in performance testing?

Little's Law, L = λW, links three numbers you see in every load test: concurrency (L), throughput (λ) and response time (W). If a test drives 500 requests a second with an average response time of 200 ms, about 100 requests are in flight at any moment. Testers use it to check whether a test's virtual users can generate the target load, and engineers use it to size worker pools and connection pools.

How many Gunicorn workers should I use?

Gunicorn's documentation suggests starting with (2 × CPU cores) + 1 worker processes and notes that 4 to 12 workers usually handle heavy traffic. For apps that spend much of each request waiting on the database or other services, the gthread worker with a few threads per process adds concurrency cheaply. Then check with Little's Law: slots across all servers should comfortably exceed requests per second × response time.

What is connection pool size?

Connection pool size is the maximum number of database connections a pool keeps open and lends out at once. When all are in use, further requests wait for a free one. Too small, and requests queue while the database sits idle; too large, and the database wastes effort juggling concurrent queries. A good size is close to the number of queries you need running at peak, which Little's Law estimates.