Opscraft
Nginx & reverse proxies · Lesson 3 of 6 · 5 min read

Reverse proxying and load balancing

proxy_pass and headers

location /api/ {
    proxy_pass http://api_backend;          # no URI part: path passed unchanged
    proxy_http_version 1.1;
    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

The trailing slash rule. If proxy_pass has no path (http://api_backend), the original URI is passed as is: /api/users stays /api/users. If it has any path, even just / (http://api_backend/), the part that matched the location is replaced by it: /api/users becomes /users. Many "404 from the backend" tickets come down to one slash.

Enlarge diagramproxy_pass http://app;proxy_pass http://app/;no URI partrequest /api/usersbackend sees /api/usersURI part "/"request /api/usersbackend sees /users (/api/ replaced)
Figure 1. One trailing slash in proxy_pass changes what path the backend receives.
Swipe sideways to see all of it, or tap to open it full screen.

Headers. By default Nginx sets Host to the upstream name, not the name the client asked for, so pass $host if the backend uses virtual hosting or builds URLs. The backend only sees Nginx's IP as the client, so pass X-Forwarded-For and X-Forwarded-Proto, and configure the app to trust them only from the proxy. Behind a load balancer, use the realip module (set_real_ip_from the balancer's range plus real_ip_header X-Forwarded-For;) so logs and rate limits use the real client address.

Upstreams and balancing methods

upstream api_backend {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.13:8080 backup;
    keepalive 32;               # idle connections kept open to upstreams
}
Method Behaviour Use when
round robin (default) Each server in turn, optional weight= Similar requests, similar servers
least_conn Server with the fewest active connections Requests of uneven length
ip_hash Same client IP → same server Simple stickiness (breaks behind NAT)
hash $key consistent Consistent hashing on any key Cache servers, sharding

Open-source Nginx uses passive health checks: after max_fails failures within fail_timeout, the server is skipped for fail_timeout. Active health checks (polling a /health URL) are an NGINX Plus feature; HAProxy and Envoy have them built in.

keepalive only works with proxy_http_version 1.1; and an empty Connection header (proxy_set_header Connection "";). Without it, Nginx opens a new TCP connection for every request, which wastes CPU and can exhaust ephemeral ports on busy proxies.

DNS names in upstream or proxy_pass are resolved once at startup. If the backend's IPs change (an AWS load balancer, a Kubernetes Service recreated), Nginx keeps sending to the old addresses until reload. Fix it with a resolver directive and a variable in proxy_pass, or with server ... resolve in newer versions.

Timeouts, buffering and WebSockets

Directive Default Meaning
proxy_connect_timeout 60 s Time to establish the TCP connection to the upstream
proxy_read_timeout 60 s Max gap between two reads from the upstream (not the whole response)
proxy_send_timeout 60 s Max gap between two writes to the upstream
client_body_timeout / client_header_timeout 60 s Slow clients
keepalive_timeout 75 s Idle client connections
client_max_body_size 1 MB Larger uploads get 413 Request Entity Too Large

Set timeouts deliberately in a chain: the load balancer in front should wait longer than Nginx, and Nginx longer than the app, so the layer closest to the problem times out first and logs the reason.

Enlarge diagramload balanceridle timeout 120 sNginxproxy_read_timeout 90sapprequest timeout 60 sdatabasestatementtimeout 30 s
Figure 2. Set timeouts as a chain: each layer waits a little longer than the one behind it, so the layer nearest the problem times out first.
Swipe sideways to see all of it, or tap to open it full screen.

Buffering. With proxy_buffering on (default), Nginx reads the upstream response as fast as it can and feeds slow clients itself, freeing the backend quickly. Turn it off for streaming responses (server-sent events, LLM token streams) with proxy_buffering off; or the X-Accel-Buffering: no response header.

WebSockets need the upgrade headers passed explicitly and a long read timeout:

location /ws/ {
    proxy_pass http://chat;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 1h;
}