Resolve NGINX 502 Bad Gateway Reverse Proxy

Encountering an HTTP 502 Bad Gateway in production is one of the most common yet misunderstood server errors. The most important architectural reality to internalize is: NGINX itself is working.

A 502 error proves that NGINX received the incoming HTTP request, evaluated the server block, and attempted to hand it over to your backend process (Node.js, Python Gunicorn, Puma, Go, or PHP-FPM). The failure occurred because the backend application failed to respond, crashed mid-request, lacked UNIX socket permissions, or exceeded buffer limits. If your origin is fronted by an edge CDN, review our companion guide on fixing Cloudflare Error 522 Connection Timed Out to diagnose Layer 4 perimeter firewall drops.

This production troubleshooting guide details the exact diagnosis workflow, root-cause identification, and configuration fixes.

---

Technical Flow of an NGINX Reverse Proxy

[Client Browser] 
       │ HTTP GET /api/v1/stream
       ▼
[NGINX Edge Router (Port 80/443)]
       │ TLS Termination & Header Filtering
       │ (Attempts connection via UNIX Socket or TCP loopback)
       ▼
 [✖ Upstream Failure Occurs Here]
       ├── Scenario A: Upstream process dead / not listening (111: Connection refused)
       ├── Scenario B: UNIX Socket permission denied (13: Permission denied)
       ├── Scenario C: Header buffer overflow (upstream sent too big header)
       └── Scenario D: Request timeout (upstream timed out 110: Connection timed out)

For media streaming APIs and websocket servers, resolving latency bottlenecks also depends on using high-speed Anycast resolvers, as benchmarked in our DNS performance comparison for streaming.

---

Phase 1: Real-Time Log Inspection

Never guess the cause of a 502 error. NGINX records the exact reason in its error log. Run the following command while reproducing the error in your browser or via curl:

# Stream NGINX error log in real-time
sudo tail -f -n 50 /var/log/nginx/error.log

You will almost certainly see one of four specific signatures:

1. connect() failed (111: Connection refused) while connecting to upstream
2. connect() to unix:/run/app.sock failed (13: Permission denied)
3. upstream sent too big header while reading response header from upstream
4. upstream timed out (110: Connection timed out) while reading response header

Let us review the step-by-step fix for each scenario.

---

Scenario 1: connect() failed (111: Connection refused)

Root Cause

NGINX is trying to proxy to an IP and port (e.g., 127.0.0.1:3000), but the upstream service is either dead, in a crash-loop, or listening exclusively on IPv6 or an unexpected interface.

Step-by-Step Fix:

1. Verify if your backend process is actually active:

   # For systemd managed services:
   sudo systemctl status my-app.service
   
   # For PM2 (Node.js):
   pm2 status
   pm2 logs --lines 20
   

2. Verify which ports are currently listening:

   sudo ss -tulpn | grep LISTEN
   

3. Check localhost vs 127.0.0.1 binding: In modern Linux kernels, localhost may resolve to ::1 (IPv6). If your application binds to IPv4 (127.0.0.1), NGINX will fail if configured with proxy_pass http://localhost:3000;.

   Update your NGINX configuration:

   location / {
       # Use explicit IPv4 loopback rather than localhost
       proxy_pass http://127.0.0.1:3000;
       proxy_http_version 1.1;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection 'upgrade';
       proxy_set_header Host $host;
       proxy_cache_bypass $http_upgrade;
   }
   

---

Scenario 2: failed (13: Permission denied) while connecting to upstream socket

Root Cause

When communicating over UNIX domain sockets (common with PHP-FPM, uWSGI, and Puma), the user running NGINX (www-data or nginx) lacks read/write permissions for the .sock file.

Step-by-Step Fix:

1. Inspect socket file permissions:

   ls -la /run/php/php8.2-fpm.sock
   # Output might show: srw-rw---- 1 root root ... (Wrong owner!)
   

2. Update the upstream pool configuration (e.g., /etc/php/8.2/fpm/pool.d/www.conf):

   listen.owner = www-data
   listen.group = www-data
   listen.mode = 0660
   

3. Restart both services:

   sudo systemctl restart php8.2-fpm
   sudo systemctl restart nginx
   

SELinux Restriction (RHEL, AlmaLinux, Rocky Linux):

If running on RedHat-based systems with SELinux enforcing, NGINX is blocked from establishing network relay connections by default:

# Check if SELinux blocked NGINX:
sudo getsebool httpd_can_network_connect

# Enable NGINX network socket connectivity permanently:
sudo setsebool -P httpd_can_network_connect 1

---

Scenario 3: upstream sent too big header while reading response header

Root Cause

Modern web frameworks (OAuth2, NextAuth, heavy JWT cookies, or extensive session state) generate large Set-Cookie or authorization headers. NGINX reserves a default 4KB or 8KB buffer for upstream response headers. If headers exceed this, NGINX terminates the pipe and returns 502 Bad Gateway.

Step-by-Step Fix:

Add expanded proxy buffer directives to your nginx.conf within the http block or your specific server block:

server {
    listen 443 ssl http2;
    server_name api.bufferfixer.com;

    location / {
        proxy_pass http://127.0.0.1:8080;

        # Mitigate upstream sent too big header
        proxy_buffer_size          128k;
        proxy_buffers              4 256k;
        proxy_busy_buffers_size    256k;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Validate and reload configuration:

sudo nginx -t && sudo systemctl reload nginx

If your server also streams heavy multimedia files, review our guide on configuring Plex hardware acceleration and transcoder RAM buffers to prevent CPU starvation from crashing co-located NGINX workers.

---

Scenario 4: upstream timed out (110: Connection timed out)

Root Cause

The backend application received the request but took longer to process the database query, machine learning inference, or compute task than the default proxy_read_timeout (60 seconds).

Step-by-Step Fix:

Increase upstream timeout thresholds for long-running batch endpoints:

location /api/heavy-task/ {
    proxy_pass http://127.0.0.1:5000;
    
    proxy_connect_timeout       300s;
    proxy_send_timeout          300s;
    proxy_read_timeout          300s;
    send_timeout                300s;
}

---

Scenario 5: High-Concurrency Socket Depletion (TIME_WAIT Tuning)

Under intense traffic loads, Linux systems can exhaust ephemeral port ranges when proxying millions of rapid microservice requests. If your server encounters cannot assign requested address errors, apply these kernel-level TCP optimizations in /etc/sysctl.conf:

# Optimize Linux socket reuse for high-throughput reverse proxying
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535

Apply the changes immediately without rebooting:

sudo sysctl -p

---

Verification Test Script

Run this command directly on your server host to confirm whether the upstream is responding independently of NGINX:

# Direct upstream loopback test
curl -iv -H "Host: bufferfixer.com" http://127.0.0.1:3000/health

If this returns HTTP/1.1 200 OK, your upstream is alive and the error is strictly confined to the NGINX proxy block parameters. If it hangs or outputs Connection refused, troubleshoot your backend application's boot logs and runtime environment. For additional server management tutorials, explore our Servers & Hosting archive or submit an infrastructure ticket to our engineering team.