Web Servers, Proxies and Load Balancers

Suraj Kumar

Suraj Kumar

Last updated November 16, 2026·22 min read

DevOps course · lesson 7 of 24
  1. 1What Is DevOps?
  2. 2A Language and a Service to Operate
  3. 3Linux and the Terminal
  4. 4Bash Scripting for Ops
  5. 5Git and Collaboration
  6. 6Networking for DevOps
  7. 7Web Servers, Proxies and Load Balancers
  8. 8Docker for DevOps
  9. 9Docker Compose and Multi-Service Apps
  10. 10Cloud Providers: AWS, Azure and Google Cloud
  11. 11Serverless Functions and Platforms
  12. 12Terraform and Infrastructure as Code
  13. 13Ansible and Configuration Management
  14. 14Continuous Integration with GitHub Actions and GitLab CI
  15. 15Continuous Delivery and Deployment Strategies
  16. 16Secret Management
  17. 17Kubernetes Basics
  18. 18Kubernetes in Practice
  19. 19GitOps with Argo CD and Flux
  20. 20Service Mesh: Istio, Linkerd, Consul and Envoy
  21. 21Metrics and Alerting with Prometheus and Grafana
  22. 22Logs and Tracing with Loki, Elastic and OpenTelemetry
  23. 23Reliability, SLOs and Cloud Design Patterns
  24. 24Capstone: Pebble to Production
View full syllabus

After this lesson, you will be able to explain the difference between a web server, a forward proxy, a reverse proxy, a load balancer and a caching server, configure Nginx to balance traffic across two copies of Pebble, see failover happen when one copy dies, cache responses, limit request rates and request sizes, terminate TLS with a free certificate, and choose sensibly between Nginx, Caddy, Apache, Tomcat and IIS.

At the end of Networking for DevOps, Pebble was running on a server, listening only on 127.0.0.1:8080, behind a firewall that allowed nothing but SSH. That is safe, but useless to anyone else. You could make Pebble listen on a public address and open its port, but Pebble's small Python server was never designed to face the internet: it has no HTTPS, no protection against slow or abusive clients, and only one copy runs, so a crash or a deployment means downtime. The standard answer is to put a purpose-built server in front of it.

Web servers and proxies

A web server accepts HTTP connections and answers them. It can serve files from disk directly, such as HTML, images and JavaScript, or pass requests on to an application. Nginx, Apache HTTP Server, Caddy and Microsoft IIS are web servers.

A proxy is a server that sits between a client and another server and passes traffic between them. Which side it works for makes all the difference:

Forward proxyReverse proxy
Works on behalf ofClientsServers
Clients know it existsYes, they are configured to use itUsually not; it looks like the site itself
Typical usesControl and log outbound web access in a company, reach the internet from a private network, cache downloadsHTTPS, load balancing, caching, rate limiting, hiding internal servers
ExampleA corporate proxy that all office browsers use, or SquidNginx in front of Pebble

A forward proxy works for clients such as office laptops and private servers reaching out to the internet, while a reverse proxy works for servers, receiving internet traffic and passing it to applications behind it.

You will meet forward proxies when servers in a private network must download packages through a controlled exit point, often configured with the HTTP_PROXY and HTTPS_PROXY environment variables. This lesson is mostly about reverse proxies, because that is what you build in front of your own services.

A load balancer is a reverse proxy that spreads requests across several copies of a service, so more copies can handle more traffic and one failure does not take the service down. A caching server keeps copies of responses and answers repeat requests itself, without bothering the application. In practice one Nginx process often does all three jobs, and cloud providers sell managed load balancers that do the same at a larger scale.

Pebble behind Nginx: clients reach Nginx on ports 80 and 443, which terminates TLS, applies rate and size limits, serves cached responses, and balances the rest across two Pebble instances on ports 8081 and 8082 that share one database.

Why Pebble needs a reverse proxy

A reverse proxy in front of Pebble gives you, in one place:

  • TLS termination. The proxy holds the certificate and handles HTTPS, then talks plain HTTP to Pebble on the same machine. The application never touches certificates.
  • Load balancing and failover. Several copies of Pebble can run, and the proxy routes around one that fails.
  • Zero-downtime deployments. Restart one copy at a time while the other keeps serving.
  • Protection. The proxy enforces request size limits, rate limits and timeouts, and absorbs slow clients, so Pebble only ever sees complete, reasonable requests.
  • Caching and compression. Repeat reads are answered from memory or disk.
  • One front door. Several applications can share ports 80 and 443 on one server, routed by host name or path.

Running two copies only works because Pebble follows the twelve-factor "stateless processes" principle from A Language and a Service to Operate: it keeps nothing important in memory, so any copy can answer any request. Both copies point at the same database file. SQLite handles several processes reading and writing one file for a service of Pebble's size; PostgreSQL takes over in the Docker Compose lesson.

Nginx in front of two Pebbles

Nginx (pronounced "engine x") is a fast, widely used web server and reverse proxy. At the time of writing, nginx.org lists 1.31.8 as the mainline version and 1.30.6 as the stable version; the configuration in this lesson was tested with nginx 1.27.5 and uses only long-established directives, so it works on both. Ubuntu's package is installed with sudo apt install nginx.

You can try everything in this section on a laptop before touching a server. Start two copies of Pebble on ports 8081 and 8082, sharing one database file, each in its own terminal:

bash
1PEBBLE_PORT=8081 PEBBLE_DB_PATH=./pebble.db python3 pebble.py
2PEBBLE_PORT=8082 PEBBLE_DB_PATH=./pebble.db python3 pebble.py

Here is the Nginx configuration for Pebble. Save it as pebble.conf:

nginx
1upstream pebble_app {
2    server 127.0.0.1:8081 max_fails=2 fail_timeout=10s;
3    server 127.0.0.1:8082 max_fails=2 fail_timeout=10s;
4}
5
6proxy_cache_path cache levels=1:2 keys_zone=pebble_cache:10m max_size=100m inactive=10m;
7limit_req_zone $binary_remote_addr zone=pebble_create:10m rate=5r/s;
8
9server {
10    listen 127.0.0.1:8080;
11    server_name localhost;
12
13    client_max_body_size 4k;
14
15    location / {
16        proxy_pass http://pebble_app;
17        proxy_set_header Host $host;
18        proxy_set_header X-Real-IP $remote_addr;
19        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
20        proxy_set_header X-Forwarded-Proto $scheme;
21        proxy_connect_timeout 2s;
22        proxy_read_timeout 10s;
23        proxy_next_upstream error timeout http_502;
24        add_header X-Upstream $upstream_addr always;
25    }
26
27    location = /api/links {
28        limit_req zone=pebble_create burst=5 nodelay;
29        limit_req_status 429;
30        proxy_pass http://pebble_app;
31        proxy_set_header Host $host;
32        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
33        add_header X-Upstream $upstream_addr always;
34    }
35
36    location /api/links/ {
37        proxy_pass http://pebble_app;
38        proxy_set_header Host $host;
39        proxy_cache pebble_cache;
40        proxy_cache_valid 200 30s;
41        add_header X-Cache-Status $upstream_cache_status always;
42        add_header X-Upstream $upstream_addr always;
43    }
44
45    location = /healthz {
46        access_log off;
47        proxy_pass http://pebble_app;
48    }
49}

Read it from the top:

  • The upstream block names a group of backend servers, pebble_app. With no other instruction, Nginx uses round robin: each new request goes to the next server in the list. max_fails=2 fail_timeout=10s is a passive health check: after two failed attempts to reach a server within 10 seconds, Nginx stops sending it traffic for 10 seconds, then tries it again with real requests.
  • proxy_cache_path sets up a cache on disk with 10 MB of keys in memory, and limit_req_zone creates a rate-limit counter per client IP address at five requests per second. Both live at the top level of the file, outside any server block.
  • The server block listens on 127.0.0.1:8080 for this local test; on a real server it listens on ports 80 and 443. client_max_body_size 4k rejects large request bodies at the proxy, matching Pebble's own 4,096-byte limit.
  • location / passes everything else to the upstream. The proxy_set_header lines tell Pebble the original host name, the client's real IP address and whether the client used HTTPS; without them, every request would appear to come from 127.0.0.1 over plain HTTP. The timeouts stop a stuck backend from holding connections forever. proxy_next_upstream retries a request on the other backend when the first cannot be reached, and the X-Upstream header reports which backend answered, which is handy for learning and debugging but something you would usually remove in production.
  • location = /api/links is an exact match for link creation and applies the rate limit. burst=5 nodelay allows a short burst of five extra requests at once; beyond that, clients get 429 Too Many Requests.
  • location /api/links/ caches successful link lookups for 30 seconds and reports HIT or MISS in an X-Cache-Status header.
  • location = /healthz turns off access logging for health checks, which would otherwise fill the log with noise.

Nginx needs a small main file around this. Ubuntu's package supplies one; for a laptop test, save this as nginx.conf in the same folder:

nginx
1worker_processes 1;
2error_log logs/error.log warn;
3pid logs/nginx.pid;
4events { worker_connections 256; }
5http {
6    access_log logs/access.log;
7    client_body_temp_path tmp/body;
8    proxy_temp_path tmp/proxy;
9    fastcgi_temp_path tmp/fastcgi;
10    uwsgi_temp_path tmp/uwsgi;
11    scgi_temp_path tmp/scgi;
12    include pebble.conf;
13}

Test the configuration, then start Nginx using the current folder as its home with -p. Always run nginx -t before starting or reloading; a typo then fails safely instead of taking the site down:

bash
1mkdir -p logs tmp cache
2nginx -p "$PWD/" -e logs/error.log -c nginx.conf -t
3nginx -p "$PWD/" -e logs/error.log -c nginx.conf
text
1nginx: the configuration file ./nginx.conf syntax is ok
2nginx: configuration file ./nginx.conf test is successful

Watch the load balancer work

Send four requests through Nginx and print the header that names the backend:

bash
1for i in 1 2 3 4; do curl -s -o /dev/null -D - http://localhost:8080/readyz | grep -i x-upstream; done
text
1X-Upstream: 127.0.0.1:8082
2X-Upstream: 127.0.0.1:8081
3X-Upstream: 127.0.0.1:8082
4X-Upstream: 127.0.0.1:8081

Requests alternate between the two copies. Create a link and follow it through the proxy:

bash
1curl -s http://localhost:8080/api/links -H 'Content-Type: application/json' \
2  -d '{"url": "https://nginx.org/en/docs/"}'
3curl -s -i http://localhost:8080/44sxkid

Use the code from your own response in the second command. The redirect now comes through Nginx:

text
1HTTP/1.1 302 Found
2Server: nginx/1.27.5
3Date: Fri, 13 Nov 2026 09:55:54 GMT
4Content-Length: 0
5Connection: keep-alive
6Location: https://nginx.org/en/docs/
7X-Upstream: 127.0.0.1:8081

The link was created by one copy and redirected by the other, which only works because they share the database.

Failover. Now simulate a crash: kill the Pebble on port 8082 with kill -9, as you did in Linux and the Terminal, and send four more requests:

text
1X-Upstream: 127.0.0.1:8082, 127.0.0.1:8081
2X-Upstream: 127.0.0.1:8081
3X-Upstream: 127.0.0.1:8082, 127.0.0.1:8081
4X-Upstream: 127.0.0.1:8081

Every request still returned 200. When a request was sent to 8082 and the connection was refused, Nginx retried it on 8081, so the header lists both. After the second failure, Nginx marked 8082 as unavailable, and its error log says so:

text
1[warn] ... upstream server temporarily disabled while connecting to upstream, client: 127.0.0.1, server: localhost, request: "GET /readyz HTTP/1.1", upstream: "http://127.0.0.1:8082/readyz", host: "localhost:8080"

The [error] lines just before it say connect() failed with "Connection refused". When both copies are down, there is nothing to retry and Nginx answers for them:

text
1HTTP/1.1 502 Bad Gateway
2Server: nginx/1.27.5

That is the 502 from the HTTP status table in the previous lesson: the proxy is fine, the application behind it is not. Start both copies again, wait for the 10-second fail_timeout to pass, and requests alternate between 8081 and 8082 once more.

Passive checks only notice a failure when real traffic hits it. Commercial Nginx Plus, HAProxy, Kubernetes and cloud load balancers add active health checks that call an endpoint such as Pebble's /readyz on a schedule and remove an instance before users are affected. That is exactly what the readiness endpoint is for.

Caching, rate limits and size limits

Look up the same link three times:

bash
1for i in 1 2 3; do curl -s -D - -o /dev/null http://localhost:8080/api/links/44sxkid | grep -iE 'x-cache|x-upstream'; done
text
1X-Cache-Status: MISS
2X-Upstream: 127.0.0.1:8082
3X-Cache-Status: HIT
4X-Cache-Status: HIT

The first request went to Pebble and the response was stored. The next two were answered by Nginx from its cache, which is why there is no X-Upstream header: no backend was contacted. For 30 seconds, Pebble does no work for repeat lookups. Caching is powerful and easy to get wrong. Only cache responses that are the same for every user, set short lifetimes until you understand the effects, and never cache anything personal or anything that changes on every request. Pebble's redirects are not cached here on purpose, because a link might be corrected or removed.

Now send twelve link-creation requests as fast as curl can:

bash
1for i in $(seq 1 12); do
2  curl -s -o /dev/null -w '%{http_code} ' http://localhost:8080/api/links \
3    -H 'Content-Type: application/json' -d '{"url":"https://example.com/'$i'"}'
4done; echo
text
1201 201 201 201 201 201 429 429 429 429 429 429

One request plus a burst of five got through; the rest were refused with 429 Too Many Requests and never reached Pebble. A request with a body larger than 4 KB gets 413 from Nginx, also without reaching Pebble. Limits like these turn a flood from a broken script or an abusive client into a few cheap rejections at the edge. Choose real limits from your own traffic data, not from this example.

Nginx's access log records each request in the standard "combined" format, including the rejected ones:

text
1127.0.0.1 - - [13/Nov/2026:15:26:06 +0530] "POST /api/links HTTP/1.1" 429 169 "-" "curl/8.7.1"
2127.0.0.1 - - [13/Nov/2026:15:26:06 +0530] "POST /api/links HTTP/1.1" 413 183 "-" "curl/8.7.1"

Stop Nginx with nginx -p "$PWD/" -c nginx.conf -s quit, which finishes active requests before exiting, and stop the two Pebbles.

Load balancing methods

Round robin is a good default when every request costs about the same. Nginx offers others:

MethodDirectiveHow it choosesGood for
Round robinnone, the defaultThe next server in turnSimilar servers, similar requests
Weightedserver ... weight=3;More requests to servers with a higher weightServers of different sizes, or sending a small share to a new version
Least connectionsleast_conn;The server with the fewest active connectionsRequests that vary a lot in duration
IP haship_hash;Always the same server for the same client IPOld applications that keep sessions in memory
Generic hashhash $request_uri consistent;The same server for the same key, such as a URLSpreading a cache across servers

Avoid needing sticky sessions in new applications. If a user must always return to the same server, that server's failure logs them out, and the load can never spread evenly. Stateless services, like Pebble, make every method work.

Run it on the server with HTTPS

On the Ubuntu server from earlier lessons, Pebble ran as one systemd service on port 8080. To run two copies, turn the unit into a template. Rename /etc/systemd/system/pebble.service to /etc/systemd/system/pebble@.service and add one line in the [Service] section, after EnvironmentFile=:

ini
1Environment=PEBBLE_PORT=%i

%i is replaced by whatever follows the @ in the unit name, so pebble@8081 runs on port 8081. Settings read from an EnvironmentFile= override Environment= lines, so also delete the PEBBLE_PORT line from /etc/pebble/pebble.env, and change PEBBLE_BASE_URL to your public address, such as https://pebble.example. Then switch over:

bash
1sudo systemctl disable --now pebble
2sudo systemctl daemon-reload
3sudo systemctl enable --now pebble@8081 pebble@8082
4systemctl status 'pebble@*' --no-pager

Install Nginx and add the site. On Ubuntu, site files go in /etc/nginx/sites-available/ and are switched on with a link from /etc/nginx/sites-enabled/. Copy pebble.conf there with three changes: listen 80; instead of the local address, server_name pebble.example; with your domain, and proxy_cache_path /var/cache/nginx/pebble ... for the cache location:

bash
1sudo apt install -y nginx
2sudo mkdir -p /var/cache/nginx/pebble
3sudo nano /etc/nginx/sites-available/pebble
4sudo ln -s /etc/nginx/sites-available/pebble /etc/nginx/sites-enabled/pebble
5sudo rm /etc/nginx/sites-enabled/default
6sudo nginx -t
7sudo systemctl reload nginx
8sudo ufw allow 'Nginx Full'

'Nginx Full' is a ufw profile, installed with the package, that opens ports 80 and 443. Pebble's ports 8081 and 8082 stay closed, because only Nginx on the same machine talks to them.

For HTTPS you need a domain name whose A record points at the server, as covered in the previous lesson. Certbot, from the Electronic Frontier Foundation, obtains a free certificate from Let's Encrypt using the ACME protocol, edits the Nginx configuration to use it, and sets up automatic renewal. Its official instructions install it as a snap package:

bash
1sudo snap install --classic certbot
2sudo ln -s /snap/bin/certbot /usr/local/bin/certbot
3sudo certbot --nginx -d pebble.example
4sudo certbot renew --dry-run

Certbot asks for an email address for expiry notices and whether to redirect HTTP to HTTPS; say yes to the redirect. renew --dry-run proves renewal will work without changing anything. With certificate lifetimes shrinking, as described in Networking for DevOps, that automatic renewal is not optional. Afterwards, curl -sI https://pebble.example/healthz should return 200 over HTTPS, and http:// requests should get a 301 redirect to https://.

Caddy: HTTPS by default

Caddy is a newer web server whose main attraction is automatic HTTPS. For any public domain in its configuration, Caddy obtains a certificate from an ACME certificate authority, Let's Encrypt or ZeroSSL by default, renews it, and redirects HTTP to HTTPS, with no extra tools. Requirements are the same as for Certbot: DNS pointing at the server and ports 80 and 443 reachable. The whole Pebble setup is a few lines in a file called Caddyfile:

text
1pebble.example {
2    reverse_proxy 127.0.0.1:8081 127.0.0.1:8082 {
3        lb_policy round_robin
4        health_uri /readyz
5        health_interval 10s
6    }
7}

health_uri adds active health checks against Pebble's readiness endpoint. Caddy is an excellent choice for small teams and new services. Nginx remains more common, has more documentation and third-party examples, and gives finer control for complex setups.

Apache, Tomcat and IIS

Apache HTTP Server, usually just "Apache" and packaged as apache2 on Debian and Ubuntu or httpd on RHEL, is one of the oldest and most widely deployed web servers; version 2.4.71 was released on 11 November 2026. It is configured with modules, and its reverse proxy and load balancer come from mod_proxy and mod_proxy_balancer. The equivalent of Pebble's upstream looks like this:

apache
1<Proxy "balancer://pebble">
2    BalancerMember "http://127.0.0.1:8081"
3    BalancerMember "http://127.0.0.1:8082"
4</Proxy>
5ProxyPass        "/" "balancer://pebble/"
6ProxyPassReverse "/" "balancer://pebble/"

Apache also allows per-folder configuration through .htaccess files, which shared hosting relies on. You will meet it on older servers and in many PHP applications.

Apache Tomcat is a different project: a Java application server that runs Java web applications packaged as WAR files. It is not a general web server; teams usually put Nginx or Apache in front of it, just as you put Nginx in front of Pebble. IIS (Internet Information Services) is Microsoft's web server for Windows Server, common for .NET applications and managed with IIS Manager or PowerShell. Its Application Request Routing extension adds reverse proxy and load balancing features.

ServerStrengthsTypical use
NginxFast, efficient, flexible reverse proxy, huge communityFront door for most Linux web services, including Pebble
CaddyAutomatic HTTPS, very short configurationSmall teams, new services, internal tools
Apache HTTP ServerMature, modular, .htaccess supportExisting deployments, PHP applications, shared hosting
Apache TomcatRuns Java servlet applicationsJava web apps, usually behind Nginx or Apache
IISIntegrated with Windows Server and .NETWindows-based organizations

Mistakes beginners make

  • Exposing the application's own port to the internet instead of putting a proxy in front.
  • Forgetting the proxy_set_header lines, so the application logs every request as coming from 127.0.0.1.
  • Reloading Nginx without running nginx -t first.
  • Running one copy of the service behind a load balancer and calling it highly available.
  • Keeping user sessions in application memory, then needing sticky sessions.
  • Caching responses that are personal or change often.
  • Setting rate limits without measuring normal traffic first.
  • Leaving proxy timeouts at their defaults for an application with very different needs.
  • Obtaining a certificate by hand and forgetting renewal.
  • Opening backend ports in the firewall "just to test" and never closing them.

Recap and try it yourself

A web server answers HTTP requests; a forward proxy acts for clients and a reverse proxy acts for servers. A load balancer is a reverse proxy that spreads traffic across copies of a service and routes around failures, and a caching server answers repeat requests itself. Nginx now sits in front of two stateless Pebble instances, balancing with round robin, retrying and temporarily removing a failed instance, caching link lookups, rejecting floods with 429 and oversized bodies with 413, and returning 502 only when no backend is left. On the server, a systemd template runs the two copies, Nginx owns ports 80 and 443, and Certbot or Caddy provides automatically renewed HTTPS. Apache, Tomcat and IIS fill the same roles in other environments.

Try it yourself:

  1. Run the local setup. Add weight=3 to the 8081 server, reload with nginx -s reload, send twenty requests and count how many each backend received.
  2. Switch the upstream to least_conn;. Then add a temporary time.sleep(2) to one copy's readiness endpoint and watch how least connections responds compared with round robin.
  3. Do a rolling restart on the server: sudo systemctl restart pebble@8081, wait for it to be ready, then restart pebble@8082, while a loop of curl requests runs in another terminal. Did any request fail?
  4. Change the cache lifetime to 5 seconds and confirm with X-Cache-Status that a lookup returns to MISS after it expires.

Sources checked on 13 November 2026: nginx.org downloads, HTTP load balancing, ngx_http_upstream_module, ngx_http_proxy_module and ngx_http_limit_req_module; Certbot's Nginx instructions; Caddy's automatic HTTPS and reverse_proxy documentation; the Apache HTTP Server project and its mod_proxy_balancer documentation; Apache Tomcat; Microsoft's IIS documentation; the systemd systemd.exec manual page; and the roadmap.sh DevOps roadmap. Pebble and its domain are fictional.

Related lessons in other courses