Introduction
HAProxy is the industry standard for high-performance load balancing and proxy. This comprehensive guide covers deployment, configuration, load balancing algorithms, health checking, SSL termination, rate limiting, and advanced features for achieving high availability and scalability.
HAProxy Architecture Overview
HAProxy operates in two modes: TCP (layer 4) for raw connections (MySQL, Redis, SMTP), or HTTP (layer 7) for HTTP(S) traffic with content switching. Single-threaded event-driven model: one process per CPU core. Core components: frontend (listener, entry point), backend (server pool), listen (frontend+backend combined). ACLs (Access Control Lists) for routing decisions. Maps for dynamic hostname routing. Runtime API for dynamic updates without reload. Max connections: ~200k per core. Configuration: /etc/haproxy/haproxy.cfg (global, defaults, frontend, backend sections).
Installation and Basic Configuration
Ubuntu/Debian: sudo apt install haproxy -y. RHEL/CentOS: sudo yum install haproxy -y. Enable: sudo systemctl enable haproxy, sudo systemctl start haproxy. Basic config: global section (daemon, maxconn, user/group). Defaults: mode http, log global, retries 3, timeout connect 5s, timeout client 50s, timeout server 50s. Simple frontend: frontend web_frontend, bind *:80, default_backend web_servers. Backend: backend web_servers, balance roundrobin, server web1 192.168.1.10:80 check, server web2 192.168.1.11:80 check. Check syntax: haproxy -f /etc/haproxy/haproxy.cfg -c. Reload without dropping connections: haproxy -f haproxy.cfg -sf $(pidof haproxy).
Load Balancing Algorithms
roundrobin: (default) distributes equally, supports weights, dynamic adjustments. leastconn: selects server with fewest connections (for long-lived sessions). first: fills server list sequentially (warming up caches). source: consistent hash on client IP (sticky sessions). uri: hash on request URI (cache friendly). url_param: hash on URL parameter (user ID). hdr(name): hash on HTTP header. random: random distribution (good for large server pools). Algorithm selection: roundrobin for short-lived HTTP, leastconn for API with keepalive, source/uri for caching consistency. Weight management: server web1 10.0.0.1:80 weight 3, server web2 10.0.0.2:80 weight 1 (3x traffic to web1).
Health Checks
Basic TCP check: check parameter. HTTP check: option httpchk GET /health, http-check expect status 200. Advanced: http-check send meth GET uri /health ver HTTP/1.0 hdr Host example.com, http-check expect rstatus ^(2|3)[0-9][0-9]. Inter/rise/fall: check inter 2s rise 2 fall 3 (check every 2s, declare UP after 2 successes, DOWN after 3 failures). MySQL check: option mysql-check user haproxy_check. Redis: option redis-check. Custom port: server web1 10.0.0.1:80 check port 8080 (health on different port). Agent check: server web1 10.0.0.1:80 check weight 100 agent-check agent-port 9999 (dynamic weight via external script). Observe mode: server web1 check observe layer4 (health from traffic patterns).
Session Persistence (Stickiness)
Cookie-based persistence: backend web, cookie SRVNAME insert indirect nocache, server web1 10.0.0.1:80 cookie web1, server web2 10.0.0.2:80 cookie web2. Client receives SRVNAME=web1 cookie. Source IP stickiness: stick-table type ip size 1m expire 30s, stick on src. App session: stick on hdr(Cookie) table app_session (not recommended, can cause hash imbalance). Insert vs rewrite vs prefix: insert (add cookie), rewrite (modify existing), prefix (add before existing). Learn from server: cookie LEARN insert nocache dynamic (extract from Set-Cookie header). Stick table size: 1m for 1M unique entries, memory ~20-30MB. Expire after session timeout.
SSL/TLS Termination
Terminate SSL at HAProxy: bind *:443 ssl crt /etc/haproxy/certs/example.com.pem. Combine certificate and key in one PEM file: cat example.com.crt example.com.key > example.com.pem. Multiple domains: crt /etc/haproxy/certs/ (directory with multiple PEM files). SNI support: bind *:443 ssl crt /etc/haproxy/certs/ alpn h2,http/1.1 (HTTP/2 support). Redirect HTTP to HTTPS: frontend http, bind *:80, redirect scheme https code 301 if !{ ssl_fc }. Client certificate validation: verify required, ca-file /etc/haproxy/ca.pem. Add headers for backend: http-request set-header X-Forwarded-Proto https, http-request set-header X-Forwarded-Ssl on. SSL performance: use ssl-engine (hardware offload), ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256, ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11. OCSP stapling: bind *:443 ssl crt ... ocsp-update on.
Access Control Lists (ACLs)
ACLs for content-based routing: acl is_api path_beg /api/, acl is_admin hdr(host) -i admin.example.com, acl is_static path_end .css .js .png, acl is_post method POST. Use in frontend: use_backend api_servers if is_api, use_backend admin_servers if is_admin, use_backend static_servers if is_static, default_backend app_servers. Conditions: if/unless, multiple conditions (and/or): if is_api is_post (both true), if is_api !is_post (API but not POST). Capture headers for logging: capture request header User-Agent len 80, capture response header Content-Type len 20. Regex performance: avoid complex regex in high-traffic paths. Use maps for large routing tables: acl is_domain hdr(host) -f /etc/haproxy/domains.map. use_backend %[hdr(host),lower,map_dom(/etc/haproxy/domains.map)].
Rate Limiting and DDoS Protection
Limit connection rate per IP: stick-table type ip size 1m expire 30s store conn_cur, conn_rate(10s). tcp-request connection reject if { src_conn_cur ge 10 }. Limit HTTP requests: stick-table type ip size 1m expire 1m store http_req_rate(10s). http-request track-sc0 src. http-request deny deny_status 429 if { sc0_http_req_rate gt 50 }. Burst handling: store http_req_rate(1s) as well, handle peaks. Whitelist: acl whitelist src -f /etc/haproxy/whitelist.lst, http-request allow if whitelist. Slowloris protection: timeout http-request 10s, timeout client 20s. Connection limits per server: server web1 10.0.0.1:80 maxconn 1000. Global connection limits: maxconn 20000 in global section.
Logging and Monitoring
Syslog configuration: log 127.0.0.1 local0, log 127.0.0.1 local1 notice. /etc/rsyslog.d/haproxy.conf: local0.* /var/log/haproxy.log. Log format: log-format %ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r. capture request header User-Agent length 256. CSV export: stats socket /run/haproxy/admin.sock mode 660 level admin. Prometheus exporter: haproxy_exporter (consumes stats socket). Statistics page: listen stats bind *:8404, stats enable, stats uri /haproxy, stats refresh 10s, stats auth admin:password. Active monitoring: use external tools (Zabbix, Nagios) to check /haproxy?stats. key metrics: current sessions, queued requests, server status, response times.
High Availability (Keepalived + HAProxy)
Run HAProxy on two nodes with Keepalived for failover. Master/Backup setup: keepalived.conf: vrrp_instance VI_1 { state MASTER, interface eth0, virtual_router_id 51, priority 101 (master) / 100 (backup), authentication { auth_type PASS, auth_pass secret }, virtual_ipaddress { 192.168.1.100/24 dev eth0 } }. Notify script: notify_master /etc/keepalived/master.sh (can start haproxy if stopped). Health check: track_script { chk_haproxy }. Script: /etc/keepalived/chk_haproxy.sh: if killall -0 haproxy; then exit 0; else exit 1; fi. Preempt mode: set nopreempt on backup to avoid flapping. VRRP multicast: disable if not on same L2 network (use unicast). Floating IP: application connects to VIP, HAProxy routes to real servers. Failover time ~3 seconds (3 heartbeats).
Resilience and Queuing
Queue management: maxconn parameter per server, queued requests wait for available connection. Full connection handling: option redispatch (send to another server if original is down). maxconn queue: default-server maxqueue 100 (limit queue length). Slow start after recovery: slowstart 30s (gradually increase traffic). On-marked-down shutdown-sessions: kill sessions when server marked down. Graceful shutdown: weight 0, then drain, then remove. retry-on: alloc-failure, conn-failure, empty-response, 5xx. retries 3. Option allbackups: use all backup servers when main ones down. Backup server: server web-backup 10.0.0.3:80 backup (only used when all primary down).
Content Switching and Rewriting
Modify request headers: http-request set-header X-Forwarded-For %[src], http-request add-header X-Client-ID %[req.hdr(X-Session-ID)]. Remove headers: http-request del-header X-Forwarded-For. URL rewriting: http-request set-path /newpath%[path], http-request set-uri /new%[path]. Redirect: http-request redirect location https://newexample.com%s if { hdr(host) -i old.com }. Response rewriting: http-response set-header X-Custom "Custom Value". Compress responses: compression algo gzip, compression type text/html text/css application/javascript. Add cache headers: http-response set-header Cache-Control "public, max-age=3600" if { status 200 }.
Real-World Examples
Microservices routing: frontend api, bind *:8080, use_backend %[path,word(2,/)]. Backend examples: backend users_service, server users 10.0.0.10:8081, backend orders_service, server orders 10.0.0.11:8082. WebSocket support: mode http, option http-server-close (not needed for WebSocket), timeout tunnel 1h (long connection allowed). gRPC: mode tcp (or http with h2), bind *:50051 proto h2, server grpc1 10.0.0.20:50051. Database load balancing: mode tcp, balance leastconn, server mysql1 10.0.0.30:3306 check, server mysql2 10.0.0.31:3306 check, backup: server mysql3 10.0.0.32:3306 backup.
Security Hardening
Hide version: option dontlognull (don't log empty connections), `http-request del-header X-Haproxy-Server`. Max headers: tune.maxhttpheader 16384. Request size limit: http-buffer-request, max-keep-alive-queue. Block dangerous methods: http-request deny if { method CONNECT } or { method TRACE }. Protection headers: http-response set-header X-Frame-Options SAMEORIGIN, http-response set-header X-XSS-Protection "1; mode=block". Response inspection: http-response deny if { res.hdr(Content-Type) -i image/svg+xml } (block SVG XSS). TLS security: ssl-min-ver TLSv1.2, ssl cipher suites ECDHE-ECDSA-AES128-GCM-SHA256. Admin socket security: bind to localhost only.
Performance Tuning
CPU pinning: nbproc 4 (deprecated), use cpu-map 1 0, cpu-map 2 1, cpu-map 3 2, cpu-map 4 3 (manual binding). Threading (recommended): nbthread 4, cpu-map auto:1/0-3. Backlog: tune.backlog 4096. Buffer size: tune.bufsize 16384 (increase for large headers), tune.buffers.limit (memory limit). Pipe size: tune.pipesize 1048576 (1MB for large file transfers). Zero-copy forwarding: tune.sndbuf 0, tune.rcvbuf 0 (default). HTTP multiplexing: option http-keep-alive (default in 2.0+). increased connection reuse: http-reuse always. SSL engine offloading: async engines, ssl-engine intel. Monitor with `haproxy -vv` for CPU features (AESNI acceleration).
Conclusion
HAProxy provides enterprise-grade load balancing. Start with simple round-robin and health checks, add persistence for stateful apps, implement SSL termination with Let's Encrypt, then rate limiting and DDoS protection. Use Keepalived for HA failover. Stats page and Prometheus exporter for monitoring. Practice configuration reloads without downtime using pid management.