Contents
HAProxy Configuration Guide
HAProxy has a lot of special configuration options that can be hard to understand. In this guide we’ll explain how to use some of these strange config directives. We’ll get you up and running with a reliable, monitored load balancer.
Where is the HAProxy Configuration File Located
Your HAProxy config file will be in located at /etc/haproxy/haproxy.cfg or /usr/local/etc/haproxy/haproxy.cnf on most linux distributions.
HAProxy ACL
HAProxy uses ACLs (Access Control Lists) to control how client requests are routed. You can think of ACLs as a named rule that’s evaluated for every request (e.g. is_static_file). You can then use those ACLs as if statements to control how the request is routed within HAProxy.
Defining an ACL
acl <name> <request-part> <flags> <op> <arg>
The name section is what we’re going to to call the ACL.
The fetch tells HAProxy what piece of the client request or context we’d like to evaluate. Common fetches include hdr(user-agent) to get the user agent, path_beg to match the start of a path, path_end to match the end of a path.
Here are a list of common fetches (see docs for full list)
- req_ver
- method
- path: exact string match
- path_beg: prefix match
- path_dir: sub directory match
- path_dom: domain match
- path_end: suffix match
- path_len: length match
- path_reg: regular expression match
- path_sub: substring match
- query
- hdr(name): extracts an http header and matches the value (exact match)
- hdr_beg(name): prefix match
- hdr_dir(name): sub directory match
- hdr_dom(name): domain match
- hdr_end(name): suffix match
- hdr_len(name): length match
- hdr_reg(name): regular expression match
- hdr_sub(name): substring match
- src: client ip address
Flags tell HAProxy any special instructions when performing the evaluation. You can pass “-i” to ignore case and “-m” to match a specific pattern. There more flags in the official docs.
Common ACLS
Here are a few common ACLs to get you started.
# is the request for a static file
acl is_static path_beg /static /images /js /css
# is the request an image
acl is_img path_end .jpg .gif .png
# is www (case insensitive)
acl is_www hdr_beg(host) -i www
Using ACL Results for Routing
Once you’ve created an ACL, you can use it as a boolean operator to control request routing. Control statements, like http-request deny, operate on a first match basis. Some common control statements include use_backend, http-request deny. Here are a few examples:
ACL Use_Backend Example
acl is_websocket path_beg -i /ws/
use_backend backend_websocket if is_websocket
default_backend backend_standard
This example creates an is_websocket ACL by checking that the path starts with /ws/ (case insensitive). If is_websocket is true, the websocket backend will be used. Otherwise the standard backend will be used.
Combine ACL Example
acl ua_is_bot hdr_sub(user-agent) -i Bot
acl path_is_weird path_sub -i zzzzzzzzz
http-request deny if ua_is_bot || path_is_weird
In this example, we’re creating two ACLs. One checks the user-agent for “Bot”. The other checks the path for a string of z’s. We can combine those ACLs in our control statement by using the OR operator.
SSL Passthrough
You can pass your SSL traffic through HAProxy without decrypting it. SSL pass through allows you to move the decryption load to backend servers. This can be really useful if you have a high traffic domain or a smaller CPU load balancer. Here’s a simple example of SSL pass through:
frontend ft_passthrough
bind *:443
mode tcp
default_backend bk_passthrough
backend bk_passthrough
balance leastconn
server bk_1 10.0.0.10
server bk_2 10.0.0.11
server bk_3 10.0.0.12
SSL Passthrough with Stick Sessions
Sticky sessions work in pass through mode too. Sticky sessions force client traffic to connect to the same backend server each time. This is especially useful for backend services that can’t share session information between backend servers. Here’s an example of a ssl passthrough with sticky sessions:
frontend ft_passthrough
bind *:443
mode tcp
default_backend bk_passthrough
backend bk_passthrough
balance leastconn
stick-table type ip size 1m expire 1h
stick on src
server bk_1 10.0.0.10:443
server bk_2 10.0.0.11:443
server bk_3 10.0.0.12:443
This example will use a table of Client IP’s to track which backend to stick to. The table entry will expire 1hr after the last request from that client. The table is capped at 1M records (so we don’t use up all our memory).
For more examples of sticky sessions, you may find this article helpful.
SSL Passthrough with Multiple Domains (SNI)
The problem with tcp mode is that we don’t have access to the routing attributes we used in http mode like the domain name. We do this on purpose because we don’t want to decrypt the entire HTTPS stream, or parse the HTTP request. But, we can actually extract the domain name from the HTTPS stream without decrypting or parsing the entire request. We can do this by inspecting the SNI section of the HTTPS setup request before we forward the TCP session to a backend.
Here’s how we can set up domain routing without HTTP mode or full SSL decryption. We wait for the HTTPS setup request and inspect it before forwarding the tcp connection to the backend. We can use tcp-request inspect-delay and ssl_fc_sni to achieve this. Here’s an example:
frontend ft_passthrough_inspect
bind :443 ssl crt cert.pem
mode tcp
tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }
use_backend bk_cats_dot_com if { ssl_fc_sni -i cats.com }
use_backend bk_dogs_dot_com if { ssl_fc_sni -i dogs.com }
default_backend bk_default
backend bk_cats_dot_com
mode tcp
balance roundrobin
server bk_1 10.0.0.1:443
server bk_2 10.0.0.2:443
UDP Load Balancing
Websockets
You can load balance your websocket connections through HAProxy too. We need to configure HAProxy to detect which HTTP requests are websockets and route our traffic accordingly.
To detect a websocket request, we can make use of the hdr ACLs. Here’s an example:
frontend ft_web
listen :443
mode http
acl has_upgrade hdr(Connection) -i upgrade
acl has_ws_upgrade hdr(Upgrade) -i websocket
use_backend bk_websocket if has_upgrade has_ws_upgrade
#...
In our backend, we may want to perform some additional validation before forwarding the websocket to our backend. We can check for Sec-Websocket-Key and Sec-Websocket-Version. (mdn header documentation). Here’s an example:
backend bk_websocket
balance leastconn
## websocket protocol validation
acl hdr_connection_upgrade hdr(Connection) -i upgrade
acl hdr_upgrade_websocket hdr(Upgrade) -i websocket
acl hdr_websocket_key hdr_cnt(Sec-WebSocket-Key) eq 1
acl hdr_websocket_version hdr_cnt(Sec-WebSocket-Version) eq 1
http-request deny if ! hdr_connection_upgrade ! hdr_upgrade_websocket ! hdr_websocket_key ! hdr_websocket_version
server websrv1 10.0.0.2:8080 maxconn 30000
server websrv1 10.0.0.3:8080 maxconn 30000
#...
Timeout
The timeout directives help you tune the performance of HAProxy. Slow connecting clients, or long connections can bog things down. Setting some reasonable timeouts will help HAProxy shed slow load. But, be careful. Don’t set your timeouts too small or clients won’t be able to complete reasonable requests.
There are a few timeout parameters to look at: connect, client and server.
- Connect timeout specifies how much time HAProxy can spend setting up the TCP connection with your backend server. We recommend setting this to 10s.
- Client timeout specifies how much of a gap there can be when waiting for a client to send data. Gaps in client data can be caused by network congestion, slow reading of an upload file etc. We recommend setting this to 30s.
- Server timeout specifies how much of a gap there can be when waiting for a server to send data. This can be caused by server processing. Note, that for tcp mode server timeout and client timeout should be the same. We recommend setting this to 240s. If you have long server processes, we recommend tuning those processes before decreasing this value.
If we put all that together, here’s an example of what our configuration file would look like:
defaults
timeout connect 10s
timeout client 30s
timeout server 240s
.
X Forwarded For (Setting Client IP in header)
Backend servers that receive traffic from HAProxy can no longer rely on request.ClientIP to tell them the user’s IP address. This is a big problem especially for logging and identification functions. We can get HAProxy to attach the user’s IP to the request we forward to the backend using option forwardfor. This will modify the HTTP request we send to the backend to include the X-Forwarded-For header. Here’s an example of what that would look like:
frontend ft_web
bind :443
mode http
option forwardedfor except 10.0.0.0/8
#
The except allows us to only include forwarding details for client requests. You can also use an ACL here to control when the header is added.
GRPC Load Balancing
Since GRPC is based on TCP, we can load balance GRPC traffic as well. We can configure HAProxy to listen for our TCP traffic and balance our sessions across multiple backends. HAProxy won’t inspect or decrypt your GRPC payload, it will simply route it to one of your backends. It’s as simple as creating a frontend and backend in tcp mode. Let’s look at a few examples:
frontend ft_grpc
bind :443
mode tcp
timeout client 120s
timeout server 120s
timeout connect 10s
default_backend bk_grpc
backend bk_grpc
mode tcp
balance leastconn
server bk_1 10.0.0.1:443
server bk_2 10.0.0.2:443
In this example, HAProxy will route our request to the backend with the last number of connections. Connections that are inactive for 120s or more will be automatically closed.