Ready to know about downtime before your customers?
Status List delivers uptime monitoring and professional hosted status pages for sites of all shapes and sizes.
Trusted by 1000+ companies
Contents
HAProxy Stats Socket Guide
The stats socket gives users a way to control and query a running HAProxy instance. This is a great tool wether you want to write a custom app integration or just send over a few commands on the cli. Most people use the stats socket for uptime and monitoring integrations.
Enabling Stats Socket
It’s pretty straightforward to turn on the HAProxy stats socket. Edit your haproxy.cfg file to include the following lines in the global section.
global
# stats socket <bind> level <admin|operator|user>
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 2m
Bind Parameter
The bind parameter tells HAProxy what type of socket to create. You can create a unix socket by passing a file path and a mode parameter. The mode parameter only applies to unix sockets and gives the file permissions to set. You can also create a TCP socket by setting the bind parameter in the format ipv#@ip:port (e.g. ipv4@10.0.0.1:8081).
Level Parameter
The level parameter tells HAProxy what commands to enable for this integration. The admin value gives full access to all socket commands. operator gives access to standard commands (e.g. resetting counters), but not configuration changes. user can’t make changes, it can only view information.
Putting it together
You can set multiple stats directives in the global configuration to create multiple sockets.
Once you’ve updated haproxy.cfg, don’t forget to restart the HAProxy service.
Non-Interactive Mode
This is the most common method used by uptime and monitoring applications. In this mode, HAProxy accepts a single line of commands and returns a single block of results before disconnecting from the socket. Non-interactive mode is turned on by default. You don’t need to do anything special to turn it on.
Note: We’ll be using socat which you can install with your OS package manager.
Command Structure
Non-interactive socket commands are formatted as a series of commands separated by semicolons ending in a new line. When we send multiple commands, HAProxy will provide the output separated by a blank line.
# single command
echo "show info" | socat /var/run/haproxy.sock stdio
# output to stdout
# 2 commands in one shot
echo "show info;show stat" | socat /var/run/haproxy.sock stdio
# output to stdout
Common Command Examples
Show All Metrics
There are a few commands that show us information about HAProxy’s internal status. The show info command gives overview status information. show stat gives us frontend/backend uptime details. Finally, show table gives us sticky table debug information.
echo "show info;show stat;show table" | socat /var/run/haproxy.sock stdio
Name: HAProxy
Version: 1.8.23
Release_date: 2019/11/25
Nbthread: 1
Nbproc: 1
Process_num: 1
Pid: 591804
Uptime: 0d 0h09m47s
Uptime_sec: 587
Memmax_MB: 0
PoolAlloc_MB: 0
PoolUsed_MB: 0
PoolFailed: 0
Ulimit-n: 8042
Maxsock: 8042
Maxconn: 4000
Hard_maxconn: 4000
CurrConns: 6
CumConns: 350
CumReq: 795
MaxSslConns: 0
CurrSslConns: 6
CumSslConns: 343
Maxpipes: 0
PipesUsed: 0
PipesFree: 0
ConnRate: 1
ConnRateLimit: 0
MaxConnRate: 7
SessRate: 1
SessRateLimit: 0
MaxSessRate: 7
SslRate: 1
SslRateLimit: 0
MaxSslRate: 7
SslFrontendKeyRate: 0
SslFrontendMaxKeyRate: 3
SslFrontendSessionReuse_pct: 100
SslBackendKeyRate: 0
SslBackendMaxKeyRate: 0
SslCacheLookups: 304
SslCacheMisses: 7
CompressBpsIn: 0
CompressBpsOut: 0
CompressBpsRateLim: 0
ZlibMemUsage: 0
MaxZlibMemUsage: 0
Tasks: 26
Run_queue: 0
Idle_pct: 100
node: my-load-balancer
Stopping: 0
Jobs: 11
Listeners: 4
# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,agent_status,agent_code,agent_duration,check_desc,agent_desc,check_rise,check_fall,check_health,agent_rise,agent_fall,agent_health,addr,cookie,mode,algo,conn_rate,conn_rate_max,conn_tot,intercepted,dcon,dses,
ft_web,FRONTEND,,,0,1,60000,1,271,114,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,1,,,,0,0,1,0,0,0,,0,1,1,,,0,0,0,0,,,,,,,,,,,,,,,,,,,,,http,,0,1,1,0,0,0,
ft_web_https,FRONTEND,,,6,12,60000,343,1404654,3291032,0,0,18,,,,,OPEN,,,,,,,,,1,3,0,,,,0,1,0,7,,,,19,248,9,218,1,0,,2,10,500,,,2299641,488132,0,232,,,,,,,,,,,,,,,,,,,,,http,,1,7,343,0,0,0,
stats,FRONTEND,,,0,1,2000,1,103,3804,0,0,0,,,,,OPEN,,,,,,,,,1,4,0,,,,0,0,0,1,,,,0,1,0,0,0,0,,0,1,1,,,0,0,0,0,,,,,,,,,,,,,,,,,,,,,http,,0,1,1,1,0,0,
bk_wordpress,wp0,0,0,0,1,30000,14,4958,25419,,0,,0,0,0,0,no check,10,1,0,,,587,,,1,5,1,,14,,2,0,,2,,,,0,10,4,0,0,0,,,,,0,0,,,,,25,,,0,1,3,8,,,,,,,,,,,,10.136.0.4:80,,http,,,,,,,,
bk_wordpress,BACKEND,0,0,0,1,6000,14,4958,25419,0,0,,0,0,0,0,UP,10,1,0,,0,587,0,,1,5,0,,14,,1,0,,2,,,,0,10,4,0,0,0,,,,14,0,0,0,0,0,0,25,,,0,1,3,8,,,,,,,,,,,,,,http,roundrobin,,,,,,,
bk_github,gh0,0,0,0,0,30000,0,0,0,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,6,1,,0,,2,0,,0,L4OK,,0,0,0,0,0,0,0,,,,,0,0,,,,,-1,,,0,0,0,0,,,,Layer4 check passed,,2,3,4,,,,127.0.0.1:8081,,http,,,,,,,,
bk_github,BACKEND,0,0,0,0,6000,0,0,0,0,0,,0,0,0,0,UP,10,1,0,,0,587,0,,1,6,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,0,0,0,0,0,0,0,-1,,,0,0,0,0,,,,,,,,,,,,,,http,roundrobin,,,,,,,
bk_conv,websrv1,0,0,0,0,30000,0,0,0,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,7,1,,0,,2,0,,0,L4OK,,1,0,0,0,0,0,0,,,,,0,0,,,,,-1,,,0,0,0,0,,,,Layer4 check passed,,2,3,4,,,,10.136.0.2:8080,,http,,,,,,,,
bk_conv,BACKEND,0,0,0,0,6000,0,0,0,0,0,,0,0,0,0,UP,10,1,0,,0,587,0,,1,7,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,0,0,0,0,0,0,0,-1,,,0,0,0,0,,,,,,,,,,,,,,http,roundrobin,,,,,,,
bk_redirect,BACKEND,0,0,0,1,6000,1,271,114,0,0,,0,0,0,0,UP,0,0,0,,0,587,,,1,8,0,,0,,1,0,,1,,,,0,0,1,0,0,0,,,,1,0,0,0,0,0,0,-1,,,0,0,0,0,,,,,,,,,,,,,,http,roundrobin,,,,,,,
bk_letsencrypt,letsencrypt,0,0,0,0,,0,0,0,,0,,0,0,0,0,no check,1,1,0,,,587,,,1,9,1,,0,,2,0,,0,,,,0,0,0,0,0,0,,,,,0,0,,,,,-1,,,0,0,0,0,,,,,,,,,,,,127.0.0.1:8888,,http,,,,,,,,
bk_letsencrypt,BACKEND,0,0,0,0,6000,0,0,0,0,0,,0,0,0,0,UP,1,1,0,,0,587,0,,1,9,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,0,0,0,0,0,0,0,-1,,,0,0,0,0,,,,,,,,,,,,,,http,roundrobin,,,,,,,
bk_web,websrv1,0,0,0,2,1000,207,750248,955237,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,10,1,,207,,2,1,,4,L7OK,200,2,0,119,3,84,1,0,,,,,0,0,,,,,0,OK,,0,1,20,178,,,,Layer7 check passed,,2,3,4,,,,10.136.0.2:8080,,http,,,,,,,,
bk_web,websrv2,0,0,0,2,1000,199,588608,2290567,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,10,2,,199,,2,1,,5,L7OK,200,1,0,119,2,78,0,0,,,,,1,0,,,,,0,OK,,0,1,18,95,,,,Layer7 check passed,,2,3,4,,,,10.136.0.3:8080,,http,,,,,,,,
bk_web,BACKEND,0,0,0,3,6000,406,1338856,3245804,0,0,,0,0,0,0,UP,20,2,0,,0,587,0,,1,10,0,,406,,1,2,,9,,,,0,238,5,162,1,0,,,,406,1,0,2299641,488132,0,232,0,,,0,1,29,235,,,,,,,,,,,,,,http,leastconn,,,,,,,
bk_ws,websrv1,0,0,2,3,30000,42,42267,10332,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,11,1,,42,,2,0,,1,L4OK,,0,19,0,0,23,0,0,,,,,0,0,,,,,6,,,0,1,1,1477,,,,Layer4 check passed,,2,3,4,,,,10.136.0.2:8080,,http,,,,,,,,
bk_ws,websrv2,0,0,3,3,30000,20,18573,5961,,0,,0,0,0,0,UP,10,1,0,0,0,587,0,,1,11,2,,20,,2,0,,1,L4OK,,0,5,0,0,15,0,0,,,,,0,0,,,,,201,,,0,1,1,585,,,,Layer4 check passed,,2,3,4,,,,10.136.0.3:8080,,http,,,,,,,,
bk_ws,BACKEND,0,0,5,6,6000,62,60840,16293,0,0,,0,0,0,0,UP,20,2,0,,0,587,0,,1,11,0,,62,,1,0,,1,,,,19,0,0,38,0,0,,,,57,0,0,0,0,0,0,6,,,0,1,1,2031,,,,,,,,,,,,,,http,leastconn,,,,,,,
Using Show All Output
As you can see, we have ALL the metrics we could need to monitor this instance in great detail. There are a few things to note in this output.
In the first section, we get overall service status. Things like uptime, memory usage and queue sizes. Those are great things to watch to see if our load balancer is having a hard time keeping up.
The second section is a CSV of front ends and backends. The front ends tell us about client request behaviour. The backends tell us about backend performance. This is useful to really dig into an application or user issue.
To make this data useful, we could really wire it up to a reporting dashboard or reply on the built-in HAProxy Stats Page.
Reset Stat Counters
If we want our “show all metrics” commands to be helpful, we need to reset the counters. We can do that with the clear counters all command. Here’s an example of how we can do that:
echo "clear counters all" | socat /var/run/haproxy.sock stdio
Turn on/off backend server
During an incident, we may want to disable or reenable a specific backend server. We can temporarily do that with the stats socket’s disable server and enable server commands. Here’s an example:
echo "disable server bk_mybackend/srv_myserver" | socat /var/run/haproxy.sock stdio
More Commands
There are a lot more commands available. You can read more at https://docs.haproxy.org/2.4/management.html
Interactive Mode
In this mode, we have access to all the same commands as non-interactive, but the socket dialog is a bit different. Instead of processing a single line of semicolon separate commands, HAProxy takes one command at a time and waits for the user.
In interactive mode, HAProxy sends a “>” character and waits for us to enter a command line. After we submit the line, HAProxy sends back the result and waits for a new command.
This mode may be useful for an application integration where opening a new connection for every command is too time consuming.
Here is a command line example of an interactive mode
socat /var/run/haproxy.sock readline
> show info
Name: HAProxy
Version: 1.8.23
Release_date: 2019/11/25
Nbthread: 1
Nbproc: 1
Process_num: 1
Pid: 591804
Uptime: 0d 0h33m18s
Uptime_sec: 1998
Memmax_MB: 0
PoolAlloc_MB: 0
PoolUsed_MB: 0
PoolFailed: 0
Ulimit-n: 8042
Maxsock: 8042
Maxconn: 4000
Hard_maxconn: 4000
CurrConns: 7
CumConns: 1118
CumReq: 2644
MaxSslConns: 0
CurrSslConns: 7
CumSslConns: 1102
Maxpipes: 0
PipesUsed: 0
PipesFree: 0
ConnRate: 0
ConnRateLimit: 0
MaxConnRate: 5
SessRate: 0
SessRateLimit: 0
MaxSessRate: 5
SslRate: 0
SslRateLimit: 0
MaxSslRate: 5
SslFrontendKeyRate: 0
SslFrontendMaxKeyRate: 2
SslFrontendSessionReuse_pct: 0
SslBackendKeyRate: 0
SslBackendMaxKeyRate: 0
SslCacheLookups: 1002
SslCacheMisses: 7
CompressBpsIn: 0
CompressBpsOut: 0
CompressBpsRateLim: 0
ZlibMemUsage: 0
MaxZlibMemUsage: 0
Tasks: 27
Run_queue: 0
Idle_pct: 100
node: regal-load-balancer
Stopping: 0
Jobs: 12
Listeners: 4
> quit
Round up
HAProxy’s stats socket is really helpful for uptime details and incident management. You can integrate the socket into your application, or just run it from the command line. If you enjoyed this article, you may also like HAProxy Stats Page Setup Guide.