SKUDONET comes with basic health checks for backends, but these checks may not be enough to ensure that the backends are working properly. To address this, SKUDONET implements a service called Farmguardian, which executes and manages advanced health checks.

Farmguardian acts as an advanced monitoring tool for backends. It does this by reading configurations and getting a list of backends from plugins that check the health status of each backend. The farmguardian then updates the backend status for each farm, determining whether the load balancer should send traffic to that backend.

This section explains how to configure Farmguardian checks using both global settings and settings specific to a farm.

Global Settings

The Global tab contains the following settings for Farmguardian:

Name: The name of the Farmguardian check. This cannot be edited.
Command: The command that is executed to check the status of the backends.
Timeout: The maximum amount of time that the backend has to respond.
Interval: The time between batches of health checks against all the backends. Note that after each interval, the Farmguardian check is stopped and relaunched, so the interval must be long enough for all the backends to be checked (timeout * number of backends + 1).
Description: A brief description of the Farmguardian check.
Cut Connections: When this option is enabled, the current connections of the detected backends that are down are flushed, forcing an immediate reconnection to available backends. If disabled, the current connections will be drained without disconnecting any client.
Enable logs: Enables or disables logging for the Farmguardian check. When logging is disabled, only backend status changes are shown in the log files.
Backend alias: A name that easily identifies the interface of the backend.

Configure health checks

All Farmguardian plugins are located in the directory /usr/local/skudonet/app/libexec/ on the load balancer.

Farmguardian uses plugins to configure advanced health checks that can detect whether a real server is working as expected using customized options. There are many health checks available for each protocol, service, and application.

The most important plugins are described below.

check_ftp: Tests whether a server is accepting FTP connections.

Usage:

check_ftp -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_fping: This plugin uses the fping command to quickly ping the specified host.

Usage:

 check_fping  -w limit -c limit [-b size] [-n number] [-T number] [-i number]

check_http: This plugin checks whether the HTTP service is running on the specified host. It can test both regular HTTP and secure HTTPS connections, follow redirects, search for specific text or patterns, and report on connection times, certificate expiration times, and HTTP status codes.

Usage:

 check_http -H  | -I  [-u ] [-p ]
       [-J ] [-K ]
       [-w ] [-c ] [-t ] [-L] [-E] [-a auth]
       [-b proxy_auth] [-f <ok|warning|critcal|follow|sticky|stickyport>]
       [-e ] [-d string] [-s string] [-l] [-r  | -R ]
       [-P string] [-m :] [-4|-6] [-N] [-M ]
       [-A string] [-k string] [-S ] [--sni] [-C [,]]
       [-T ] [-j method]

check_imap: This plugin checks whether an IMAP server is accepting connections on the specified host.

Usage:
check_imap -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_ldaps: This plugin performs a health check of an LDAPS service on the specified host by sending a given search query.

Usage:
 check_ldaps -H  -b  [-p ] [-a ] [-D ]
       [-P ] [-w ] [-c ] [-t timeout]
       [-2|-3] [-4|-6]

check_mysql: This plugin checks whether a MySQL server is accessible and can accept connections.

Usage:
 check_mysql [-d database] [-H host] [-P port] [-s socket]
       [-u user] [-p password] [-S] [-l] [-a cert] [-k key]
       [-C ca-cert] [-D ca-dir] [-L ciphers] [-f optfile] [-g group]

check_mysql_query: This plugin assesses the result of a query against predefined benchmarks.

Usage:
 
check_mysql_query -q SQL_query [-w warn] [-c crit] [-H host] [-P port] [-s socket]
       [-d database] [-u user] [-p password] [-f optfile] [-g group]

check_pgsql: Checks if a PostgreSQL database is running and can accept connections.

Usage:
check_pgsql [-H ] [-P ] [-c ] [-w ]
 [-t ] [-d ] [-l ] [-p ]
[-q ] [-C ] [-W ]

check_pop: This plugin checks if a POP server is running and can accept connections on the specified host.

Usage:
check_pop -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_radius: Checks whether a RADIUS server is running and accepting connections.

Usage:
check_radius -H host -F config_file -u username -p password
			[-P port] [-t timeout] [-r retries] [-e expect]
			[-n nas-id] [-N nas-ip-addr]

check_simap: This plugin checks whether a secure IMAP server is running and can accept connections on the specified host.

Usage:
check_simap -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_smtp: This plugin will check if the SMTP server on the specified host is running and accepting connections.

Usage:
check_smtp -H host [-p port] [-4|-6] [-e expect] [-C command] [-R response] [-f from addr]
[-A authtype -U authuser -P authpass] [-w warn] [-c crit] [-t timeout] [-q]
[-F fqdn] [-S] [-D warn days cert expire[,crit days cert expire]] [-v] 

check_snmp: Monitors the health and performance of remote machines and gather system data using the SNMP protocol.

Usage:
check_snmp -H  -o  [-w warn_range] [-c crit_range]
[-C community] [-s string] [-r regex] [-R regexi] [-t timeout] [-e retries]
[-l label] [-u units] [-p port-number] [-d delimiter] [-D output-delimiter]
[-m miblist] [-P snmp version] [-N context] [-L seclevel] [-U secname]
[-a authproto] [-A authpasswd] [-x privproto] [-X privpasswd] [-4|6]

check_spop: This plugin checks whether a secure POP server is running and accepting connections on the specified host using a secure connection.

Usage:
check_spop -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_ssh: Attempts to establish an SSH connection with the specified server and port.

Usage:
check_ssh  [-4|-6] [-t ] [-r ] [-p ] 

check_ssmtp: This plugin checks if an SSMTP server is running and can accept connections on the specified host.

Usage:
check_ssmtp -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

check_tcp: This plugin checks whether a TCP service is running and accepting connections on the specified host.

Usage:
check_tcp -H host -p port [-w ] [-c ] [-s ]
[-e ] [-q ][-m ] [-d ]
[-t ] [-r ] [-M ] [-v] [-4|-6] [-j]
[-D [,]] [-S ] [-E]

To learn more, run the following command in the plugins directory:

plugin_name --help

Farmguardian uses these plugins to check the health of the backends and manages the execution error output of the executed plugin to determine the backend status as follows:

If the error output == 0 then the backend is OK > $? = 0
If the error output <> 0 then the backend is NOT OK > $? <> 0

Custom plugins

Sysadmins can fully program and configure these plugins to any protocol or application.
This example shows a custom plugin: check_load.sh.

#!/bin/bash
###
###comments:
###snmp utils should be installed
###snmpd should be installed and configured in the backends
###
MAXVALUE=4
COMMUNITY="public"
EXECUTE=`snmpget -v 2c -c $COMMUNITY $1 .1.3.6.1.4.1.2021.10.1.3.1 |cut -d ':' -f2 | cut -d '.' -f1 | sed s/ // | sed s/"//`

echo "SNMP CPU load check for $1 is $EXECUTE"
# If the result is true, exit with 1; error; else exit = 0; OK
if (( $EXECUTE >= $MAXVALUE )); then
#error output; the server is overloaded and the load balancer isn't going to send more connections
exit 1
else
#, not error; the server can accept more connections
exit 0
fi

Constants

When Farmguardian executes a plugin, it can use some constants or tokens as arguments, like:

• HOST: Farmguardian will take care of modifying this constant by the real server IP address.
• PORT: Farmguardian will take care of modifying this constant by the real server port.

These constants can be used for every plugin, Farmguardian will use them to run the health check with the real parameters in place.

Farms

This page shows a list of farms and services using the currently selected Farmguardian health check.

Farms and services can be added to or removed from this Farmguardian health check by dragging and dropping between the two tables.

Next Article: Monitoring | Reports