Continuing the nginx series, here is a high availability nginx setup using heartbeat. With this setup we have caching load balanced web servers and real time fail over high availability. Nginx proxies traffic for back end Apache web servers living on private IP's. Content is served from shared storage. HOST1 is the primary server, HOST2 is the secondary. An aliased IP address and the nginx service are automatically started on HOST2 in the event of a HOST1 failure.

1) Install the heartbeat software on each server.

  sudo apt-get install heartbeat

2) Make sure nginx will not start from init.d.

  sudo sysv-rc-conf --level nginx 2345 off

3) Set up a public aliased IP address for nginx to live on. This is the address for web traffic. Configure it on each machine as a manually activated interface, heartbeat will control it.

/etc/network/interfaces

  iface eth0:0 inet static
  address 192.156.134.105
  netmask 255.255.255.192
  network 192.156.134.64
  broadcast 192.156.134.127

4) Make sure the nginx setup works on the new IP, adjust the listen parameter of your nginx servers and fix DNS if needed.

/etc/nginx/sites-available/some-site.conf

  listen 192.156.134.105:80

5) Configure private IP's to send heartbeat messages between servers:

HOST1: /etc/network/interfaces

  auto eth1
  iface eth1 inet static 
  address 192.168.253.1
  netmask 255.255.255.0
  network 192.168.253.0
  broadcast 192.168.253.255

HOST2: /etc/network/interfaces

  auto eth1
  iface eth1 inet static
  address 192.168.253.2
  netmask 255.255.255.0
  network 192.168.253.0
  broadcast 192.168.253.255

6) On each machine, set up a heartbeat between them using the same configuration. We used the simpler version 1 configuration style:

HOST1: /etc/ha.d/ha.cf

  keepalive 2
  deadtime 30
  warntime 10
  initdead 120
  udpport 695
  ucast eth1 192.168.253.2
  auto_failback on
  node HOST1
  node HOST2
  crm no

7) On HOST2, change the ucast address to check on HOST1:

HOST2: /etc/ha.d/ha.cf

  ucast eth1 192.168.253.1

8) Tell Heartbeat what to do when a host goes away. On each server use the same configuration. We define HOST1 as the primary node, IP 192.156.134.105 as our takeover address, and nginx as a service to fail over.

/etc/ha.d/resources

  HOST1 192.156.134.105 nginx

9) On each host place the same authkeys file, used to identify the heartbeat.

To generate an sha1 authkey:

  dd if=/dev/urandom count=4 2>/dev/null | openssl dgst -sha1

/etc/ha.d/authkeys

  auth 1
  1 sha1 0c5ec3e74e15c6ffef33d7372f491fef702b2456

10) From a fresh system restart heartbeat should start the IP alias address and nginx on the primary node. Test failover either by shutting down HOST1 or move the IP and nginx service back and forth using heartbeat commands.

IP and nginx to HOST2:

  user@HOST1:$ sudo /usr/lib/hb_standby

IP and nginx back to HOST1:

  user@HOST1:$ sudo /usr/lib/hb_takeover