Configuration of multi-protocols and fault-tolerant Proxy Service

For example, you have the task to deploy fault-tolerant proxy service with following functionality:

- access to web-resources
- native access to ftp resources with writing capabilities
- access to Google Apps for Business or Yandex Mail for domain or etc. with following protocols: IMAP,SMTP

Solution is Linux Centos + Cororsync + pacemaker + squid + delegate.

Three servers are necessary for installation. Two servers will be proxy nodes and third will be quorum server. (Of cause you can deploy service without quorum but this approach is unwanted ). Following example will describe situation with local and internet networks. If you have DMZ you will need to configure one network interface on each of servers only.

Squid will be http,https, ftp over http - proxy (port 8080)
Delegate will be IMAP,SMTP and native FTP proxy. Delegate can work as http proxy but have low performance.

1. Installing of OS Centos 6.3 on each of servers and configure interfaces.
2. Adding necessary repositories on each server
2.1. EPEL

wget http://mirror.yandex.ru/epel/6/x86_64/epel-release-6-8.noarch.rpm  (check actual package version before)
rpm -i epel-release-6-8.noarch.rpm

2.2 squid-repo

Squid 3.1 is a standard version which include in Centos 6.3. If http/1.1 answers is needed than squid 3.2 is required. Follow the official site information and add squid repo

cat /etc/yum.repos.d/squid.repo
[squid]
name=Extra Packages squid proxy for Centos 6
baseurl=http://repo.ngtech.co.il/rpm/centos/6/x86_64/
enabled=1
gpgcheck=0

3. Updating OS and installing squid and other packages on each server.

yum update
reboot (if needed)
yum install htop systat blktrace scp ntp bind bind-utils pacemaker corosync make gcc \
gcc-c++ openssl-devel openssh-clients squid

4. Configuring local NTPD on each server

/etc/ntp.conf
driftfile /var/lib/ntp/drift
restrict default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
restrict 127.0.0.1
restrict -6 ::1
server your_1.ntp.server

Setting starting service at boot and start

chkconfig ntpd on
service ntpd start

5. Configuring local DNS - cache server on each proxy node

cat /etc/named.conf
//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
options {
      listen-on port 53 { 127.0.0.1; };
      listen-on-v6 port 53 { ::1; };
      directory       "/var/named";
      dump-file       "/var/named/data/cache_dump.db";
      statistics-file "/var/named/data/named_stats.txt";
      memstatistics-file "/var/named/data/named_mem_stats.txt";
      recursion yes;
      /* Path to ISC DLV key */
      bindkeys-file "/etc/named.iscdlv.key";
      managed-keys-directory "/var/named/dynamic";
      max-ncache-ttl  3600;
      max-cache-ttl 28800;
      allow-query { any; };
      allow-recursion { any; };
      forwarders { **your_dns_server1**; **your_dns_server2**; };
};
logging {
      channel default_debug {
              file "data/named.run";
              severity dynamic;
      };
};
zone "." IN {
      type hint;
      file "named.ca";
};
include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";

Set nameserver

cat /etc/resolv.conf
nameserver 127.0.0.1

6. Installing Delegate on proxy-node1 an proxy-node2

6.1 Downloading final stable version http://www.delegate.org/delegate/download/ and unpacking.

6.2 Compile with make

6.3 Copy binary delegated from /usr/src/delegatex.x.x/src/delegate to /usr/sbin directory three times for each protocols

 
cp delegated /usr/sbin/delegated-imap
cp delegated /usr/sbin/delegated-smtp
cp delegated /usr/sbin/delegated-ftp

6.4 Creating scripts

FTP

cat /etc/init.d/delegate-ftp
#!/bin/sh
# Startup script for anacron
#
# chkconfig: 34 60 40
# description: Run Delegate daemons
. /etc/init.d/functions
prog="delegated-ftp"
startoptions="-r -P0.0.0.0:21 SERVER=ftp ADMIN=adm@your_domain.dom CACHE=no REMITTABLE=ftp,ftps PERMIT=*:*:192.168.* \
MAXIMA=delegated:100,standby:80 TIMEOUT=shutout:300,restart:1d,acc:100,con:100,dns:4,dnsinv:1 \
RESOLV=sys,file  SRCIF=217.9.80.9:*:*  LOGFILE=ftp[date+.%d]"
stopoptions="-P0.0.0.0:21 -Fkill"
lockfile=/var/lock/subsys/$prog
start() {
  echo -n $"Starting $prog: "
  #
  daemon $prog $startoptions
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch $lockfile
  return $RETVAL
}
stop() {
  echo -n $"Stopping $prog: "
  #
  daemon $prog $stopoptions
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && rm -f $lockfile
  return $RETVAL
}
case "$1" in
      start)
          start
          ;;
      stop)
          stop
          ;;
      status)
          status $prog
          ;;
      restart)
          stop
          start
          ;;
      *)
          echo $"Usage: $0 {start|stop|restart|status}"
          exit 1
esac
exit 0

chmod 755 /etc/init.d/delegate-ftp

IMAP

For YANDEX

cat /usr/sbin/delegate-imapo
#!/bin/sh
/usr/sbin/delegated-imap -r -P0.0.0.0:143 SERVER=imap ADMIN=adm@your_domain.dom  SRCIF=217.9.80.9:*:* CACHE=no \
REMITTABLE=imap,imaps PERMIT=*:*:192.168.* STLS=fsv MOUNT="//*%S/%S imaps://imap.yandex.ru/*%(1)@%(0)" LOGFILE="imap[date+.%d]"

For GOOGLE

cat /usr/sbin/delegate-imapo
#!/bin/sh
/usr/sbin/delegated-imap -r -P0.0.0.0:143 SERVER=imap ADMIN=adm@your_domain.dom  SRCIF=217.9.80.9:*:* CACHE=no \
REMITTABLE=imap,imaps PERMIT=*:*:192.168.* STLS=fsv MOUNT="//*%S/%S imaps://imap.gmail.com/*%(1)@%(0)" LOGFILE="imap[date+.%d]"

chmod 755 /usr/sbin/delegated-imapo

cat /etc/init.d/delegate-imap
#!/bin/sh
# Startup script for anacron
#
# chkconfig: 34 60 40
# description: Run Delegate daemons
. /etc/init.d/functions
prog="delegated-imap"
progo="delegated-imapo"
stopoptions="-P0.0.0.0:143 -Fkill"
lockfile=/var/lock/subsys/$prog
start() {
  echo -n $"Starting $prog: "
  #
  daemon $progo
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch $lockfile
  return $RETVAL
}
stop() {
  echo -n $"Stopping $prog: "
  #
  daemon $prog $stopoptions
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && rm -f $lockfile
  return $RETVAL
}
case "$1" in
      start)
          start
          ;;
      stop)
          stop
          ;;
      status)
          status $prog
          ;;
      restart)
          stop
          start
          ;;
      *)
          echo $"Usage: $0 {start|stop|restart|status}"
          exit 1
esac
exit 0

chmod 755 /etc/init.d/delegate-imap

SMTP

For YANDEX

cat /usr/sbin/delegate-smtpo
#!/bin/sh
/usr/sbin/delegated-smtp -r -P0.0.0.0:25 SERVER=smtps://smtp.yandex.ru/  CACHE=no REMITTABLE=* CACHE=no PERMIT=*:*:192.168.* \
LOGFILE="smtp[date+.%d]" STLS=fsv SRCIF=217.9.80.9:*:*

For GOOGLE

cat /usr/sbin/delegate-smtpo
#!/bin/sh
/usr/sbin/delegated-smtp -r -P0.0.0.0:25 SERVER=smtps://smtp.gmail.com/  CACHE=no REMITTABLE=* CACHE=no PERMIT=*:*:192.168.* \
LOGFILE="smtp[date+.%d]" STLS=fsv SRCIF=217.9.80.9:*:*

chmod 755 /usr/sbin/delegated-smtpo

cat /etc/init.d/delegate-smtp
#!/bin/sh
# Startup script for anacron
#
# chkconfig: 34 60 40
# description: Run Delegate daemons
. /etc/init.d/functions
prog="delegated-smtp"
progo="delegated-smtpo"
stopoptions="-P0.0.0.0:25 -Fkill"
lockfile=/var/lock/subsys/$prog
start() {
  echo -n $"Starting $prog: "
  #
  daemon $progo
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && touch $lockfile
  return $RETVAL
}
stop() {
  echo -n $"Stopping $prog: "
  #
  daemon $prog $stopoptions
  RETVAL=$?
  echo
  [ $RETVAL -eq 0 ] && rm -f $lockfile
  return $RETVAL
}
case "$1" in
      start)
          start
          ;;
      stop)
          stop
          ;;
      status)
          status $prog
          ;;
      restart)
          stop
          start
          ;;
      *)
          echo $"Usage: $0 {start|stop|restart|status}"
          exit 1
esac
exit 0

chmod 755 /etc/init.d/delegate-smtp

6.5. Starting delegated on both servers.

Because delegated is very stable application you can just start it on both proxy servers. (I'm using delegate more the seven years)

chkconfig delegate-ftp on
chkconfig delegate-smtp on 
chkconfig delegate-imap on

service delegate-ftp start
service delegate-smtp start
service delegate-imap start

Use cron for restart services once a week for log rotation (not more often than once a day without changing of LOGFILE=“protocol[date+.%d]”)

7. Configuring squid on each server

Detail configuration will not describe. For detail configuration try out this link.
For this example:

cat /etc/squid/squid.conf
acl localhost src 127.0.0.1/32 ::1
#acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/24  # RFC1918 possible internal network
#acl localnet src fc00::/7       # RFC 4193 local private network range
#acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
acl CONNECT method CONNECT
shutdown_lifetime 3 second
http_access allow manager localhost
http_access deny manager
http_access allow localnet
http_access allow localhost
# And finally deny all other access to this proxy
http_access deny all
http_port 8080
tcp_outgoing_address 217.9.80.9
hierarchy_stoplist cgi-bin ?
cache_mem 2048 MB
maximum_object_size_in_memory 2048 KB
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
# Add any of your own refresh_pattern entries above these.
refresh_pattern .               0       20%     240

Remove starting at boot

chkconfig squid off

8. Configure Corosync on each server

cat /etc/corosync/corosync.conf

compatibility: whitetank
totem {
      version: 2
      secauth: on
      threads: 0
      interface {
              ringnumber: 0
              bindnetaddr: 192.168.0.N # where N is IP address of front end network
              mcastaddr: 226.94.1.1
              mcastport: 5405
              ttl: 1
      }
}
logging {
      fileline: off
      to_stderr: no
      to_logfile: yes
      to_syslog: yes
      logfile: /var/log/cluster/corosync.log
      debug: off
      timestamp: on
      logger_subsys {
              subsys: AMF
              debug: off
      }
}
amf {
      mode: disabled
}

cat /etc/corosync/service.d/pcmk
service {
      # Load the Pacemaker Cluster Resource Manager
      name: pacemaker
      ver:  1
}
END

Create authkey on one server and copy it to another servers.

corosync-keygen.

Restart services

service corosync restart
service pacemaker restart

Check

crm status

Last updated: Fri Jan 11 09:31:55 2013
Last change: Tue Dec 11 14:33:11 2012 via crm_resource on proxy-node1 
Stack: openais
Current DC: proxy-node1 - partition with quorum
Version: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d14
3 Nodes configured, 3 expected votes
============
Online: [ proxy-node1  proxy-node2 proxy-quorum  ]

9. Configure pacemaker

On one proxy node

crm
crm(live)#
crm(live)#configure
crm(live)configure#crm configure property no-quorum-policy=stop 
crm(live)configure#crm configure property stonith-enabled=false
crm(live)configure#primitive LShareIP ocf:heartbeat:IPaddr2 params ip="192.168.0.9" cidr_netmask="23" op monitor interval="30s" on_fail="standby"
crm(live)configure#primitive GShareIP ocf:heartbeat:IPaddr2 params ip="217.9.80.9" cidr_netmask="24" op monitor interval="30s" on_fail="standby"
crm(live)configure#primitive squid lsb:squid op monitor interval="120s" on_fail="standby" \
op start interval="0" timeout="120s" \
op stop  interval="0" timeout="120s"
crm(live)configure#commit

On one quorum node

crm
crm(live)#node
crm(live)node#standby

Starting at boot

chkconfig corosync on
chkconfig pacemaker on

10. Configuring iptables

cat /etc/sysconfig/iptables
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i eth0 -p icmp -j ACCEPT
-A INPUT -i eth0 -j DROP
COMMIT

eth0 is external interface.

service iptables restart
chkconfig iptables on

Special gratitude Yutaka Sato for very nice proxy application (delegate).