=======Deploying of multi-protocols and fault-tolerant Proxy Service ======= ==== Introduction. ==== For example, you have the task to deploy fault-tolerant proxy service with following functionality: - access to web-resources\\ - native access to ftp resources with writing capabilities\\ - access to Google Apps for Business or Yandex Mail for domain or etc. with following protocols: IMAP,SMTP\\ Solution is Linux Centos + Cororsync + pacemaker + squid + delegate. ==== Organisation Scheme ==== Three servers are necessary for installation. Two servers will be proxy nodes and third will be quorum server. (Of cause you can deploy service without quorum but this approach is unwanted ). Following example will describe situation with local and internet networks. If you have DMZ you will need to configure one network interface on each of servers only. {{:ru:jobs:proxymulti.jpg?500|500}} [[http://www.squid-cache.org/|Squid]] will be http,https, ftp over http - proxy (port 8080)\\ [[http://www.delegate.org/delegate/|Delegate]] will be IMAP,SMTP and native FTP proxy. Delegate can work as http proxy but have low performance. ==== Deployment ==== 1. Installing of OS Centos 6.3 on each of servers and configure interfaces.\\ 2. Adding necessary repositories on each server\\ 2.1. EPEL wget http://mirror.yandex.ru/epel/6/x86_64/epel-release-6-8.noarch.rpm (check actual package version before) rpm -i epel-release-6-8.noarch.rpm 2.2 squid-repo Squid 3.1 is a standard version which include in Centos 6.3. If [[http://wiki.squid-cache.org/Features/HTTP11|http/1.1 answers]] is needed than squid 3.2 is required. Follow the official site [[http://wiki.squid-cache.org/SquidFaq/BinaryPackages#KnowledgeBase.2BAC8-RedHat.Squid-3.2|information]] and add squid repo cat /etc/yum.repos.d/squid.repo [squid] name=Extra Packages squid proxy for Centos 6 baseurl=http://repo.ngtech.co.il/rpm/centos/6/x86_64/ enabled=1 gpgcheck=0 3. Updating OS and installing squid and other packages on each server. yum update reboot (if needed) yum install htop systat blktrace scp ntp bind bind-utils pacemaker corosync make gcc \ gcc-c++ openssl-devel openssh-clients squid 4. Configuring local NTPD on each server /etc/ntp.conf driftfile /var/lib/ntp/drift restrict default kod nomodify notrap nopeer noquery restrict -6 default kod nomodify notrap nopeer noquery restrict 127.0.0.1 restrict -6 ::1 server your_1.ntp.server Setting starting service at boot and start chkconfig ntpd on service ntpd start 5. Configuring local DNS - cache server on each proxy node cat /etc/named.conf // // named.conf // // Provided by Red Hat bind package to configure the ISC BIND named(8) DNS // server as a caching only nameserver (as a localhost DNS resolver only). // // See /usr/share/doc/bind*/sample/ for example named configuration files. // options { listen-on port 53 { 127.0.0.1; }; listen-on-v6 port 53 { ::1; }; directory "/var/named"; dump-file "/var/named/data/cache_dump.db"; statistics-file "/var/named/data/named_stats.txt"; memstatistics-file "/var/named/data/named_mem_stats.txt"; recursion yes; /* Path to ISC DLV key */ bindkeys-file "/etc/named.iscdlv.key"; managed-keys-directory "/var/named/dynamic"; max-ncache-ttl 3600; max-cache-ttl 28800; allow-query { any; }; allow-recursion { any; }; forwarders { **your_dns_server1**; **your_dns_server2**; }; }; logging { channel default_debug { file "data/named.run"; severity dynamic; }; }; zone "." IN { type hint; file "named.ca"; }; include "/etc/named.rfc1912.zones"; include "/etc/named.root.key"; Set nameserver cat /etc/resolv.conf nameserver 127.0.0.1 6. Installing Delegate on proxy-node1 an proxy-node2 6.1 Downloading final stable version http://www.delegate.org/delegate/download/ and unpacking.\\ 6.2 Compile with make\\ 6.3 Copy binary delegated from /usr/src/delegatex.x.x/src/delegate to /usr/sbin directory three times for each protocols\\ cp delegated /usr/sbin/delegated-imap cp delegated /usr/sbin/delegated-smtp cp delegated /usr/sbin/delegated-ftp 6.4 Creating scripts FTP cat /etc/init.d/delegate-ftp #!/bin/sh # Startup script for anacron # # chkconfig: 34 60 40 # description: Run Delegate daemons . /etc/init.d/functions prog="delegated-ftp" startoptions="-r -P0.0.0.0:21 SERVER=ftp ADMIN=adm@your_domain.dom \ CACHE=no REMITTABLE=ftp,ftps PERMIT=*:*:192.168.* MAXIMA=delegated:100,standby:80 \ TIMEOUT=shutout:300,restart:1d,acc:100,con:100,dns:4,dnsinv:1 \ RESOLV=sys,file SRCIF=217.9.80.9:*:* LOGFILE=ftp[date+.%d]" stopoptions="-P0.0.0.0:21 -Fkill" lockfile=/var/lock/subsys/$prog start() { echo -n $"Starting $prog: " # daemon $prog $startoptions RETVAL=$? echo [ $RETVAL -eq 0 ] && touch $lockfile return $RETVAL } stop() { echo -n $"Stopping $prog: " # daemon $prog $stopoptions RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f $lockfile return $RETVAL } case "$1" in start) start ;; stop) stop ;; status) status $prog ;; restart) stop start ;; *) echo $"Usage: $0 {start|stop|restart|status}" exit 1 esac exit 0 chmod 755 /etc/init.d/delegate-ftp IMAP For YANDEX cat /usr/sbin/delegate-imapo #!/bin/sh /usr/sbin/delegated-imap -r -P0.0.0.0:143 SERVER=imap ADMIN=adm@your_domain.dom SRCIF=217.9.80.9:*:* CACHE=no \ REMITTABLE=imap,imaps PERMIT=*:*:192.168.* STLS=fsv MOUNT="//*%S/%S imaps://imap.yandex.ru/*%(1)@%(0)" LOGFILE="imap[date+.%d]" For GOOGLE cat /usr/sbin/delegate-imapo #!/bin/sh /usr/sbin/delegated-imap -r -P0.0.0.0:143 SERVER=imap ADMIN=adm@your_domain.dom SRCIF=217.9.80.9:*:* CACHE=no \ REMITTABLE=imap,imaps PERMIT=*:*:192.168.* STLS=fsv MOUNT="//*%S/%S imaps://imap.gmail.com/*%(1)@%(0)" LOGFILE="imap[date+.%d]" chmod 755 /usr/sbin/delegated-imapo Tuning Gelegate If you will face with heavy duty problems just tune MAXIMA parameters. For example MAXIMA=listen:50,delegated:300 cat /etc/init.d/delegate-imap #!/bin/sh # Startup script for anacron # # chkconfig: 34 60 40 # description: Run Delegate daemons . /etc/init.d/functions prog="delegated-imap" progo="delegated-imapo" stopoptions="-P0.0.0.0:143 -Fkill" lockfile=/var/lock/subsys/$prog start() { echo -n $"Starting $prog: " # daemon $progo RETVAL=$? echo [ $RETVAL -eq 0 ] && touch $lockfile return $RETVAL } stop() { echo -n $"Stopping $prog: " # daemon $prog $stopoptions RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f $lockfile return $RETVAL } case "$1" in start) start ;; stop) stop ;; status) status $prog ;; restart) stop start ;; *) echo $"Usage: $0 {start|stop|restart|status}" exit 1 esac exit 0 chmod 755 /etc/init.d/delegate-imap SMTP For YANDEX cat /usr/sbin/delegate-smtpo #!/bin/sh /usr/sbin/delegated-smtp -r -P0.0.0.0:25 SERVER=smtps://smtp.yandex.ru/ CACHE=no REMITTABLE=* CACHE=no PERMIT=*:*:192.168.* \ LOGFILE="smtp[date+.%d]" STLS=fsv SRCIF=217.9.80.9:*:* For GOOGLE cat /usr/sbin/delegate-smtpo #!/bin/sh /usr/sbin/delegated-smtp -r -P0.0.0.0:25 SERVER=smtps://smtp.gmail.com/ CACHE=no REMITTABLE=* CACHE=no PERMIT=*:*:192.168.* \ LOGFILE="smtp[date+.%d]" STLS=fsv SRCIF=217.9.80.9:*:* chmod 755 /usr/sbin/delegated-smtpo Tuning Delegate If you will face with heavy duty problems just tune MAXIMA parameters. For example MAXIMA=listen:50,delegated:300 cat /etc/init.d/delegate-smtp #!/bin/sh # Startup script for anacron # # chkconfig: 34 60 40 # description: Run Delegate daemons . /etc/init.d/functions prog="delegated-smtp" progo="delegated-smtpo" stopoptions="-P0.0.0.0:25 -Fkill" lockfile=/var/lock/subsys/$prog start() { echo -n $"Starting $prog: " # daemon $progo RETVAL=$? echo [ $RETVAL -eq 0 ] && touch $lockfile return $RETVAL } stop() { echo -n $"Stopping $prog: " # daemon $prog $stopoptions RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f $lockfile return $RETVAL } case "$1" in start) start ;; stop) stop ;; status) status $prog ;; restart) stop start ;; *) echo $"Usage: $0 {start|stop|restart|status}" exit 1 esac exit 0 chmod 755 /etc/init.d/delegate-smtp 6.5. Starting delegated on both servers. Because delegated is very stable application you can just start it on both proxy servers. (I'm using delegate more the seven years) chkconfig delegate-ftp on chkconfig delegate-smtp on chkconfig delegate-imap on service delegate-ftp start service delegate-smtp start service delegate-imap start Use cron for restart services once a week for log rotation (not more often than once a day without changing of LOGFILE="protocol[date+.%d]") 7. Configuring squid on each server Detail configuration will not describe. For detail configuration try out this [[http://wiki.squid-cache.org/ConfigExamples|link]].\\ For this example: cat /etc/squid/squid.conf acl localhost src 127.0.0.1/32 ::1 #acl localnet src 10.0.0.0/8 # RFC1918 possible internal network #acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/24 # RFC1918 possible internal network #acl localnet src fc00::/7 # RFC 4193 local private network range #acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl CONNECT method CONNECT shutdown_lifetime 3 second http_access allow manager localhost http_access deny manager http_access allow localnet http_access allow localhost # And finally deny all other access to this proxy http_access deny all http_port 8080 tcp_outgoing_address 217.9.80.9 hierarchy_stoplist cgi-bin ? cache_mem 2048 MB maximum_object_size_in_memory 2048 KB # Leave coredumps in the first cache dir coredump_dir /var/spool/squid # Add any of your own refresh_pattern entries above these. refresh_pattern . 0 20% 240 Remove starting at boot chkconfig squid off 8. Configuring Corosync on each server cat /etc/corosync/corosync.conf compatibility: whitetank totem { version: 2 secauth: on threads: 0 interface { ringnumber: 0 bindnetaddr: 192.168.0.N # where N is IP address of front end network mcastaddr: 226.94.1.1 mcastport: 5405 ttl: 1 } } logging { fileline: off to_stderr: no to_logfile: yes to_syslog: yes logfile: /var/log/cluster/corosync.log debug: off timestamp: on logger_subsys { subsys: AMF debug: off } } amf { mode: disabled } cat /etc/corosync/service.d/pcmk service { # Load the Pacemaker Cluster Resource Manager name: pacemaker ver: 1 } END Create authkey on one server and copy it to another servers. corosync-keygen. Restart services service corosync restart service pacemaker restart Check crm status Last updated: Fri Jan 11 09:31:55 2013 Last change: Tue Dec 11 14:33:11 2012 via crm_resource on proxy-node1 Stack: openais Current DC: proxy-node1 - partition with quorum Version: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d14 3 Nodes configured, 3 expected votes ============ Online: [ proxy-node1 proxy-node2 proxy-quorum ] 9. Configuring pacemaker On one proxy node crm crm(live)# crm(live)#configure crm(live)configure#crm configure property no-quorum-policy=stop crm(live)configure#crm configure property stonith-enabled=false crm(live)configure#primitive LShareIP ocf:heartbeat:IPaddr2 params ip="192.168.0.9" cidr_netmask="23" op monitor interval="30s" on_fail="standby" crm(live)configure#primitive GShareIP ocf:heartbeat:IPaddr2 params ip="217.9.80.9" cidr_netmask="24" op monitor interval="30s" on_fail="standby" crm(live)configure#primitive squid lsb:squid op monitor interval="120s" on_fail="standby" \ op start interval="0" timeout="120s" \ op stop interval="0" timeout="120s" crm(live)configure#commit On one quorum node crm crm(live)#node crm(live)node#standby Starting at boot chkconfig corosync on chkconfig pacemaker on 10. Configuring iptables cat /etc/sysconfig/iptables # Firewall configuration written by system-config-firewall # Manual customization of this file is not recommended. *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -i eth0 -p icmp -j ACCEPT -A INPUT -i eth0 -j DROP COMMIT eth0 is external interface. service iptables restart chkconfig iptables on ====PS==== Special gratitude [[http://staff.aist.go.jp/y.sato/|Yutaka Sato]] for very nice proxy application (delegate). ==== Centos/Redhat 6.4 ==== Today i have updated the system to centos 6.4. Crm configuration command disappeared. \\ If you want to use crm configuration command with centos/redhat 6.4 you need to install crmsh: yum install crmsh\\ I'm using follow repo [network_ha-clustering] name=High Availability/Clustering server technologies (RedHat_RHEL-6) type=rpm-md baseurl=http://download.opensuse.org/repositories/network:/ha-clustering/RedHat_RHEL-6/ gpgcheck=1 gpgkey=http://download.opensuse.org/repositories/network:/ha-clustering/RedHat_RHEL-6/repodata/repomd.xml.key enabled=1 Or you can use pcs (yum install pcs) ==== About author ==== [[https://www.linkedin.com/pub/alexey-vyrodov/59/976/16b|Profile]] of the author