IT Tips and Tricks

Introduction:

I’ve been confronted with the task of installing 2 web servers and a load balancer all made of Apache2.

Requirements:

– 1 Load balancer made of apache2 configured for 2 Web sites
– The load balancer should implement Cookies based stikyness for clients to land all their requests on the same web server initially selected by the load balancer.
– 2 Web servers (Apache2) behind the load balancer for the following PHP based Web Site.
– Each web site on each server should be individually accessible directly (not behing LB).
– The web site configuration based on the Apache mod_fcgi php5-cgi to allow for php.ini flexibility and Apache process user full access to all its htdocs. This web site is based on Typo3.
– One MySQL server per web server synchronized between each other using loop replication.
‘loop replication’ meaning each server is a Master and a slave for the other Master which does the same.

Constants:

(Fictive IPs here given as examples)
LB1: Load Balancer, eth0:1.1.1.1 eth1:192.168.0.1
WWW8: Web server/MySQL Server. eth0: 1.1.1.2 eth1:192.168.0.88
WWW9: Web server/MySQL Server. eth0: 1.1.1.3 eth1:192.168.0.89
Web site: http://www.myweb1.com

NOTE: It is not whown here how to configure an NFS server to hold the web space that will be used by both web servers. This way any changes of htdocs files on the NFS server will be immediately reflected in both web servers. In this case the NFS web space is mounted on mountpoint:
/var/www on each web server (WWW8 and WWW9).
IMPORTANT: Make sure that the following Apache VirtualHost configuration is the only one in this Load-Balancer system OR that it’s the first one seen by apache. We don’t specify any ServerName here, therefore all requests should land in this VirtualHost.

Configuration of Load Balancer:

File: /etc/hosts
192.168.0.88 www8.srv www8
192.168.0.89 www9.srv www9
File: /etc/apache2/sites-available/www.myweb1.com.conf
<VirtualHost *:80>
DocumentRoot /var/www/
ProxyRequests Off
ProxyPreserveHost On
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
ProxyPass / balancer://mycluster/
<Proxy balancer://mycluster>
BalancerMember http://www8.srv route=www8
BalancerMember http://www9.srv route=www9
ProxySet stickysession=ROUTEID
</Proxy>
ProxyPass /balancer-manager !
<Location /balancer-manager>
SetHandler balancer-manager
Order deny,allow
Allow from all
AuthType Basic
AuthName "Restricted area"
AuthUserFile /etc/apache2/web.auth
<limit GET PUT POST>
require valid-user
satisfy all
</limit>
</Location>
CustomLog /var/log/apache2/www.myweb1.com_acc.log combined
ErrorLog /var/log/apache2/www.myweb1.com_err.log
</VirtualHost>
Note: The www8.srv and www9.srv are already known in system(in /etc/hosts or in local DNS) as the hostnames of the 2 web servers.
Directory structure on the NFS server where the web space files are held:
/var/www/www.myweb1.com/
|-fcgi/
| |-php.ini
| \-php-fcgi-starter (fcgi Wrapper)
|
|-htdocs/ (Holds the PHP scripts/CMS etc.)
|
|-phptmp/ (holds the PHP temporary Sessions IDs)
Important note: On the NFS server the ownership of this full structure(starting from ….www.myweb1.com/) MUST be the same as the Apache user used in Suexec(ftp-myweb-com:ftp-myweb-com) of both Web servers below. Same name and same UID. The file php-fcgi-starter must have the execute rights (-rwxr-x—)

Content of file: php-fcgi-starter
#!/bin/bash
export PHPRC="/var/www/www.myweb1.com/fcgi/"
export PHP_FCGI_CHILDREN=2
export PHP_FCGI_MAX_REQUESTS=500
exec /usr/bin/php5-cgi

Make sure the file php.ini contains these configuration entries:
session.save_path = "/var/www/www.myweb1.com/phptmp"
doc_root = "/var/www/www.myweb1.com/htdocs"


To do on both Web servers:

Create an Apache/Suexec user for this web site. Any FTP connection to the NFS server to make changes to the htdocs should also login as this user.
useradd -d /var/www/www.myweb1.com/ -s /bin/bash ftp-myweb-com

Apache Configuration

Note: Web server 1 and 2 are identical Apache configuration
File: /etc/apache2/sites-available/www.myweb1.com.conf
<VirtualHost *:80>
ServerName www.myweb1.com
ServerAlias myweb1.com
DocumentRoot /var/www/www.myweb1.com/htdocs
SuexecUserGroup ftp-myweb-com ftp-myweb-com
<Directory /var/www/www.myweb1.com/htdocs>
Order Deny,Allow
Allow From All
DirectoryIndex index.html index.php
Options -Indexes +FollowSymLinks +ExecCGI
FCGIWrapper /var/www/www.myweb1.com/fcgi/php-fcgi-starter .php
AddHandler fcgid-script .php
AllowOverride All FileInfo Options
</Directory>
ErrorLog /var/log/apache2/www.myweb1.com_error_log
CustomLog /var/log/apache2/www.myweb1.com_access_log combined
</VirtualHost>

CLIENTS IPs in logs: Since we are using the Proxy module of Apache to send the requests from the load balancer to the web servers, each web server will then see the IP of the load balancer instead of the client in the logs. To remedy to this, you need to change the log format of Apache in order to log the real client IP which happens to be found in the request header X-Forwarded-For: .
Edit the file /etc/apache2/apache2.conf and change the line:
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
TO:
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
NOTE: In the above example of web servers I use an extra NFS server to hold the htdocs which should be mounted by both WWW8 and WWW9 servers. As I wanted to improve upon the independence of the htdocs, I used local htdocs directories(on board the web servers) and used GlusterFS to synchronize them between each other. This way there is no need for any external NFS server for having the same content in htodcs for both web servers.
See the site: https://tipstricks.itmatrix.eu/?s=GlusterFS&x=0&y=0 for explanations on how to configure GlusterFS for such purpose.