msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

24 Mar 16 Using HTTPS as proxy backend in Apache 2.4

Introduction:
In Apache 2.4 in a Vhost in order to be able to proxy to a backend with HTTPS using either a self-signed or expired certificate on the backend we need to include the following directives:
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off

You also need to enable the required Apache2 modules as follows:
a2enmod proxy
a2enmod proxy_http
a2enmod proxy_connect
service apache2 restart

Example when using Apache 2.4 to proxy to Webmin port 10000:
RewriteEngine On
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
RewriteRule ^/(.*) https://127.0.0.1:10000/$1 [P]
ProxyPassReverse / https://127.0.0.1:10000

09 Mar 16 Testing SSL Connections with SSLyze, Nmap or OpenSSL

Introduction:
OpenSSL is a great tool to check SSL connections to servers. The difficulty here is when one want a full scan of all possible SSL Cyphers and protocols used by a server. That is where SSLyze comes in handy. This tool is a Python script which will scan the target host/port for SSL handshake and report what works/support and what not. Unfortunately this lovely tool is not included in the Ubuntu/Debian distributions, and this is where this post comes handy.

IMPORTANT: Besides executing all the tests below one thing very important (as noted in the This link) is to upgrade OpenSSL to the latest version as follows:
OpenSSL 1.0.2 users should upgrade to 1.0.2g
OpenSSL 1.0.1 users should upgrade to 1.0.1s

SSLyze

Installing the dependencies and tool
cd /root/bin
wget https://github.com/nabla-c0d3/sslyze/archive/0.13.4.tar.gz
tar fvxz 0.13.4.tar.gz
apt-get install python-pip python-dev
pip install nassl

Using SSLyze
python /root/bin/sslyze-0.13.4/sslyze_cli.py --regular www.itmatrix.eu:443

NMAP

Scanning the full server for weaknesses including weak SSL Versions using NMAP.
Note: This operation can take a long time to execute.
apt-get install nmap
nmap -sV -sC www.itmatrix.eu

OR better(for checking the HTTPS,SMTPS,IMAPS,POP3S)
nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995 www.itmatrix.eu

OpenSSL

Checking the SSL connection with OpenSSL
echo 'q' | openssl s_client -host www.itmatrix.eu -port 443
Note: In this above case since the SSLv2 support is normally disabled for OpenSSL in Debian/Ubuntu distributions, you will not be able to see if the server is supporting it. To overcome this and enable SSLv2 support(for your testing Linux) then follow the instructions in this site:
http://www.hackwhackandsmack.com/?p=46

NOTE:
For more information regarding protection against DROWN(SSLv2) or POODLE(SSLv3) attacks see:
https://drownattack.com
http://www.softwaresecured.com/2016/03/01/how-to-confirm-whether-you-are-vulnerable-to-the-drown-attack/
http://www.mogilowski.net/lang/de-de/2014/10/23/disabling-sslv3-for-poodle-on-debian/
https://www.owasp.org/index.php/Testing_for_Weak_SSL/TLS_Ciphers,_Insufficient_Transport_Layer_Protection_%28OTG-CRYPST-001%29
https://zmap.io/sslv3/

17 Dec 15 Issue free and CA signed SSL certificates for web servers from LetsEncrypt

Introduction:
SSL Certificates provide two functions:
1. Authentication
2. Encryption

Encryption can be achieved without authentication but, for some reason, someone decided to join them together in one certificate. It seem to make sense for banks and serious e-commerce sites which need to be properly authenticated. Therefore when the HTTPS protocol got developed it was not possible to encrypt-only the stream of HTTP. This situation made us dependent to Certificate Authentication Authorities to obtain a certificate even if we only wanted encryption. Now some genius group of people at https://letsencrypt.org/ finally created the possibility to obtaining certificates which preform simple authentication verification, by calling the URL and expecting a specific response, and if successful issues a free 90 days valid and CA signed SSL certificate. For system administrators this process of requesting and install such free certificate has therefore become quite simple. Here is one method of doing just this in a Debian/Ubuntu web server.
References: http://www.admin-magazine.com/Articles/Getting-a-free-TLS-certificate-from-Let-s-Encrypt?utm_source=ADMIN+Newsletter&utm_campaign=ADMIN_Update_Free_Certificates_with_Let%27s_Encrypt_2016-20-07&utm_medium=email

STEPS:

Installing LetsEncrypt

apt-get update && apt-get install git
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto --email user@mydomain.com --agree-tos --help
echo "export PATH=$PATH:/usr/local/lib/letsencrypt" >> /root/.bashrc
. /root/.bashrc

NOTE: Make sure your web site you want to add HTTPS to is already configured and live in your web server.
The reason is that during the process of requesting a certificate, LetsEncrypt will create an extra sub-directory({htdocs}/.well-known/acme-challenge/) and a special temporary file in the htdocs of the site (pointed to by DocumentRoot directive in Apache) then call that file on the site from the LetsEncrypt server to authenticate the URL. If the the URL called is invalid it won’t issue the certificate. For this reason your site needs to be live and you need to give the path of the htdocs. After the authentication process, the temporary file will be erased but not the sub directories. They will stay empty.

Troubleshooting:

InsecurePlatformWarning
If you get the following error message then in Debian Wheezy you can solve it by importing SSl into Python. See below.
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

Importing Python SSL support:
python
>>> import ssl
>>> (CTRL-D)

Upgrading LetsEncrypt client program

rm -rf /root/.local/share/letsencrypt
rm -rf /usr/local/lib/letsencrypt.old &>/dev/null
mv /usr/local/lib/letsencrypt /usr/local/lib/letsencrypt.old
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt

Requesting the certificate

Eg. for the domain blog.mydomain.com
NOTE: on first time request the script will ask you to give an email address for contact purposes as well as to accept the terme and conditions of using this tool. Afterwards it will not ask you these questions.
letsencrypt-auto certonly --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com
The certificates and key will be stored in /etc/letsencrypt/live/blog.mydomain.com/ as:
cert.pem : Certificate
chain.pem : CA Certificate
privkey.pem : Private key
fullchain.pem : Combination of the certificate and the CA Certificate

Instead of moving the certificate, just configure Apache or other web server to point to the certs files where they are.
This way a cron job can be created to regularly renew the certificate automatically without manual intervention.
The certificate will be valid for 90 Days only; no exceptions.
This means that the same above command will need to be run every 3 months or earlier with the addition of the option –renew-by-default.
The limit of certificates you can ask for a certain domain is: currently 5 certificates / 7 days.

Renewing single certificate:

In order to renew the certificate automatically it is suggested to use a cron job and adding the option –renew-by-default in the command eg. as follows:
letsencrypt-auto certonly --renew-by-default --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com

Renewing all installed Letsencrypt certificates:

/usr/local/lib/letsencrypt/letsencrypt-auto renew
Note: It is recommended to send the output of the command by email to verify if the process was successful.

Extra Info

The certificates of LetsEncrypt are stored in /etc/letsencrypt/ directories in different ways. It is simply NOT recommended to delete any of the certificates, files or symlinks in these directories because the files in the ‘keys’ and ‘csr’ directories are not identified to refer to a specific certificate. So just deleting some files but not others related to the same cert might confuse the client command and you then can’t request any more certificates. The error message from the client program is something like:
letsencrypt TypeError: coercing to Unicode: need string or buffer, NoneType found
If you ever get to that non-return point then just delete all directories: archive, csr, keys, live and renewal BUT not accounts. Then re-issue certificates requests for already existing sites. The certificates will then be renewed and you can then also request new ones.

For more information of the subject see:
https://letsencrypt.readthedocs.org/en/latest/using.html

Comfortable script

If you want to be able to issue a certificate and you want it to self-renew after 80 days, this script might be of some use.
#!/bin/bash
# Purpose: Issue or renew a certificate from LetsEncrypt
# It will also issue an 'at'command which will be responsible to automatically renew the certificate automatically
# This script also issues a new at comand which will do the same in around 3 Months days depending on the settings here
# Syntax: cert_request.sh -s SITE_NAME -d SITE_HTDOCS
# Changes: 30.12.2015 First implementation of the script
# 10.01.2016 Took out the read of wpinstall.cfg config file. Added checks for the letsencrypt and at programs
#--------------------------------------------------------------
. /root/.bashrc
RENEW_DAYS="80"
# Absolute path to this script.
SCRIPT=$(readlink -f $0)
CERTS_DIR="/etc/letsencrypt/live"
# Absolute path this script is in.
scriptdir=$(dirname $SCRIPT)
encryptprgm="/usr/local/lib/letsencrypt/letsencrypt-auto"
atprgm="/usr/bin/at"
EMAIL="admin@itmatrix.eu"
#
# Check the syntax
function usage () {
echo "Usage: cert_request.sh -s SITE_NAME -d SITE_HTDOCS"
echo "-s SITE_NAME Full web site address WITHOUT the 'http://' eg.: www.myblog.com"
echo "-d SITE_HTDOCS The absolute path where WordPress will be installed. eg. /www/sites/www.mysite.com/htdocs"
exit 1
}
#
if [ $# -ne 4 ]; then
echo "ERROR: Wrong number of given argunents."
usage
fi
# Make sure the letsencrypt client prgm is installed
if ! [ -e $encryptprgm ] ; then
echo "ERROR: the letsencrypt program isn not installed. Install it and retry."
echo "See instructions at: //tipstricks.itmatrix.eu/install-new-and-signed-ssl-certificate-for-web-servers"
exit 1
fi
# Make sure the at is installed
if ! [ -e $atprgm ] ; then
echo "ERROR: the 'AT' program isn not installed. Install it and retry."
echo "apt-get install at"
exit 1
fi
# Everything look good so far. Lets start.
# get the command options
while getopts "s:d:" OPTION
do
case $OPTION in
s) SITE_NAME=$OPTARG
;;
d) SITE_HTDOCS=$OPTARG
;;
h|?|*)
echo "ERROR: argument(s) unknown."
usage
;;
esac
done
echo "Requesting certificate at LetsEncrypt"
# Does it exist already, then renew only, otherwise request renewing the cert
if [ -d $CERTS_DIR/${SITE_NAME} ] ; then
echo "The certificate already exists. Requesting a renewal"
RENEW="--renew-by-default"
else
RENEW=""
fi
#
if ($encryptprgm certonly $RENEW --webroot -w $SITE_HTDOCS -d ${SITE_NAME} &>/dev/null); then
# Enable the Apache SSL configuration and restart Apache
(echo "Certificate request successful."
echo "Issuing a renewal of the certificate in 80 days using 'at' command"
service apache2 restart
echo "$SCRIPT -s $SITE_NAME -d $SITE_HTDOCS" | $atprgm now + $RENEW_DAYS days)| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" admin@itmatrix.eu
echo -e "------- SITES LIST --------\n$(ls -1 /etc/apache2/sites-enabled/ | egrep -v '^00|^wptest1')\n\n--------- CERTIFICATES LIST ---------$(ls -l /etc/letsencrypt/live/ | cut -c29-)\n\n------- AT Jobs LIST -------\n$(/root/bin/atlist.sh)" | mail -s "Request/Renewal of Certificate request LIST" $EMAIL
cat /tmp/cert_request.sh.log
exit 0
else
(echo "ERROR: The certificate request/renewal FAILED.")| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" $EMAIL
cat /tmp/cert_request.sh.log
exit 2
fi

#

Status of the certificates

Here is a useful script that will display the following information:
– List of AT Jobs ready to start at the required time
– List of present certificates and their file timestamps
Since each letsencrypt is only valid for 90 days this will give you an overview of how old the present certificates are and when is the next time the requests for certificates will be done.
#!/bin/bash
# Description: Displays all 'at' jobs and their respective commands
# Systax: atlist.sh
# Changes: 05.11.2016 First inplementation
########################################################################
# Get the short jobs list and expand from there
echo "================ AT Jobs ready to start at the required times ==============="
atq | while read line ; do
jobnr=$(echo $line | awk '{print $1}')
echo $line
# Pickup all the command lines after first line matching '}'.
# This excludes all the environment variables and the at exit line.
# Exclude the matching '}' line and empty lines
# Add an offset of 8 chars to each command line.
# at -c $jobnr | grep -A100 -m1 -e '^\}' | grep -v '^\}' | sed -e '/^$/d' -e 's/^/ /'
at -c $jobnr | at -c $jobnr | sed -e '1,/^\}/d' -e '/^$/d' -e 's/^/ /'
done
echo ; echo
echo "=============== Age of present certificates ====================="
ls -l /etc/letsencrypt/live/*/cert.pem | awk '{print $6" "$7" "$8" "$9}' | sed -e 's|/etc/letsencrypt/live/||' -e 's|/cert.pem||'

Installing and using CERBOT

Introduction:
Cerbot tool simply adds more user-friendly features to Lestencrypt original seen above.
See this site for more information:
https://certbot.eff.org/docs/install.html

Use:
It can make a certificate request to Letsencryp server and also create ta temporary web server to allow to verify the HTTP connection of a site for which the certificate is requested. These features can be called simply by adding option on the command line.
For example: If I want a web site to be SSL only then generally the HTTP virtual Host would do a redirection to HTTPS automatically for any HTTP request. In this case it blocks the verification of the web site from Letsencrypt server. There are 2 ways that I know to avoid this difficulty.
1) Create a condition on the redirection directive in Apache that does not redirect to HTTPS if the URI is /.well-known/…..
Pro: No downtime of the web server
Con: An extra condition in the redirection directive must be done for each SSL site.
2) Stop the web server. Run cerbot with the –standalone option . Restart the web server.
This solution would only be good for a web server cluster environment if you don’t want to have downtime on your site.
Pro: No need for extra redirection condition for sites. Really good for Nginx servers where re-directions conditions are difficult to create
Con: The site gets a downtime during this procedure which can last long if you have many sites needing creation/renewal of certificates.

Installation:
wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto

Or if you have Ubuntu 16.04 or later, the following commands will also preferable for the installation:
apt-get install certbot python-certbot-apache
Then to get help on ‘certbot’
certbot --help
An example of a ‘certbot‘ command using its own web server for authentication, this way avoiding the difficulties of having to interfere with Apache/NginX for the authentication process.
The following example will request a single certificate which will be valid for these 4 subdomains using the (SAN) mechanism.
certbot certonly --standalone -d www.mydomain.com -d www.mydomain2.com -d mailman.mydomain.com -d mail.mydomain.com
In this above example you need to make sure Apache/NginX is not using the port 80. If it does, shut the web server down before this command and restart it right after it. It will result in a short web access downtime. This downtime might be acceptable depending on whether you have the web server behind a load -balancer or the web service can allow such downtime.
Example:
service apache2 stop
certbot certonly --standalone -d www.mydomain.com -d www.mydomain2.com -d mailman.mydomain.com -d mail.mydomain.com
service apache2 start

10 Dec 15 Creating a web certificate CSR file.

The process of buying an SSL certificate for a web site is usually as follows:
– You create a secret key and CSR files using the method showm in this post.
– You cut and paste the content of the CSR file into a field in a SSL Vendor web site
– The SSL vendor produces a certificate based on the CSR you provided and send it to you.
– You download the CA Certificate from the SSL provider’s site
– You install the private keyfile, the CA certificate and the certificate in the web server and bobs’s-your-uncle.

The following procdeure is an extract from the site:
https://support.globalsign.com/customer/portal/articles/1221018-generate-csr—openssl
Generate a CSR & Private Key:
openssl req -out CSR.csr -new -newkey rsa:2048 -nodes -keyout privatekey.key

Fill out the following fields as prompted:
Note: The following characters can not be accepted: <> ~ ! @ # $ % ^ * / \ ( ) ?.,&


Field Example
============ ==========================================
Country Name US (2 Letter Code)
State or Province New Hampshire (Full State Name)
Locality Portsmouth (Full City name)
Organization GMO GlobalSign Inc (Entity's Legal Name)
Organizational Unit Support (Optional, e.g. a department)
Common Name www.globalsign.com (Domain or Entity name

06 Nov 15 Configuring HAproxy load balancer in Ubuntu 14.04

Goal:
In this example HTTP requests are proxied directly as HTTP requests to the HTTP web servers. In the case of HTTPS requests, they are handled with the certificates by HAproxy and then proxied to the web servers as HTTP requests.

SSLCertificates:
The certificates for all virtualhosts being proxied are stored as one PEM format file per certificate/key combination in the directory:
/etc/ssl/private/
The CAs are also stored as one PEM format file per CA in the directory:
/etc/ssl/certs/

Steps:
Install HAproxy:
apt-get update && apt-get install haproxy

Configure HAproxy for HTTP and HTTPS load-balancing:

Edit the file /etc/haproxy/haproxy.cfg
Content:
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
user haproxy
group haproxy
daemon
#
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
#
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
tune.ssl.default-dh-param 2048
#
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
# Added to create separate error and access logs
option log-separate-errors
#
# ------- HTTP Frontend --------------
frontend http_in
bind *:80
mode http
reqadd X-Forwarded-Proto:\ http
default_backend http_out
#
# ------- HTTPS Frontend --------------
frontend https_glwp-in
bind *:443 ssl crt /etc/ssl/haproxy_certs/
mode http
reqadd X-Forwarded-Proto:\ https
default_backend http_out
#
#------------------------------------
listen stats :2000
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /stats
stats auth admin:mypasswd
#
# ------- HTTP Backend --------------
backend http_out
balance roundrobin
stick-table type ip size 200k expire 60m
stick on src
option forwardfor
option httpclose
http-request set-header X-Forwarded-Port %[dst_port]
option httpchk HEAD /
server web1 webserv1.mynet.net:80 check
server web2 webserv2.mynet.net:80 check
server web3 webserv3.mynet.net:80 check
server web4 webserv4.mynet.net:80 check

Preserving the source IP of client in TCP Proxying

In the above examples the protocols that are being load-balanced are application protocols, where you can retain the Source IP by retrieving it from the HTTP/HTTPS header X-Forwarded-For: (obtained by the option: option forwardfor), but if you use HAProxy as a TCP layer load balancer, in order to retain the source IP(client’s IP) see the following article: http://blog.haproxy.com/2012/06/05/preserve-source-ip-address-despite-reverse-proxies/
It’s a tiny bit complex to understand and implement, especially in the backend server. I have not tried it yet, so I can’t guarantee its validity therefore I can’t give any examples. From what I understand, the only changes needed to the TCP proxying directives(not explained here) are the following 2 requirements:
1) HAProxy Backend configuration includes the extra entry: source 0.0.0.0 usesrc clientip
2) The backend server network settings needs to be configured to have the HaProxy host IP address as the default Gateway.

This way the backend server sees the source IP of the client as if the client connected directly to the backend server and the responses from the backend server are returned via the HAProxy Host.
To be continued soon with practical examples …..

Happy load-balancing đŸ™‚

17 Oct 15 Installing pure-ftpd in Debian/Ubuntu

Difficulty with FTP servers and firewall:
If you configure a firewall for a host which runs an FTP server you normally need to leave the ports 1024-65365 range open, since you never know which port the FTP server will use to send data to the FTP client. This situation is quite critical if you have a host which has sensitive ports above 1024 which need to be closed to Internet. Of course you can select each port and close it in the firewall, but I definitely prefer using the firewall method which closes everything and opens only the ports that are needed access from Internet. Here is where pure-ftpd come to the rescue. This FTP server has the capability to select the range of ports which will be used for transferring data to the FTP client. This makes the configuration of a firewall much easier.

In the following example, pure-ftp has the following configuration:
– provides FTP and FTPS with jailed Users(Users are confined to their home directory).
– no anonymous clients
– IP Version 4 only
– ports for data transfer are limited to the range 20000-20099

STEPS:
apt-get install pure-ftpd
echo '20000 20099' > /etc/pure-ftpd/conf/PassivePortRange
echo "yes" > /etc/pure-ftpd/conf/NoAnonymous
echo "yes" > /etc/pure-ftpd/conf/ChrootEveryone
echo "yes" > /etc/pure-ftpd/conf/IPV4Only
echo "1" > /etc/pure-ftpd/conf/TLS

If you want to force clients to use TLS only for FTP connections then use the command
echo "3" > /etc/pure-ftpd/conf/TLS

Exceptions to chroot
If you want to confine all users to their home directories EXCEPT some trusted users, you need to:
– create a new system group where you add the trusted users in it
– instead of using the above command ‘echo “yes” > /etc/pure-ftpd/conf/ChrootEveryone’
insert the the GID of the trusted group into the file /etc/pure-ftpd/conf/TrustedGID.
Example: We want chroot for all users except ‘martin’ and ‘jannine’. Meaning martin and jannine will be able to navigate in other parts of the system other than their home directories, but all other users will be confined to their home directories:
groupadd ftptrusted
usermod -G ftptrusted martin
usermod -G ftptrusted jannine
GID=$(grep ftptrusted /etc/group | cut -d: -f3)
echo "$GID" > /etc/pure-ftpd/conf/TrustedGID
rm /etc/pure-ftpd/conf/ChrootEveryone

NOTE: To create a properly authority signed certificate file for pure-ftpd, make sure you have both following components in the file /etc/ssl/private/pure-ftpd.pem:
– Private key (in PEM format)
– Certificate (in PEM format)

If instead you want to run it with a self-signed certificate then run the following commands:
mkdir -p /etc/ssl/private/
openssl req -x509 -nodes -days 97300 -newkey rsa:2048 -keyout /etc/ssl/private/pure-ftpd.pem -out /etc/ssl/private/pure-ftpd.pem
chmod 600 /etc/ssl/private/pure-ftpd.pem

Restart pure-ftpd to register the new configuration and certificate.
service pure-ftpd restart

26 Aug 15 Fine tune Ubuntu TCP stack for web server

The following tips taken from the site will help reduce the TCP latency of Ubuntu as a web server :
http://www.cyberciti.biz/faq/linux-tcp-tuning/

 

 

25 Aug 15 Limiting the number of connected clients on a VirtualHost in Apache

Problem:
When a DDOS attack or a burst of requests are coming at the same time in my Apache2 Web server, the whole server can run out of RAM and crash.

Possible solution:
Limit the number of simultaneous connections to your Web server per VirtualHost

Method: One simple and effective method done directly on the Apache web server is by using the modules : mod_bw(mod_bandwidth) and mod_vhost_limit. I prefer the mod_vhost_limit since it is much easier to configure and has proved more effective according to my tests.

Environment:
Compatible with Apache 2.2 xx and Apache 2.4.xx.
Note:For Apache 2.4.xx, a ‘Patching’ of the original source code must be done before compiling the module.

Steps:
Install the build environment tools:
apt-get install build-essential apache2-dev
Download the module sources and extract it.
wget http://apache.ivn.cl/files/source/mod_vhost_limit-0.2.tgz
tar fvxz mod_vhost_limit-0.2.tgz

ONLY FOR Apache 2.4.xx
Getting the patch and patching the original source.
Ref: https://github.com/pld-linux/apache-mod_vhost_limit
wget https://github.com/pld-linux/apache-mod_vhost_limit/archive/master.zip
unzip master.zip
cp apache-mod_vhost_limit-master/* mod_vhost_limit-0.2/
cd mod_vhost_limit-0.2/
patch mod_vhost_limit.c < mod_vhost_limit-apache24.patch
cd ..

FOR BOTH Apache 2.2..xx and Apache 2.4.xx
Compile, install and enable the module:
cd mod_vhost_limit-0.2
/usr/bin/apxs2 -i -a -c mod_vhost_limit.c
service apache2 restart

Use the module in a VirtualHost configuration:
<VirtualHost ......>
...........
# Limits the concurrent requests to 1000 for this vhost
<IfModule vhost_limit_module>
MaxVhostClients 1000
</IfModule>
...........
</VirtualHost>

24 Aug 15 Using mod_cluster in Apache

The newly discovered Apache module mod_cluster seems to offer many advantages compared to mod_jk which could be used in the new Apache/Jboss environment.

The following features are listed:
– Dynamic configuration of httpd workers
– Server-side load balance factor calculation
– Fine grained web-app lifecycle control
– AJP is optional
– Compatible staring at JBoss 4.2.x onwards

Other useful links:
http://www.jboss.org/mod_cluster
http://anonsvn.jboss.org/repos/mod_cluster/tags/release/
http://docs.jboss.org/mod_cluster/1.2.0/html/

Start-up Instructions are found at:
http://docs.jboss.org/mod_cluster/1.2.0/html/Quick_Start_Guide.html#d0e357

TESTING Environment:
List of components of a test environment:
1 VM with Newly compiled Apache & mod_cluster
2 VMs with identical JBoss application

Created three VMs as follows:
Hostname Function IP HTTP-Port AJP-Port
testweb APACHE 192.168.100.1 80 n/a
backend1 JVM1 192.168.100.2 6003 6001
backend2 JVM2 192.168.100.3 6003 6001

Manual testing of direct access to backends:
http://backend1.mydomain.srv:6003/sysnode/
http://backend2.mydomain.srv:6003/sysnode/

Create new Apache vhost as follows:
########################
# Relevant part of httpd.conf
#######################
# Extra modules needed for mod_cluster
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule manager_module modules/mod_manager.so
LoadModule advertise_module modules/mod_advertise.so

Adding to the content of /etc/hosts
192.168.100.1 vhost-01
Content of vhosts.conf
LogLevel debug
Listen vhost-01:80
NameVirtualHost vhost-01:80
#
#------------ Directives for mod_cluster only---------------
<ifmodule proxy_cluster_module>
# Displays the first 100 sessions IDs in manager interface
# Turn on the session ID display of 100 in the mod_cluster-manager page
MaxsessionId 10000
CreateBalancers 2
MaxContext 10000
</ifmodule>
#
# ------ Build the MCPM Connector and advertize connector on Web server
<ifmodule manager_module>
Listen vhost-01:8081
<Virtualhost vhost-01:8081>
# MCPM configuration
KeepAliveTimeout 60
MaxKeepAliveRequests 0
EnableMCPMReceive
#
CustomLog /www/http_logs/MCPM_connector_acc.log j_common
ErrorLog /www/http_logs/MCPM_connector_err.log
</virtualhost>
</ifmodule>
#
#---------- Managemant console interface ----------------------
<ifmodule manager_module>
Listen vhost-01:8000
<Virtualhost vhost-01:8000>
# mod_cluster Manager interface
<Location /mcm>
SetHandler mod_cluster-manager
Order deny,allow
Deny from all
Allow from env=IP_ALLOWED
# Displays verbose extra info about the mod_cluster config
AllowDisplay On
</Location>
CustomLog /www/http_logs/cluster_manager_acc.log j_common
ErrorLog /www/http_logs/cluster_manager_err.log
</virtualhost>
</ifmodule>
#
# --------------------- CLIENT Virtual Host 1----------------------------
#
# http://test1.mydomain.srv
<Virtualhost vhost-01:80>
ServerName test1.mydomain.srv
DocumentRoot /www/htdocs1
#
#------------ Directives for mod_cluster ---------------
<ifmodule proxy_cluster_module>
ProxyRequests Off
UseCanonicalName On
ProxyPreserveHost On
# Allowing all proxy requests
<proxy balancer://jboss.web1>
AddDefaultCharset off
Order deny,allow
Allow from All
</proxy>
ProxyPass / balancer://jboss.web1/ stickysession=JSESSIONID|jsessionid nofailover=On
ProxyPassReverse / balancer://jboss.web1/
</ifmodule>
#
CustomLog /www/http_logs/test1.mydomain.srv_acc.log j_common
ErrorLog /www/http_logs/test1.mydomain.srv_err.log
</virtualhost>
#
# --------------------- CLIENT Virtual Host 2----------------------------
# http://test2.mydomain.srv
<virtualhost vhost-01:80>
ServerName test2.mydomain.srv
DocumentRoot /www/htdocs2
#
#------------ Directives for mod_cluster ---------------
<ifmodule proxy_cluster_module>
ProxyRequests Off
UseCanonicalName On
ProxyPreserveHost On
# Allowing all proxy requests
<proxy balancer://jboss.web2>
AddDefaultCharset off
Order deny,allow
Allow from All
</proxy>
ProxyPass / balancer://jboss.web2/ stickysession=JSESSIONID|jsessionid nofailover=On
ProxyPassReverse / balancer://jboss.web2/
</ifmodule>
#
CustomLog /www/http_logs/test2.mydomain.srv_acc.log j_common
ErrorLog /www/http_logs/test2.mydomain.srv_err.log
</virtualhost>

The mod-cluster have now been tested with the following conditions:
– When an app is finished starting-up, it takes about 3-5 sec before Apache has registered the APP
– The requests sent to Apache were then well balanced between the registered APPs and sent to vhost-asigned APPs custer.

Live upgrade a backend APP servers:
Register all 4 APPs to the same cluster and test how the registration of new APPs and de-registration of running APPs is performing.

Method suggested to do an APP upgrade.
– Start the new apps and wait till they are all up and well running.
– New requests will be directed to old and new apps.
– Disable gracefully the old apps from within the management console. This will stop new requests from going to old apps.
– From this point all new requests will only be sent to the new apps.
– Then the number of still open connections to old apps will be dynamically go shown untill the old apps don’t get any new requests.
– When these numbers for all old apps are down to ‘0’ then you can take the old apps down.

NOTE 1:
During test we had problems to redirect old SSID to new nodes when the old node serving the SSID was not any more available.
The mod_cluster was creating a ‘Service unavailable’ error.

To fix this the following JBOSS properties(postSetEnv.sh) have been set:
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.stickySessionRemove=true"
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.stickySessionForce=false"

Here is the full set of properties related to mod_cluster:
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.balancer=jboss.web1"
(value here has to be modified to a node name variable)
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.proxyList=192.168.100.1:8081"
(values to be set to the proper web server's internal LAN IP)

FIXED VALUES
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.enable=true"
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.advertise=false"
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.excludedContexts="
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.stickySessionRemove=true"
export JAVA_OPTS="${JAVA_OPTS} -Djboss.mod_cluster.stickySessionForce=false"

24 Aug 15 Configuring Apache to handle WebSocks

WebSocks is supported by Apache starting at version 2.4.xx. Here are some minimal configuration for Apache 2.4.xx. in Ubuntu 14.04.x

Install the proper modules:
a2enmod proxy
a2enmod proxy_wstunnel

Configure the VirtualHost (only the WebSock part is shown here)

# Make sure the backend server gets the right URL in 'Location:' http Header
ProxyPreserveHost On
#
# Forward only the websocks requests to the Websocks handler in the backend server
RewriteRule ^/websock/(.*) ws://my.backend.net:8080/websock/$1 [P]
ProxyPassReverse / ws://my.backend.net:8080/
#
# Forward the rest of the requests to the backend server as http requests
ProxyPass / http://my.backend.net:8080/
ProxyPassReverse / http://my.backend.net:8080/