msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

30 Oct 16 Resolving Mysql error: Too many open files

Introduction:
As I upgraded from Mysql 5.5 to 5.6 suddenly some sites were showing the following error:
...... Too many open files
The issue has to do with the present limitations given to the system and PAM system to open max 1024 files. After doing some research I found this site below here which is in German in which the following is explained:
https://www.filewalker.de/ulimit-mysql-open_files_limit/

STEPS:
Check the files-open limits or running mysql server:
mysql -p -u root
mysql> SHOW VARIABLES LIKE 'open%';

The very possible output:
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 1024 |
+------------------+-------+
1 row in set (0.00 sec)

That means that Mysql server gets to open maximum 1024 which seems too little for Mysql 5.6.

Raising this limit
Edit the file /etc/security/limits.conf and add the following lines:
mysql hard nofile 65535
mysql soft nofile 65535

This will raise the limit of open files to 65535 for the user mysql only.
If you want to rise this limit for all users then replace the word mysql for *
eg.
* hard nofile 65535
* soft nofile 65535

And according to this site edit the file /etc/pam.d/common-session and add this line at the end:
session required pam_limits.so
Note: I’m not sure this step is really needed though. Some people did try without it and it also worked. For me, in Debian Wheezy, I had to do this otherwise I was still getting the error.

For systems that run systemd instead of InitV do the following:
Edit file /usr/lib/systemd/system/mysqld.service
Add these 2 lines at the end:
LimitNOFILE=65535
LimitNPROC=65535

Restart Mysql server and test it again
service mysql restart
mysql -p -u root
mysql>> SHOW VARIABLES LIKE 'open%';

The hopeful output:
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| open_files_limit | 65535 |
+------------------+-------+
1 row in set (0.00 sec)

This error should no more appear.

21 Mar 16 Redirecting HTTP to HTTPS in NginX

Here is a working method of redirecting any requested HTTP URL to HTTPS in NginX VirtualHosts that handles both HTTP and HTTPS.
For example, to have a single vhost support both HTTP and HTTPS you have normally the following directives:
# Support for HTTP and HTTPS
listen 80;
listen 443 ssl;
ssl_certificate /etc/letsencrypt/live/www.myserver.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.myserver.com/privkey.pem;

Then to redirect all HTTP requests to HTTPS within this vhost without creating infinite loops you add the following redirect:
if ($scheme != "https") {rewrite ^ https://$host/$request_uri permanent;}
Other methods can be seen here:
http://serverfault.com/questions/67316/in-nginx-how-can-i-rewrite-all-http-requests-to-https-while-maintaining-sub-dom

09 Mar 16 Testing SSL Connections with SSLyze, Nmap or OpenSSL

Introduction:
OpenSSL is a great tool to check SSL connections to servers. The difficulty here is when one want a full scan of all possible SSL Cyphers and protocols used by a server. That is where SSLyze comes in handy. This tool is a Python script which will scan the target host/port for SSL handshake and report what works/support and what not. Unfortunately this lovely tool is not included in the Ubuntu/Debian distributions, and this is where this post comes handy.

IMPORTANT: Besides executing all the tests below one thing very important (as noted in the This link) is to upgrade OpenSSL to the latest version as follows:
OpenSSL 1.0.2 users should upgrade to 1.0.2g
OpenSSL 1.0.1 users should upgrade to 1.0.1s

SSLyze

Installing the dependencies and tool
cd /root/bin
wget https://github.com/nabla-c0d3/sslyze/archive/0.13.4.tar.gz
tar fvxz 0.13.4.tar.gz
apt-get install python-pip python-dev
pip install nassl

Using SSLyze
python /root/bin/sslyze-0.13.4/sslyze_cli.py --regular www.itmatrix.eu:443

NMAP

Scanning the full server for weaknesses including weak SSL Versions using NMAP.
Note: This operation can take a long time to execute.
apt-get install nmap
nmap -sV -sC www.itmatrix.eu

OR better(for checking the HTTPS,SMTPS,IMAPS,POP3S)
nmap --script ssl-cert,ssl-enum-ciphers -p 443,465,993,995 www.itmatrix.eu

OpenSSL

Checking the SSL connection with OpenSSL
echo 'q' | openssl s_client -host www.itmatrix.eu -port 443
Note: In this above case since the SSLv2 support is normally disabled for OpenSSL in Debian/Ubuntu distributions, you will not be able to see if the server is supporting it. To overcome this and enable SSLv2 support(for your testing Linux) then follow the instructions in this site:
http://www.hackwhackandsmack.com/?p=46

NOTE:
For more information regarding protection against DROWN(SSLv2) or POODLE(SSLv3) attacks see:
https://drownattack.com
http://www.softwaresecured.com/2016/03/01/how-to-confirm-whether-you-are-vulnerable-to-the-drown-attack/
http://www.mogilowski.net/lang/de-de/2014/10/23/disabling-sslv3-for-poodle-on-debian/
https://www.owasp.org/index.php/Testing_for_Weak_SSL/TLS_Ciphers,_Insufficient_Transport_Layer_Protection_%28OTG-CRYPST-001%29
https://zmap.io/sslv3/

17 Dec 15 Issue free and CA signed SSL certificates for web servers from LetsEncrypt

Introduction:
SSL Certificates provide two functions:
1. Authentication
2. Encryption

Encryption can be achieved without authentication but, for some reason, someone decided to join them together in one certificate. It seem to make sense for banks and serious e-commerce sites which need to be properly authenticated. Therefore when the HTTPS protocol got developed it was not possible to encrypt-only the stream of HTTP. This situation made us dependent to Certificate Authentication Authorities to obtain a certificate even if we only wanted encryption. Now some genius group of people at https://letsencrypt.org/ finally created the possibility to obtaining certificates which preform simple authentication verification, by calling the URL and expecting a specific response, and if successful issues a free 90 days valid and CA signed SSL certificate. For system administrators this process of requesting and install such free certificate has therefore become quite simple. Here is one method of doing just this in a Debian/Ubuntu web server.
References: http://www.admin-magazine.com/Articles/Getting-a-free-TLS-certificate-from-Let-s-Encrypt?utm_source=ADMIN+Newsletter&utm_campaign=ADMIN_Update_Free_Certificates_with_Let%27s_Encrypt_2016-20-07&utm_medium=email

STEPS:

Installing LetsEncrypt

apt-get update && apt-get install git
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto --email user@mydomain.com --agree-tos --help
echo "export PATH=$PATH:/usr/local/lib/letsencrypt" >> /root/.bashrc
. /root/.bashrc

NOTE: Make sure your web site you want to add HTTPS to is already configured and live in your web server.
The reason is that during the process of requesting a certificate, LetsEncrypt will create an extra sub-directory({htdocs}/.well-known/acme-challenge/) and a special temporary file in the htdocs of the site (pointed to by DocumentRoot directive in Apache) then call that file on the site from the LetsEncrypt server to authenticate the URL. If the the URL called is invalid it won’t issue the certificate. For this reason your site needs to be live and you need to give the path of the htdocs. After the authentication process, the temporary file will be erased but not the sub directories. They will stay empty.

Troubleshooting:

InsecurePlatformWarning
If you get the following error message then in Debian Wheezy you can solve it by importing SSl into Python. See below.
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

Importing Python SSL support:
python
>>> import ssl
>>> (CTRL-D)

Upgrading LetsEncrypt client program

rm -rf /usr/local/lib/letsencrypt.old &>/dev/null
mv /usr/local/lib/letsencrypt /usr/local/lib/letsencrypt.old
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt

Requesting the certificate

Eg. for the domain blog.mydomain.com
NOTE: on first time request the script will ask you to give an email address for contact purposes as well as to accept the terme and conditions of using this tool. Afterwards it will not ask you these questions.
letsencrypt-auto certonly --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com
The certificates and key will be stored in /etc/letsencrypt/live/blog.mydomain.com/ as:
cert.pem : Certificate
chain.pem : CA Certificate
privkey.pem : Private key
fullchain.pem : Combination of the certificate and the CA Certificate

Instead of moving the certificate, just configure Apache or other web server to point to the certs files where they are.
This way a cron job can be created to regularly renew the certificate automatically without manual intervention.
The certificate will be valid for 90 Days only; no exceptions.
This means that the same above command will need to be run every 3 months or earlier with the addition of the option –renew-by-default.
The limit of certificates you can ask for a certain domain is: currently 5 certificates / 7 days.

Renewing single certificate:

In order to renew the certificate automatically it is suggested to use a cron job and adding the option –renew-by-default in the command eg. as follows:
letsencrypt-auto certonly --renew-by-default --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com

Renewing all installed Letsencrypt certificates:

letsencrypt-auto renew
Note: It is recommended to send the output of the command by email to verify if the process was successful.

Extra Info

The certificates of LetsEncrypt are stored in /etc/letsencrypt/ directories in different ways. It is simply NOT recommended to delete any of the certificates, files or symlinks in these directories because the files in the ‘keys’ and ‘csr’ directories are not identified to refer to a specific certificate. So just deleting some files but not others related to the same cert might confuse the client command and you then can’t request any more certificates. The error message from the client program is something like:
letsencrypt TypeError: coercing to Unicode: need string or buffer, NoneType found
If you ever get to that non-return point then just delete all directories: archive, csr, keys, live and renewal BUT not accounts. Then re-issue certificates requests for already existing sites. The certificates will then be renewed and you can then also request new ones.

For more information of the subject see:
https://letsencrypt.readthedocs.org/en/latest/using.html

Comfortable script

If you want to be able to issue a certificate and you want it to self-renew after 80 days, this script might be of some use.
#!/bin/bash
# Purpose: Issue or renew a certificate from LetsEncrypt
# It will also issue an 'at'command which will be responsible to automatically renew the certificate automatically
# This script also issues a new at comand which will do the same in around 3 Months days depending on the settings here
# Syntax: cert_request.sh -s SITE_NAME -d SITE_HTDOCS
# Changes: 30.12.2015 First implementation of the script
# 10.01.2016 Took out the read of wpinstall.cfg config file. Added checks for the letsencrypt and at programs
#--------------------------------------------------------------
. /root/.bashrc
RENEW_DAYS="80"
# Absolute path to this script.
SCRIPT=$(readlink -f $0)
CERTS_DIR="/etc/letsencrypt/live"
# Absolute path this script is in.
scriptdir=$(dirname $SCRIPT)
encryptprgm="/usr/local/lib/letsencrypt/letsencrypt-auto"
atprgm="/usr/bin/at"
EMAIL="admin@itmatrix.eu"

# Check the syntax
function usage () {
echo "Usage: cert_request.sh -s SITE_NAME -d SITE_HTDOCS"
echo "-s SITE_NAME Full web site address WITHOUT the 'http://' eg.: www.myblog.com"
echo "-d SITE_HTDOCS The absolute path where WordPress will be installed. eg. /www/sites/www.mysite.com/htdocs"
exit 1
}

if [ $# -ne 4 ]; then
echo "ERROR: Wrong number of given argunents."
usage
fi

# Make sure the letsencrypt client prgm is installed
if ! [ -e $encryptprgm ] ; then
echo "ERROR: the letsencrypt program isn not installed. Install it and retry."
echo "See instructions at: //tipstricks.itmatrix.eu/install-new-and-signed-ssl-certificate-for-web-servers"
exit 1
fi

# Make sure the at is installed
if ! [ -e $atprgm ] ; then
echo "ERROR: the 'AT' program isn not installed. Install it and retry."
echo "apt-get install at"
exit 1
fi

# Everything look good so far. Lets start.

# get the command options
while getopts "s:d:" OPTION
do
case $OPTION in
s) SITE_NAME=$OPTARG
;;
d) SITE_HTDOCS=$OPTARG
;;
h|?|*)
echo "ERROR: argument(s) unknown."
usage
;;
esac
done

echo "Requesting certificate at LetsEncrypt"
# Does it exist already, then renew only, otherwise request renewing the cert
if [ -d $CERTS_DIR/${SITE_NAME} ] ; then
echo "The certificate already exists. Requesting a renewal"
RENEW="--renew-by-default"
else
RENEW=""
fi

if ($encryptprgm certonly $RENEW --webroot -w $SITE_HTDOCS -d ${SITE_NAME} &>/dev/null); then
# Enable the Apache SSL configuration and restart Apache
(echo "Certificate request successful."
echo "Issuing a renewal of the certificate in 80 days using 'at' command"
service apache2 restart
echo "$SCRIPT -s $SITE_NAME -d $SITE_HTDOCS" | $atprgm now + $RENEW_DAYS days)| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" admin@itmatrix.eu
echo -e "------- SITES LIST --------\n$(ls -1 /etc/apache2/sites-enabled/ | egrep -v '^00|^wptest1')\n\n--------- CERTIFICATES LIST ---------$(ls -l /etc/letsencrypt/live/ | cut -c29-)\n\n------- AT Jobs LIST -------\n$(/root/bin/atlist.sh)" | mail -s "Request/Renewal of Certificate request LIST" $EMAIL
cat /tmp/cert_request.sh.log
exit 0
else
(echo "ERROR: The certificate request/renewal FAILED.")| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" $EMAIL
cat /tmp/cert_request.sh.log
exit 2
fi

Status of the certificates

Here is a useful script that will display the following information:
– List of AT Jobs ready to start at the required time
– List of present certificates and their file timestamps
Since each letsencrypt is only valid for 90 days this will give you an overview of how old the present certificates are and when is the next time the requests for certificates will be done.
#!/bin/bash
# Description: Displays all 'at' jobs and their respective commands
# Systax: atlist.sh
# Changes: 05.11.2016 First inplementation
########################################################################
# Get the short jobs list and expand from there
echo "================ AT Jobs ready to start at the required times ==============="
atq | while read line ; do
jobnr=$(echo $line | awk '{print $1}')
echo $line
# Pickup all the command lines after first line matching '}'.
# This excludes all the environment variables and the at exit line.
# Exclude the matching '}' line and empty lines
# Add an offset of 8 chars to each command line.
# at -c $jobnr | grep -A100 -m1 -e '^\}' | grep -v '^\}' | sed -e '/^$/d' -e 's/^/ /'
at -c $jobnr | at -c $jobnr | sed -e '1,/^\}/d' -e '/^$/d' -e 's/^/ /'
done
echo ; echo
echo "=============== Age of present certificates ====================="
ls -l /etc/letsencrypt/live/*/cert.pem | awk '{print $6" "$7" "$8" "$9}' | sed -e 's|/etc/letsencrypt/live/||' -e 's|/cert.pem||'

10 Dec 15 Creating a web certificate CSR file.

The process of buying an SSL certificate for a web site is usually as follows:
– You create a secret key and CSR files using the method showm in this post.
– You cut and paste the content of the CSR file into a field in a SSL Vendor web site
– The SSL vendor produces a certificate based on the CSR you provided and send it to you.
– You download the CA Certificate from the SSL provider’s site
– You install the private keyfile, the CA certificate and the certificate in the web server and bobs’s-your-uncle.

The following procdeure is an extract from the site:
https://support.globalsign.com/customer/portal/articles/1221018-generate-csr—openssl
Generate a CSR & Private Key:
openssl req -out CSR.csr -new -newkey rsa:2048 -nodes -keyout privatekey.key

Fill out the following fields as prompted:
Note: The following characters can not be accepted: <> ~ ! @ # $ % ^ * / \ ( ) ?.,&


Field Example
============ ==========================================
Country Name US (2 Letter Code)
State or Province New Hampshire (Full State Name)
Locality Portsmouth (Full City name)
Organization GMO GlobalSign Inc (Entity's Legal Name)
Organizational Unit Support (Optional, e.g. a department)
Common Name www.globalsign.com (Domain or Entity name

18 Aug 15 Customizing apache2-suexec-custom

Apache2 Suexec module comes with at least 2 flavors in many distributions of Linux.
– apache2-suexec-pristine: Apache HTTP Server standard suexec program for mod_suexec
– apache2-suexec-custom: Apache HTTP Server configurable suexec program for mod_suexec

Many times when I’ve been trying to use the standard mod_suexec with mod_fcgi (useful for WordPress installations),the mod_suexec complains that the binary I’m trying to execute is outside the allowed path of mod_suexec. Obviously the mod_suexec has been compiled with a specific allowed path where I can execute commands which happens to NOT be the one I’m using. Therefore I reverted to installing the package apache2-suexec-custom instead. This module is configurable as far as the allowed path where I can run programs.
This configuration file is found in /etc/apache2/suexec/www-data.
Its initial content in Ubuntu 14.04 is as follows:
/var/www
public_html/cgi-bin
# The first two lines contain the suexec document root and the suexec userdir
# suffix. If one of them is disabled by prepending a # character, suexec will
# refuse the corresponding type of request.
# This config file is only used by the apache2-suexec-custom package. See the
# suexec man page included in the package for more details.

Here is what part of the man page says about it. The rest is self explanatory.

If suexec is called by a user with name ‘username’, it will look into /etc/apache2/suexec/username for configuration. If the file does not exist, suexec will abort. By creating several config files, you can allow several different apache run users to use suexec.

The first line in the file is used as the document root (/var/www in the standard suexec) and the second line in the file is used as the suffix that is appended to users’ home directories (public_html in standard suexec).

If any of the lines is commented out (with #), suexec will refuse the corresponding type of request. It is recommended to comment out the userdir suffix if you don’t need it.

IMPORTANT:
suexec requires the CGI script to be under the server’s DocumentRoot (not the VirtualHost DocumentRoot). It is permitted, however, for the VirtualHost DocumentRoot to be a symlink to a directory that appears under the real DocumentRoot.
– The wrapper script should be of chmod 750

Tips for using suexec
– Suexec requires that user cgi scripts not be writable by others, e.g., be 755
– Suexec requies that the directory containing a cgi script not be writable by others, e.g., be 755 and not 775.
– If suexec fails to execute a user’s script, /var/log/httpd/error_log will only indicate “Premature end of script headers”; however, /var/log/httpd/suexec_log will indicate the actual problem, e.g., “error: directory is writable by others”

15 Aug 15 Installing wp_cli

wp_cli is a really good PHP script which helps installing the latest WordPress version in an htdocs.
Here is how to install it in a Linux system:
This will install wp_cli PHP script as /usr/local/bin/wp in the system.
(Logged in as root)
cd /tmp
wget https://raw.github.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
chmod 755 wp-cli.phar
mv wp-cli.phar /usr/local/bin/wp

To get the help screen:
/usr/local/bin/wp --allow-root
To get the info screen:
/usr/local/bin/wp --info --allow-root
Find more information on wp_cli at: http://wp-cli.org/

WARNING!! for the sake of simplicity and laziness I was logged in as root BUT I when you start using this script I recommend strongly to run it inside the intended htdocs as the Apache user. Normally this is the user which Apache runs under when running the php scripts in a suexec/fcgi configuration.

NOTE: Unfortunately from WordPress version 4.4 the older version of wp_cli will create the following error message which prevents wp_cli to automatically configure wp-config.php.
PHP Fatal error: Class 'WP_Widget' not found in
No big deal but you will have to spend an extra 30 sec. to configure WordPress in its web interface before being able to login.
Solution: Upgrade wp_cli to the latest version by simply re-running the initial installation steps.
See this link for more info: http://wp-cli.org/blog/versions-0.21.1-and-0.20.4.html

08 Aug 15 Install a multisite(WPMU) WordPress with wp_cli

Prerequisites:
– Create a new fcgi driven user in Linux system(in this case usrblog)
useradd -s /bin/bash -d /www/clients/mywpsite.com/htdocs/ usrblog
passwd usrblog

– Install a Virtual host which uses suexec and fcgi in Apache/NginX
– Create a new database in MySQL (we will call it myblog)
– Create a new mysql user and assign the database(myblog) access and control to this user.
– Create web space(DocumentRoot) in system which will be owned by the system user usrblog

Continuing the installation using the wp_cli script
– Install the script as per: http://wp-cli.org/
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
chmod +x wp-cli.phar
sudo mv wp-cli.phar /usr/local/bin/wp

Switch user to usrblog
su - usrblog
wp core download
wp core config --dbhost=dbhost --dbname=dbname --dbuser=username --dbpass=password
chmod 644 wp-config.php
wp core multisite-install --subdomains --url=yourwebsite.com --title="Your Blog Title" --admin_name=wordpress_admin --admin_password=4Long&Strong1 --admin_email=you@example.com

The rest is done via the web administration site of the wordpress:
– http://mysite.com/wp-admin
– Select: my site ==> Network Admin ==> Dashboard ==> Settings ==> Network Setup
Copy the rewrite and paste the presented rewrite rules into an new .htaccess file in the htdocs of the site.