msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

02 Jan 18 Configuring Letsencrypt in ISPConfig 3.1

Intro:
Since a while now the wonderful idea of creating the service Letsencrypt has made lots of admins happy.
Here is how we can also use Letsencrypt with ISPConfig 3.1.

Ref: https://www.howtoforge.com/community/threads/ssl-how-to-for-ispconfig-3-with-letsencrypt.74738/

STEPS:
Define ISPconfig to use the new SSL certificate with symbolic links.
(If you don’t know how to use symbolic links this how-to is not for you)
/usr/local/ispconfig/interface/ssl/
ispserver.crt -> /etc/letsencrypt/live/mydomain.com/fullchain.pem
ispserver.key -> /etc/letsencrypt/live/mydomain.com/privkey.pem

Define Postfix to use the new SSL certificate in /etc/postfix/main.cf.
(If you don’t know how to add these entries this how-to is not for you)
smtpd_tls_cert_file = /etc/letsencrypt/live/mydomain.com/cert.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mydomain.com/privkey.pem
smtpd_tls_CAfile = /etc/letsencrypt/live/mydomain.com/fullchain.pem

Define Dovecot to use the new SSL certificate in /etc/dovecot/dovecot.conf.
(If you don’t know how to add these entries this how-to is not for you)
ssl_cert = </etc/letsencrypt/live/mydomain.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mydomain.com/privkey.pem

# Restart/reload the services involved
sudo service postfix reload
sudo service dovecot reload
sudo service apache2 restart

17 Dec 15 Issue free and CA signed SSL certificates for web servers from LetsEncrypt

Introduction:
SSL Certificates provide two functions:
1. Authentication
2. Encryption

Encryption can be achieved without authentication but, for some reason, someone decided to join them together in one certificate. It seem to make sense for banks and serious e-commerce sites which need to be properly authenticated. Therefore when the HTTPS protocol got developed it was not possible to encrypt-only the stream of HTTP. This situation made us dependent to Certificate Authentication Authorities to obtain a certificate even if we only wanted encryption. Now some genius group of people at https://letsencrypt.org/ finally created the possibility to obtaining certificates which preform simple authentication verification, by calling the URL and expecting a specific response, and if successful issues a free 90 days valid and CA signed SSL certificate. For system administrators this process of requesting and install such free certificate has therefore become quite simple. Here is one method of doing just this in a Debian/Ubuntu web server.
References: http://www.admin-magazine.com/Articles/Getting-a-free-TLS-certificate-from-Let-s-Encrypt?utm_source=ADMIN+Newsletter&utm_campaign=ADMIN_Update_Free_Certificates_with_Let%27s_Encrypt_2016-20-07&utm_medium=email

STEPS:

Installing LetsEncrypt

apt-get update && apt-get install git
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt
./letsencrypt-auto --email user@mydomain.com --agree-tos --help
echo "export PATH=$PATH:/usr/local/lib/letsencrypt" >> /root/.bashrc
. /root/.bashrc

NOTE: Make sure your web site you want to add HTTPS to is already configured and live in your web server.
The reason is that during the process of requesting a certificate, LetsEncrypt will create an extra sub-directory({htdocs}/.well-known/acme-challenge/) and a special temporary file in the htdocs of the site (pointed to by DocumentRoot directive in Apache) then call that file on the site from the LetsEncrypt server to authenticate the URL. If the the URL called is invalid it won’t issue the certificate. For this reason your site needs to be live and you need to give the path of the htdocs. After the authentication process, the temporary file will be erased but not the sub directories. They will stay empty.

Troubleshooting:

InsecurePlatformWarning
If you get the following error message then in Debian Wheezy you can solve it by importing SSl into Python. See below.
InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning

Importing Python SSL support:
python
>>> import ssl
>>> (CTRL-D)

Upgrading LetsEncrypt client program

rm -rf /root/.local/share/letsencrypt
rm -rf /usr/local/lib/letsencrypt.old &>/dev/null
mv /usr/local/lib/letsencrypt /usr/local/lib/letsencrypt.old
cd /usr/local/lib/
git clone https://github.com/letsencrypt/letsencrypt

Requesting the certificate

Eg. for the domain blog.mydomain.com
NOTE: on first time request the script will ask you to give an email address for contact purposes as well as to accept the terme and conditions of using this tool. Afterwards it will not ask you these questions.
letsencrypt-auto certonly --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com
The certificates and key will be stored in /etc/letsencrypt/live/blog.mydomain.com/ as:
cert.pem : Certificate
chain.pem : CA Certificate
privkey.pem : Private key
fullchain.pem : Combination of the certificate and the CA Certificate

Instead of moving the certificate, just configure Apache or other web server to point to the certs files where they are.
This way a cron job can be created to regularly renew the certificate automatically without manual intervention.
The certificate will be valid for 90 Days only; no exceptions.
This means that the same above command will need to be run every 3 months or earlier with the addition of the option –renew-by-default.
The limit of certificates you can ask for a certain domain is: currently 5 certificates / 7 days.

Renewing single certificate:

In order to renew the certificate automatically it is suggested to use a cron job and adding the option –renew-by-default in the command eg. as follows:
letsencrypt-auto certonly --renew-by-default --webroot -w /www/clients/blog.mydomain.com/htdocs -d blog.mydomain.com

Renewing all installed Letsencrypt certificates:

/usr/local/lib/letsencrypt/letsencrypt-auto renew
Note: It is recommended to send the output of the command by email to verify if the process was successful.

Extra Info

The certificates of LetsEncrypt are stored in /etc/letsencrypt/ directories in different ways. It is simply NOT recommended to delete any of the certificates, files or symlinks in these directories because the files in the ‘keys’ and ‘csr’ directories are not identified to refer to a specific certificate. So just deleting some files but not others related to the same cert might confuse the client command and you then can’t request any more certificates. The error message from the client program is something like:
letsencrypt TypeError: coercing to Unicode: need string or buffer, NoneType found
If you ever get to that non-return point then just delete all directories: archive, csr, keys, live and renewal BUT not accounts. Then re-issue certificates requests for already existing sites. The certificates will then be renewed and you can then also request new ones.

For more information of the subject see:
https://letsencrypt.readthedocs.org/en/latest/using.html

Comfortable script

If you want to be able to issue a certificate and you want it to self-renew after 80 days, this script might be of some use.
#!/bin/bash
# Purpose: Issue or renew a certificate from LetsEncrypt
# It will also issue an 'at'command which will be responsible to automatically renew the certificate automatically
# This script also issues a new at comand which will do the same in around 3 Months days depending on the settings here
# Syntax: cert_request.sh -s SITE_NAME -d SITE_HTDOCS
# Changes: 30.12.2015 First implementation of the script
# 10.01.2016 Took out the read of wpinstall.cfg config file. Added checks for the letsencrypt and at programs
#--------------------------------------------------------------
. /root/.bashrc
RENEW_DAYS="80"
# Absolute path to this script.
SCRIPT=$(readlink -f $0)
CERTS_DIR="/etc/letsencrypt/live"
# Absolute path this script is in.
scriptdir=$(dirname $SCRIPT)
encryptprgm="/usr/local/lib/letsencrypt/letsencrypt-auto"
atprgm="/usr/bin/at"
EMAIL="admin@itmatrix.eu"
#
# Check the syntax
function usage () {
echo "Usage: cert_request.sh -s SITE_NAME -d SITE_HTDOCS"
echo "-s SITE_NAME Full web site address WITHOUT the 'http://' eg.: www.myblog.com"
echo "-d SITE_HTDOCS The absolute path where WordPress will be installed. eg. /www/sites/www.mysite.com/htdocs"
exit 1
}
#
if [ $# -ne 4 ]; then
echo "ERROR: Wrong number of given argunents."
usage
fi
# Make sure the letsencrypt client prgm is installed
if ! [ -e $encryptprgm ] ; then
echo "ERROR: the letsencrypt program isn not installed. Install it and retry."
echo "See instructions at: //tipstricks.itmatrix.eu/install-new-and-signed-ssl-certificate-for-web-servers"
exit 1
fi
# Make sure the at is installed
if ! [ -e $atprgm ] ; then
echo "ERROR: the 'AT' program isn not installed. Install it and retry."
echo "apt-get install at"
exit 1
fi
# Everything look good so far. Lets start.
# get the command options
while getopts "s:d:" OPTION
do
case $OPTION in
s) SITE_NAME=$OPTARG
;;
d) SITE_HTDOCS=$OPTARG
;;
h|?|*)
echo "ERROR: argument(s) unknown."
usage
;;
esac
done
echo "Requesting certificate at LetsEncrypt"
# Does it exist already, then renew only, otherwise request renewing the cert
if [ -d $CERTS_DIR/${SITE_NAME} ] ; then
echo "The certificate already exists. Requesting a renewal"
RENEW="--renew-by-default"
else
RENEW=""
fi
#
if ($encryptprgm certonly $RENEW --webroot -w $SITE_HTDOCS -d ${SITE_NAME} &>/dev/null); then
# Enable the Apache SSL configuration and restart Apache
(echo "Certificate request successful."
echo "Issuing a renewal of the certificate in 80 days using 'at' command"
service apache2 restart
echo "$SCRIPT -s $SITE_NAME -d $SITE_HTDOCS" | $atprgm now + $RENEW_DAYS days)| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" admin@itmatrix.eu
echo -e "------- SITES LIST --------\n$(ls -1 /etc/apache2/sites-enabled/ | egrep -v '^00|^wptest1')\n\n--------- CERTIFICATES LIST ---------$(ls -l /etc/letsencrypt/live/ | cut -c29-)\n\n------- AT Jobs LIST -------\n$(/root/bin/atlist.sh)" | mail -s "Request/Renewal of Certificate request LIST" $EMAIL
cat /tmp/cert_request.sh.log
exit 0
else
(echo "ERROR: The certificate request/renewal FAILED.")| tee /tmp/cert_request.sh.log \
| mail -s "Request/Renewal of Certificate for $SITE_NAME" $EMAIL
cat /tmp/cert_request.sh.log
exit 2
fi

#

Status of the certificates

Here is a useful script that will display the following information:
– List of AT Jobs ready to start at the required time
– List of present certificates and their file timestamps
Since each letsencrypt is only valid for 90 days this will give you an overview of how old the present certificates are and when is the next time the requests for certificates will be done.
#!/bin/bash
# Description: Displays all 'at' jobs and their respective commands
# Systax: atlist.sh
# Changes: 05.11.2016 First inplementation
########################################################################
# Get the short jobs list and expand from there
echo "================ AT Jobs ready to start at the required times ==============="
atq | while read line ; do
jobnr=$(echo $line | awk '{print $1}')
echo $line
# Pickup all the command lines after first line matching '}'.
# This excludes all the environment variables and the at exit line.
# Exclude the matching '}' line and empty lines
# Add an offset of 8 chars to each command line.
# at -c $jobnr | grep -A100 -m1 -e '^\}' | grep -v '^\}' | sed -e '/^$/d' -e 's/^/ /'
at -c $jobnr | at -c $jobnr | sed -e '1,/^\}/d' -e '/^$/d' -e 's/^/ /'
done
echo ; echo
echo "=============== Age of present certificates ====================="
ls -l /etc/letsencrypt/live/*/cert.pem | awk '{print $6" "$7" "$8" "$9}' | sed -e 's|/etc/letsencrypt/live/||' -e 's|/cert.pem||'

Installing and using CERBOT

Introduction:
Cerbot tool simply adds more user-friendly features to Lestencrypt original seen above.
See this site for more information:
https://certbot.eff.org/docs/install.html

Use:
It can make a certificate request to Letsencryp server and also create ta temporary web server to allow to verify the HTTP connection of a site for which the certificate is requested. These features can be called simply by adding option on the command line.
For example: If I want a web site to be SSL only then generally the HTTP virtual Host would do a redirection to HTTPS automatically for any HTTP request. In this case it blocks the verification of the web site from Letsencrypt server. There are 2 ways that I know to avoid this difficulty.
1) Create a condition on the redirection directive in Apache that does not redirect to HTTPS if the URI is /.well-known/…..
Pro: No downtime of the web server
Con: An extra condition in the redirection directive must be done for each SSL site.
2) Stop the web server. Run cerbot with the –standalone option . Restart the web server.
This solution would only be good for a web server cluster environment if you don’t want to have downtime on your site.
Pro: No need for extra redirection condition for sites. Really good for Nginx servers where re-directions conditions are difficult to create
Con: The site gets a downtime during this procedure which can last long if you have many sites needing creation/renewal of certificates.

Installation:
wget https://dl.eff.org/certbot-auto
chmod a+x ./certbot-auto

Or if you have Ubuntu 16.04 or later, the following commands will also preferable for the installation:
apt-get install certbot python-certbot-apache
Then to get help on ‘certbot’
certbot --help
An example of a ‘certbot‘ command using its own web server for authentication, this way avoiding the difficulties of having to interfere with Apache/NginX for the authentication process.
The following example will request a single certificate which will be valid for these 4 subdomains using the (SAN) mechanism.
certbot certonly --standalone -d www.mydomain.com -d www.mydomain2.com -d mailman.mydomain.com -d mail.mydomain.com
In this above example you need to make sure Apache/NginX is not using the port 80. If it does, shut the web server down before this command and restart it right after it. It will result in a short web access downtime. This downtime might be acceptable depending on whether you have the web server behind a load -balancer or the web service can allow such downtime.
Example:
service apache2 stop
certbot certonly --standalone -d www.mydomain.com -d www.mydomain2.com -d mailman.mydomain.com -d mail.mydomain.com
service apache2 start

19 Jan 17 Installing TeamPass in Debian Jessie

Introduction:
TeamPass is a very good Web application which can store securely Passwords for single person or teams. Here are the steps I used to install it in Debian Jessie. These instructions can also be used with no or minimal changes to install TeamPass in other Debian or Ubuntu systems.
These instruction are partly based on this site:
http://teampass.net/2013-12-31-installation-on-linux-server
and these
http://bourntech.com/blog/install-teampass-on-ubuntu-14-6lts/
https://github.com/nilsteampassnet/TeamPass/

Steps:
Create the user that will be used as owner of the TeamPass htdocs and Apache TeamPass requests processes.
useradd -d /opt/teampass/ -s /bin/false passwords
Prepare the teampass home directories
mkdir -p /var/www/teampass/fcgi/tmp
mkdir /var/www/teampass/logs
mkdir /var/www/teampass/auth
cd /var/www/teampass/
#Get the latest released software:
wget --no-check-certificate https://github.com/nilsteampassnet/TeamPass/archive/master.zip
unzip master.zip

Install the required packages:
apt-get install php5-mcrypt php5-mysqlnd php5-gd openssl apache2-suexec-custom apache2-mpm-prefork libapache2-mod-fcgid libapache2-mod-php5 php5-cgi mariadb-server
In order to allow Apache to modify files inside the TeamPass htdocs we use FCGI/suexec Modules.
a2enmod fcgid
a2enmod suexec
a2enmod ssl

Create the fcgi_wrapper script:
touch /var/www/teampass/fcgi/php-fcgi-starter
mcedit /var/www/teampass/fcgi/php-fcgi-starter

Content:
#!/bin/sh
export PHPRC=/var/www/teampass/fcgi/
export PHP_FCGI_CHILDREN=2
export PHP_FCGI_MAX_REQUESTS=500
exec /usr/bin/php5-cgi

Make it runnable but not for others:
chmod 750 /var/www/teampass/fcgi/php-fcgi-starter
Copy the php.ini from system to /var/www/teampass/fcgi/
cp /etc/php5/apache2/php.ini /var/www/teampass/fcgi/
Adapt the php.init to the site:
mcedit /var/www/teampass/fcgi/php.ini
Add the following 2 lines at the end:
upload_tmp_dir = /var/www/teampass/fcgi/tmp
session.save_path = /var/www/teampass/fcgi/tmp

And look for the configuration: max_execution_time and change its value from 30 to 60. Eg.
max_execution_time = 60
Create the Apache2 configuration:
Content of config file in /etc/apache2/sites-available/teampass.mydomain.com.conf:
# ============ https://teampass.mydomain.com ==================
<virtualhost *:443>
ServerName teampass.mydomain.com
DocumentRoot /var/www/teampass/TeamPass-master
SuexecUserGroup passwords passwords
<directory /var/www/teampass/TeamPass-master>
Options -Indexes +FollowSymLinks +ExecCGI
FCGIWrapper /var/www/teampass/fcgi/php-fcgi-starter .php
AddHandler fcgid-script .php
DirectoryIndex index.php
Require 192.168. granted
AuthType Basic
AuthName "Private area"
AuthUserFile /var/www/teampass/auth/web.auth
Require valid-user
Satisfy all
</directory>
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/teampass.mydomain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/teampass.mydomain.com/privkey.pem
SSLCACertificateFile /etc/letsencrypt/live/teampass.mydomain.com/chain.pem
ErrorLog /var/www/teampass/logs/error_log
CustomLog /var/www/teampass/logs/access_log combined
</virtualhost>

Create the first layer(BASIC) Authentication credentials for first user:
htpasswd -c /var/www/teampass/auth/web.auth username
Give the whole directory ownership to ‘passwords’ user
chown -R passwords: /var/www/teampass/
NOTE: Before you restart your Apache2 service, make sure the Certificate is been issue and installed in the directory: /etc/letsencrypt/live/teampass.mydomain.com/
You can use the instructions on this link to install LetsEncrypt software:
https://tipstricks.itmatrix.eu/?s=letsencrypt&x=0&y=0

Enable Apache’s new configuration:
a2ensite teampass.mydomain.com
Restart Apache to activate it’s new configuration:
service apache2 restart
Prepare the suexec permissions files
echo "/var/www/teampass" >> /etc/apache2/suexec/www-data
echo "/var/www/teampass" > /etc/apache2/suexec/passwords
echo "TeamPass-master" >> /etc/apache2/suexec/passwords

IMPORTANT: We need to make sure that the cgi-script called by suexec is residing under the Server’s DocumentRoot for suexec to be allowed to run, therefore we installed the site under /var/www/teampass(which is located under the Server’s DocumentRoot(/var/www/) NOT meaning the VirtualHost’s DocumentRoot. A symlink is allowed here.

Preparing the MySQL database:

Create the new Database in MySQL:
Follow theses instructions:
1) Connect to mysql as root:
mysql -p -u root
PW: ******

2) Create the DB, user and user access rights:
CREATE DATABASE pwdb CHARACTER SET utf8 COLLATE utf8_bin;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on pwdb.* TO 'pwuser'@'localhost' IDENTIFIED BY 'password';
flush privileges;

Quit Mysql:
quit;
3) Tip: To confirm if the permissions were granted successfully, log into the DB server with the PWDB DB user(pwuser) and run the command below:
SHOW GRANTS FOR 'pwuser'@'localhost';
4) Quit Mysql:
quit;
Installing TeamPass via the web interface:
In the browser:
https://teampass.mydomain.com/install/install.php
Fill-in the appropriate, paths, MySQL credentials and extra settings and save this configuration.
You are then ready to use TeamPass

27 Apr 16 Enabling SPDY and Strict-Transport-Security to NginX in Ubuntu 14.04

In Ubuntu 14.04 NginX is been compiled with the SPDY capability. To use it one must enable it inside the server {…} block for each virtual host.
eg.
server {
server_name mprofi.com www.mprofi.com;
root /var/www/mprofi.com;
index index.php;
#
# Added to handle HTTP and HTTPS and SPDY
listen 80;
listen 443 ssl spdy;
ssl_certificate /etc/letsencrypt/live/www.mysite.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.mysite.com/privkey.pem;
#
# ENABLE STRICKT TRANSPORT SECURITY and X-Frame-Options
add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";
add_header X-Frame-Options "DENY";
}

Restart NginX
service nginx restart

21 Mar 16 Redirecting HTTP to HTTPS in NginX

Here is a working method of redirecting any requested HTTP URL to HTTPS in NginX VirtualHosts that handles both HTTP and HTTPS.
For example, to have a single vhost support both HTTP and HTTPS you have normally the following directives:
# Support for HTTP and HTTPS
listen 80;
listen 443 ssl;
ssl_certificate /etc/letsencrypt/live/www.myserver.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.myserver.com/privkey.pem;

Then to redirect all HTTP requests to HTTPS within this vhost without creating infinite loops you add the following redirect:
if ($scheme != "https") {rewrite ^ https://$host/$request_uri permanent;}
Other methods can be seen here:
http://serverfault.com/questions/67316/in-nginx-how-can-i-rewrite-all-http-requests-to-https-while-maintaining-sub-dom