msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

05 May 18 Minimize the Digests shown Headers in Mailman 2.1.xx

Problem:
Digests in Mailman are composed of a lots of unneeded headers which clutter the messages.

Solution:
Edit the Mailman configuration file manually as follows:
WARNING !!!: These headers are part of a the ‘RFC 1153’ which if changed can have unpredictable or unwanted effects.
So here I kept the headers: Date:, From:, Subject:, Keywords(if any), and Content-Type(quite important to keep)

Steps:
Rename the automatically compiled Python config file:
mv /usr/lib/mailman/Mailman/Defaults.pyc /usr/lib/mailman/Mailman/Defaults.pyc.orig

Edit the config file:
mcedit /usr/lib/mailman/Mailman/Defaults.py

and make the following changes from:

# Headers which should be kept in both RFC 1153 (plain) and MIME digests. RFC
# 1153 also specifies these headers in this exact order, so order matters.
MIME_DIGEST_KEEP_HEADERS = [
'Date', 'From', 'To', 'Cc', 'Subject', 'Message-ID', 'Keywords',
# I believe we should also keep these headers though.
'In-Reply-To', 'References', 'Content-Type', 'MIME-Version',
'Content-Transfer-Encoding', 'Precedence', 'Reply-To', 'List-Post',
# Mailman 2.0 adds these headers
'Message',
]
#
# The order in this list controls the order of the RFC 1153 digest headers.
# Also, any headers in this list will be kept in the MIME digest even if they
# don't appear in the MIME list above. Finally, headers appearing in both
# lists must be casewise the same or duplication can result in the digest.
PLAIN_DIGEST_KEEP_HEADERS = [
'Message',
# RFC 1153 headers in order
'Date', 'From', 'To', 'Cc', 'Subject', 'Message-ID', 'Keywords',
'Content-Type',
]

TO:

# Headers which should be kept in both RFC 1153 (plain) and MIME digests. RFC
# 1153 also specifies these headers in this exact order, so order matters.
#MIME_DIGEST_KEEP_HEADERS = [
# 'Date', 'From', 'To', 'Cc', 'Subject', 'Message-ID', 'Keywords',
# # I believe we should also keep these headers though.
# 'In-Reply-To', 'References', 'Content-Type', 'MIME-Version',
# 'Content-Transfer-Encoding', 'Precedence', 'Reply-To', 'List-Post',
# # Mailman 2.0 adds these headers
# 'Message',
# ]
#
MIME_DIGEST_KEEP_HEADERS = [
'Date', 'From', 'Subject', 'Keywords',
# I believe we should also keep these headers though.
'In-Reply-To', 'References', 'Content-Type', 'MIME-Version',
'Content-Transfer-Encoding', 'Precedence', 'Reply-To', 'List-Post',
]
#
# The order in this list controls the order of the RFC 1153 digest headers.
# Also, any headers in this list will be kept in the MIME digest even if they
# don't appear in the MIME list above. Finally, headers appearing in both
# lists must be casewise the same or duplication can result in the digest.
#PLAIN_DIGEST_KEEP_HEADERS = [
# 'Message',
# # RFC 1153 headers in order
# 'Date', 'From', 'To', 'Cc', 'Subject', 'Message-ID', 'Keywords',
# 'Content-Type',
# ]
#
PLAIN_DIGEST_KEEP_HEADERS = [
# RFC 1153 headers in order
'Date', 'From', 'Subject', 'Keywords',
'Content-Type',
]

Note: This might look confusing but just take a good look at the changes I made and you can see that I simply eliminated some headers from the 2 lists.
I simply kept the original version but commented it out as a reference in case things go wrong and I need to re-introduce some of them.

02 May 18 Configuring Domain Relaying with ISPConfig 3.1.xx

Intention:
Redirect (reroute) specific email addresses via, for example, an SMTP service:

Steps:
– Enter the destination domain in the Advanced Routing Table (Email ==> Email Accounts/Email Routing)
– Enter the same destination domain in the (Email ==> Global Filters / Relay Recipients) as @domain

Example:
eg. rerouting all emails of destination domain mydomain.com via a relay server (relay.myserver.com)

(Email ==> Email Accounts / Email Routing)
Select ‘Add new Transport’
Fill in the fields:
Server: (Select the server)
Domain: mydomain.com
Type: (select smtp)
No MX lookup: (leave unchecked)
Destination: relay.myserver.com
Sort by: 1
Active: (Checked)

(Email ==> Global Filters / Relay Recipients)
Select: ‘Add new relay reciepient’

Fill in the fields:
Server: (Chose the server)
Relay recipient: @mydomain.com
Active: (Checked)

02 May 18 No Type list in ISPConfig 3.1.11

Problem:
The brand new version of ISPConfig 3.1.11 when I add or modify an email transport, no value is displayed anymore on “type”.

Solution:
Ref: https://git.ispconfig.org/ispconfig/ispconfig3/issues/4924
Edit /usr/local/ispconfig/interface/web/mail/mail_transport_edit.php
Change this line:
$app->tpl->setVar($rec, null, true);
to this:
$app->tpl->setVar($rec);
and should work again.

03 Apr 18 Resetting MySQL/MariaDB root password in Ubuntu 16.04

Introduction:
In case you have forgotten the ‘root’ password in MySQL/MariaDB(10.0.x) you can reset the password as follows:
Ref: https://kofler.info/root-login-problem-mit-mariadb/

STEPS:
Stop the currently running MySQL/MariaDB
service mysql stop
Start MySQL/MariaDB in non-protected mode:
mysqld_safe --skip-grant-tables --skip-networking &
Login as root in MySQL/MariaDB
mysql -u root
Set the new root password:
update mysql.user set password=password('new-root-password-here') where user='root';
For MariaDB:
I case of MariaDB, it normally checks that the system user is root and the mysql root user has the proper password before granting access. In case of trying to login as root via PHPMyadmin this would fail even with the right MariaDB root password. To remedy to that, we need to disable the ‘unix_socket’ plugin as follows:
update mysql.user set plugin='' where user='root';
Confirm the new changes
select user,host,password,plugin from mysql.user;
The column ‘pluging’ should be empty for the user ‘root@localhost’

Exit MySQL/MariaDB:
FLUSH PRIVILEGES;
exit;

Kill the MySQL/MariaDB non-protected process:
killall mysqld
Wait a few seconds to let it finish.
Restart MySQL/MariaDB service as normal:
service mysql start

Now root login should work for PHPMyadmin as for mysql client.

03 Apr 18 Installing the missing mcrypt module for PHP 7.2

Inroduction:
Because of the module mcrypt for PHP neot being maintained since a bout 10 years the PHP team has decided to drop it from the PHP version 7.2 on.
For PHP applications that need this module here are the instructions to compile and install it for PHP 7.2.
Reference:
https://lukasmestan.com/install-mcrypt-extension-in-php7-2/

STEPS:
We need to install the proper building tools for PECL Mcrypt 1.0.1

Install mcrypt extension
sudo apt-get -y install gcc make autoconf libc-dev pkg-config
sudo apt-get -y install libmcrypt-dev
sudo pecl install mcrypt-1.0.1

When you are shown the prompt
libmcrypt prefix? [autodetect] :
Press [Enter] to autodetect.

After success installing mcrypt trought pecl, you should add mcrypt.so extension to php.ini.
The output will look like this:

Build process completed successfully
Installing '/usr/lib/php/20170718/mcrypt.so' ----> this is our path to mcrypt extension lib
install ok: channel://pecl.php.net/mcrypt-1.0.1
configuration option "php_ini" is not set to php.ini location
You should add "extension=mcrypt.so" to php.ini

Grab installing path and add to cli and apache2 php.ini configuration.
sudo bash -c "echo extension=/usr/lib/php/20170718/mcrypt.so > /etc/php/7.2/cli/conf.d/mcrypt.ini"
sudo bash -c "echo extension=/usr/lib/php/20170718/mcrypt.so > /etc/php/7.2/apache2/conf.d/mcrypt.ini"

Verify that the extension was installed
Run command:
php -i | grep "mcrypt"
The output will look like this:

/etc/php/7.2/cli/conf.d/mcrypt.ini
Registered Stream Filters => zlib.*, string.rot13, string.toupper, string.tolower, string.strip_tags, convert.*, consumed, dechunk, convert.iconv.*, mcrypt.*, mdecrypt.*
mcrypt
mcrypt support => enabled
mcrypt_filter support => enabled
mcrypt.algorithms_dir => no value => no value
mcrypt.modes_dir => no value => no value

23 Mar 18 Changing the mailman subscribers ‘moderation’ bit on the command line

Intro:
In my mailman installation with over 3K subscribers I could not find why the web interface didn’t allow me to change the ‘moderation’ bit of subscribers, or any other property. So I found this tool which allows me to the ‘moderation’ bit for any subscriber using the command line. Sinc ethe Python module for doing this is not provided with mailman you need to add it and run the command as follows:

Add the following content to the new file called: /usr/lib/mailman/bin/mod.py
#! /usr/bin/python
# mod.py
#
from Mailman import mm_cfg
import sys
#
def mod(list):
for member in list.getMembers():
if list.getMemberOption(member, mm_cfg.Moderate):
print member, "is moderated"
#
def set(list, member, value):
value = not not (int(value))
if list.isMember(member):
list.Lock()
list.setMemberOption(member, mm_cfg.Moderate, value)
print "%s's moderated flag set to %d" % (member, value)
list.Save()
list.Unlock()
else:
print member, "not a member"

Command for changing the moderation’ bit:
eg. for myname@mydomain.com in ‘people’ mailing list
Turning ON the ‘moderation’ bit:
/usr/lib/mailman/bin/withlist -r mod.set people myname@mydomain.com 1
Turning OFF the ‘moderation’ bit:
/usr/lib/mailman/bin/withlist -r mod.set people myname@mydomain.com 0
Turning ON the ‘moderation’ bit for ALL subscribers in the mailing list:
for member in $(/usr/lib/mailman/bin/list_members people) ; do
/usr/lib/mailman/bin/withlist -r mod.set people $member 1
done

02 Jan 18 Install CERTBOT in Ubuntu-16-04-xenial and Debian Stretch

Intro: Here is a 1-to-1 copy of the article on how to install certbot in Ubuntu 16.04 and Debian Stretch

Ubuntu 16.04 HOWTO:

Install
On Ubuntu systems, the Certbot team maintains a PPA. Once you add it to your list of repositories all you’ll need to do is apt-get the following packages.
$ sudo apt-get update
$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot

Advanced Get Started
Certbot supports a number of different “plugins” that can be used to obtain and/or install certificates.
Since your server architecture doesn’t yet support automatic installation you’ll have to use the certonly command to obtain your certificate.
$ sudo certbot certonly
This will allow you interactively select the plugin and options used to obtain your certificate. If you already have a webserver running, we recommend choosing the “webroot” plugin.
Alternatively, you can specify more information on the command line.
To obtain a cert using the “webroot” plugin, which can work with the webroot directory of any webserver software:
$ sudo certbot certonly --webroot -w /var/www/example -d example.com -d www.example.com -w /var/www/thing -d thing.is -d m.thing.is
This command will obtain a single cert for example.com, www.example.com, thing.is, and m.thing.is; it will place files below /var/www/example to prove control of the first two domains, and under /var/www/thing for the second pair.
Note:
To use the webroot plugin, your server must be configured to serve files from hidden directories. If /.well-known is treated specially by your webserver configuration, you might need to modify the configuration to ensure that files inside /.well-known/acme-challenge are served by the webserver.
To obtain a cert using a built-in “standalone” webserver (you may need to temporarily stop your existing webserver, if any) for example.com and www.example.com:
$ sudo certbot certonly --standalone -d example.com -d www.example.com
Automating renewal
The Certbot packages on your system come with a cron job that will renew your certificates automatically before they expire. Since Let’s Encrypt certificates last for 90 days, it’s highly advisable to take advantage of this feature. You can do automatic renewal for your certificates by running this command:
$ sudo certbot renew

Debian Stretch(9.0) HOWTO:

Install
Since Certbot is packaged for your system, all you’ll need to do is apt-get the following packages.
First you’ll have to follow the instructions here to enable the Stretch backports repo, if you have not already done so.
For this run:
$ sudo echo "deb http://ftp.debian.org/debian stretch-backports main" >> /etc/apt/sources.list
$ sudo apt-get update
$ sudo apt-get install certbot -t stretch-backports

Advanced Get Started
Certbot supports a number of different “plugins” that can be used to obtain and/or install certificates.
Since your server architecture doesn’t yet support automatic installation you’ll have to use the certonly command to obtain your certificate.
$ sudo certbot certonly
This will allow you interactively select the plugin and options used to obtain your certificate. If you already have a webserver running, we recommend choosing the “webroot” plugin.
Alternatively, you can specify more information on the command line.
To obtain a cert using the “webroot” plugin, which can work with the webroot directory of any webserver software:
$ sudo certbot certonly --webroot -w /var/www/example -d example.com -d www.example.com -w /var/www/thing -d thing.is -d m.thing.is
This command will obtain a single cert for example.com, www.example.com, thing.is, and m.thing.is; it will place files below /var/www/example to prove control of the first two domains, and under /var/www/thing for the second pair.

Note:
To use the webroot plugin, your server must be configured to serve files from hidden directories. If /.well-known is treated specially by your webserver configuration, you might need to modify the configuration to ensure that files inside /.well-known/acme-challenge are served by the webserver.

To obtain a cert using a built-in “standalone” webserver (you may need to temporarily stop your existing webserver, if any) for example.com and www.example.com:
$ sudo certbot certonly --standalone -d example.com -d www.example.com
Automating renewal
The Certbot packages on your system come with a cron job that will renew your certificates automatically before they expire. Since Let’s Encrypt certificates last for 90 days, it’s highly advisable to take advantage of this feature. You can do renewal for your certificates by running this command:
$ sudo certbot renew

02 Jan 18 Configuring Letsencrypt in ISPConfig 3.1

Intro:
Since a while now the wonderful idea of creating the service Letsencrypt has made lots of admins happy.
Here is how we can also use Letsencrypt with ISPConfig 3.1.

Ref: https://www.howtoforge.com/community/threads/ssl-how-to-for-ispconfig-3-with-letsencrypt.74738/

STEPS:
Define ISPconfig to use the new SSL certificate with symbolic links.
(If you don’t know how to use symbolic links this how-to is not for you)
/usr/local/ispconfig/interface/ssl/
ispserver.crt -> /etc/letsencrypt/live/mydomain.com/fullchain.pem
ispserver.key -> /etc/letsencrypt/live/mydomain.com/privkey.pem

Define Postfix to use the new SSL certificate in /etc/postfix/main.cf.
(If you don’t know how to add these entries this how-to is not for you)
smtpd_tls_cert_file = /etc/letsencrypt/live/mydomain.com/cert.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mydomain.com/privkey.pem
smtpd_tls_CAfile = /etc/letsencrypt/live/mydomain.com/fullchain.pem

Define Dovecot to use the new SSL certificate in /etc/dovecot/dovecot.conf.
(If you don’t know how to add these entries this how-to is not for you)
ssl_cert = </etc/letsencrypt/live/mydomain.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mydomain.com/privkey.pem

# Restart/reload the services involved
sudo service postfix reload
sudo service dovecot reload
sudo service apache2 restart

23 Dec 17 Rectify mailman URLs after a hostname change

Intro:
I had to change the server name of my mailman server. I changed it in /etc/mailman/mm_cfg.py as follows:
# Default domain for email addresses of newly created MLs
DEFAULT_EMAIL_HOST = 'mailman.myserver.com'
#-------------------------------------------------------------
# Default host for web interface of newly created MLs
DEFAULT_URL_HOST = 'mailman.myserver.com'

BUT! Some links in the mailman site were OK (new) and others were not OK(Old servername)

SOLUTION:
To remedy to this all the mailing list need to be internally modified to reflect the new hostmname in the sites URLs and the emails URLs.
Ref: https://mail.python.org/pipermail/mailman-users/2006-February/049052.html
Simply run the following 2 commands:
cd /usr/lib/mailman/bin/
./withlist -l -a -r fix_url -- -v

This runs withlist and tells it to lock the lists (-l) process all lists (-a) process by calling fix_url in the module fix_url.py with arguments of the list instance and -v which causes fix_url to report what it’s doing. The — is to separate the -v option for fix_url from the withlist options since there’s no listname to do that in this case.

For mailing lists with different URL then the site is suggesting the following:
———————————–
If you have more than one virtual host, you have to process the lists
one at a time with

bin/withlist -l -r fix_url listname -u url_host

but you could wrap that in a shell script to run the command repeatedly
for all the listname/url_host pairs.
———————————–

21 Dec 17 Blocking hosts blacklist and iptables

Intro:
I happen to have sone attacks coming from specific hosts which I decided to block access to the server. Here is how I did it using a script which deletes and reload a full iptables CHAIN based on a file containing a list of IPs/Ranges.

STEPS:
Create a file called blacklist.txt with one IP/Range per line in the same directory as the script.
eg.
14.141.107.206
23.180.0.0/14
37.59.34.120
46.140.157.157
46.218.35.59
47.74.0.40
51.15.56.170
59.62.0.0/15
59.63.188.3
61.177.172.60

Script to run at boot time
#!/bin/bash
# Tiny firewall protecting rpcbind (port 111)
scriptdir=$(dirname $(readlink -f $0))
blacklist="$scriptdir/blacklist.txt"
# Load the blacklists
HOSTS="$(cat $blacklist | egrep -v '^$|#')"
# Delete the existing custom chain
/sbin/iptables --flush BLACKLIST
/sbin/iptables -X BLACKLIST
/sbin/iptables -t filter -D INPUT -j BLACKLIST
# Create the BLACKLIST Chain and jump
/sbin/iptables -N BLACKLIST
/sbin/iptables -t filter -I INPUT -j BLACKLIST
# Fill-in the BLACKLIST Chain with rejected hosts list
for host in $HOSTS ; do
/sbin/iptables -A BLACKLIST -s $host -p tcp -j DROP
done
# Return from Blacklist
/sbin/iptables -A BLACKLIST -j RETURN
#eof

Note: iptables will complain with the following errors. Not to worry, it will still do the proper job.
iptables: Too many links.
iptables: Chain already exists.