msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

09 Nov 17 piwik: Could not open input file: ./console

Introduction:
In order to know the location of the visits your website received before you started using Piwik with GeoIP you need to run a command.
The reference to this command is at: https://piwik.org/faq/how-to/faq_167/

Problem:
Unfortunately after having logged in as root in the server this command gave me the following error:
Could not open input file: ./console
After doing research and using my own Linux experience here is a(the?) solution:
Ref: https://stackoverflow.com/questions/10637230/could-not-open-input-file-app-console

Solution:
# Make temporarily the www-data user login possible
usermod -s /bin/bash www-data
# Login as www-data
sudo su - www-data
# Change the htdocs directory to the installed Piwik.
cd /var/www/piwik.myserver.com/
# Run the command
php ./console usercountry:attribute 2012-01-01,2013-01-01
Result:
Re-attribution for date range: 2012-01-01 to 2013-01-01. 0 visits to process with provider "ip2location".
Completed. Time elapsed: 0.819s

# Get out of www-data user login and back to root login
exit
# Prevent back login of the user www-data(as it was originally)
usermod -s /usr/sbin/nologin www-data
Important Note:
In the command given you need to give the exact date range (eg. 2012-01-01,2017-11-01) which needs to be evaluated in your Piwick reports.

08 Sep 17 Prepare Debian Stretch for Installing GlusterFS 3.12

In order to install this version of GlusterFS we need to add the repositories:
Ref: https://download.gluster.org/pub/gluster/glusterfs/LATEST/Debian/
echo deb [arch=amd64] http://download.gluster.org/pub/gluster/glusterfs/3.12/LATEST/Debian/stretch/apt stretch main > /etc/apt/sources.list.d/gluster.list
wget -O - http://download.gluster.org/pub/gluster/glusterfs/3.12/rsa.pub | apt-key add -
apt-get update
apt-get install glusterfs-server xfsprogs

Format the dedicated partition for GlusterFS synchronized data:
eg. /dev/xvda3
mkfs.xfs -f -i size=512 /dev/xvda3
Example of result:
meta-data=/dev/xvda3 isize=512 agcount=4, agsize=655360 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=2621440, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

Regarding the configuration steps see:
https://wp.me/pKZRY-Kk
OR better:
https://gluster.readthedocs.io/en/latest/Install-Guide/Configure/

02 Sep 17 Transferring IMAP account mails and folders to another IMAP account on another server

Introduction:
The other day I was asked to install a completely new email server and transfer all the email accounts from the old mail server to the new one. I noticed that since the new mail server was using a different mail INBOX format I had to do some research and found this really good tool to do exactly what I needed called: imapsync

Installing the tool:
This tool programmed in Perl and is not free. It can be bought at http://imapsync.lamiral.info/.
Note: It does a great job and it’s really worth its price when you think of the time and hassle saved by using it.

Using the tool:
Example 1: Copying all the mails in folder INBOX from jim account on localhost to another server with the same credentials:
– First we do a dry-run to see what will be transferred when I run it normally:

imapsync --dry \
--host1 localhost --user1 jim --password1 'secret1' --folder INBOX --tls2 \
--host2 mail.myserver2.com --user2 jim --password2 'secret1' --nofoldersizes --nofoldersizesatend

Example 2: Copying all the mails and folders(no dry-run) from account martin@myserver1.com on localhost to a new account on another server with different credentials:
imapsync \
--host1 localhost --user1 martin@myserver1.com --password1 secret1 \
--host2 mail.myserver2.com --user2 martin@myserver2.com --password2 secret2

29 Aug 17 Installing Filebeat, Logstash, ElasticSearch and Kibana in Ubuntu 14.04

PREPARATIONS

#Ref: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
First install Java 8 in Ubuntu 14.04

# Ref: https://www.liquidweb.com/kb/how-to-install-oracle-java-8-on-ubuntu-14-04-lts/
apt-get install python-software-properties software-properties-common
apt-add-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer
java -version

Result:
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

Facilitate updating of all packages via APT repositories

apt-get install apt-transport-https
Save the repository definition to /etc/apt/sources.list.d/elastic-5.x.list:
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
apt-get update

FILEBEAT

Installing filebeat

Filebeat reads lines from defined logs, formats them properly and forwards them to logstash while maintaining a non-clogging pipeline stream
Ref: https://github.com/elastic/beats/tree/master/filebeat
Ref: https://www.elastic.co/guide/en/beats/filebeat/5.5/filebeat-getting-started.html
Ref: https://www.elastic.co/products/beats/filebeat

apt-get install filebeat
mv /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml.orig
touch /etc/filebeat/filebeat.yml
mcedit /etc/filebeat/filebeat.yml

(content)
————————

filebeat.prospectors:
- input_type: log
paths:
- /var/log/apache2/access.log
output.logstash:
hosts: ["localhost:5044"]

————————
service filebeat restart

LOGSTASH

Download logstash debian install package and configure it

# Ref: https://www.elastic.co/downloads/logstash
apt-get install logstash

# Result:
.......
Using provided startup.options file: /etc/logstash/startup.options
Successfully created system startup script for Logstash

Preparing Logstash

mcedit /etc/logstash/startup.options
(add the following line at the beginning)
LS_CONFIGS_DIR=/etc/logstash/conf.d/

(Then adjust the following line as follows)
from:
LS_OPTS="--path.settings ${LS_SETTINGS_DIR}"
to:
LS_OPTS="--path.settings ${LS_SETTINGS_DIR} --path.config ${LS_CONFIGS_DIR}"

Start/Stop/Restart logstash
service logstash {start|stop|restart}

Testing logstash

cd /etc/logstash/ ; /usr/share/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'

Type: hello world
and and press CTRL-D

(Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a CTRL-D command in the shell where Logstash is running.)

Results:

ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using –path.settings. Continuing using the defaults
Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs to console
11:22:59.822 [[main]-pipeline-manager] INFO logstash.pipeline – Starting pipeline {“id”=>”main”, “pipeline.workers”=>2, “pipeline.batch.size”=>125, “pipeline.batch.delay”=>5, “pipeline.max_inflight”=>250}
11:22:59.847 [[main]-pipeline-manager] INFO logstash.pipeline – Pipeline main started
The stdin plugin is now waiting for input:
2017-08-23T09:22:59.878Z h270746.stratoserver.net test 1
11:22:59.946 [Api Webserver] INFO logstash.agent – Successfully started Logstash API endpoint {:port=>9601}
11:23:02.861 [LogStash::Runner] WARN logstash.agent – stopping pipeline {:id=>”main”}

The errors and warnings are OK for now. The main result line above that is significant is:
2017-08-23T09:22:59.878Z h270746.stratoserver.net test 1
which adds a timestamp and server name to the input string (test 1)

Configuring logstash
# Note: this test configuration will get input from filebeat and output into a log file which can be watched with tail -f …..
mcedit /etc/logstash/conf.d/apache2.conf
(content)
input {
beats {
port => 5044
type => "apache"
}
}
filter {
if [type] == "apache" {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}"}
}
geoip {
source => "clientip"
}
}
}
output {
file {
path => "/var/log/logstash_output.log"
}
}

In order to have the proper output sent to elasticsearch then use this output configuration instead:
———————————-
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

Securing Filebeat => Logstash with SSL

Ref: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ssl-logstash.html#configuring-ssl-logstash
Note: Typing by hand below is shown in bold.

Prepare the ertificates directories:

mkdir -p /etc/logstash/certs/Logstash/ /etc/logstash/certs/Beats/
Create client certificates for FileBeat:
/usr/share/elasticsearch/bin/x-pack/certgen

........
Let's get started...

Please enter the desired output file [/etc/elasticsearch/x-pack/certificate-bundle.zip]: /etc/logstash/certs/Beats/certificate-bundle_Beats.zip
Enter instance name: Beats
Enter name for directories and files : Beats
Enter IP Addresses for instance (comma-separated if more than one) []:
Enter DNS names for instance (comma-separated if more than one) []: localhost
Certificates written to /etc/logstash/certs/Beats/certificate-bundle_Beats.zip

Create client certificates for Logstash:
/usr/share/elasticsearch/bin/x-pack/certgen

........
Let's get started...

Please enter the desired output file [/etc/elasticsearch/x-pack/certificate-bundle.zip]: /etc/logstash/certs/Logstash/certificate-bundle_Logstash.zip
Enter instance name: Logstash
Enter name for directories and files : Logstash
Enter IP Addresses for instance (comma-separated if more than one) []:
Enter DNS names for instance (comma-separated if more than one) []: localhost
Certificates written to /etc/logstash/certs/Logstash/certificate-bundle_Logstash.zip

This file should be properly secured as it contains the private keys for all
instances and the certificate authority.

After unzipping the file, there will be a directory for each instance containing
the certificate and private key. Copy the certificate, key, and CA certificate
to the configuration directory of the Elastic product that they will be used for
and follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

Extract certificates:
unzip /etc/logstash/certs/Beats/certificate-bundle_Beats.zip -d /etc/logstash/certs/Beats/
unzip /etc/logstash/certs/Logstash/certificate-bundle_Logstash.zip -d /etc/logstash/certs/Logstash/

Convert the Logstash key Logstash.key from PKCS#1 to PKCS#8 format:
Reason: the following error message in the logstash.log occured when using the PKCS1 format:
[ERROR][logstash.inputs.beats ] Looks like you either have an invalid key or your private key was not in PKCS8 format. {:exception=>java.lang.IllegalArgumentException: File does not contain valid private key: /etc/logstash/certs/Logstash/Logstash/Logstash.key}

See: https://github.com/spujadas/elk-docker/issues/112

Command:
openssl pkcs8 -in /etc/logstash/certs/Logstash/Logstash/Logstash.key -topk8 -nocrypt -out /etc/logstash/certs/Logstash/Logstash/Logstash.key.PKCS8

Configure Beats for SSL

Content of /etc/filebeat/filebeat.yml
filebeat.prospectors:
- input_type: log
paths:
- /var/log/apache2/access.log
output.logstash:
hosts: ["localhost:5044"]
ssl.certificate_authorities: ["/etc/logstash/certs/Logstash/ca/ca.crt"]
ssl.certificate: "/etc/logstash/certs/Beats/Beats/Beats.crt"
ssl.key: "/etc/logstash/certs/Beats/Beats/Beats.key"

Content of /etc/logstash/conf.d/apache.conf
input {
beats {
port => 5044
ssl => true
ssl_certificate_authorities => ["/etc/logstash/certs/Logstash/ca/ca.crt"]
ssl_certificate => "/etc/logstash/certs/Logstash/Logstash/Logstash.crt"
ssl_key => "/etc/logstash/certs/Logstash/Logstash/Logstash.key.PKCS8"
ssl_verify_mode => "force_peer"
}
}

Restart both Logstash and Filebeat
service logstash restart
service filebeat restart

NOTE: I’m still having problems with the SSL connection of Filebeat to Logstash where Filebeat throws this error in (/var/log/logstash/logstash-plain.log):
TLS internal error.
The following URL seems to have found some similar problems but because of lack of time I haven’t figured it out yet.
https://discuss.elastic.co/t/mutual-tls-filebeat-to-logstash-fails-with-remote-error-tls-internal-error/85271/3

X-Pack for Logstash

INSTALL X-Pack for logstash

X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting, machine learning, and graph capabilities into one easy-to-install package.
X-Pack also provides a monitoring UI for Logstash.

/usr/share/logstash/bin/logstash-plugin install x-pack

Result:

Downloading file: https://artifacts.elastic.co/downloads/logstash-plugins/x-pack/x-pack-5.5.2.zip
Downloading [=============================================================] 100%
Installing file: /tmp/studtmp-bc1c884de6d90f1aaa462364e5895b6b08b050f0b64587b4f5e0a8ec5300/x-pack-5.5.2.zip
Install successful

Configuring X-Pack in Logstash:

The defaults settings created during the installation works best for most cases. For more information see:
https://www.elastic.co/guide/en/logstash/5.5/settings-xpack.html

To Prevent generation of monitoring error messages in logstash.log edit /etc/logstash/logstash.yml and add the following line at the end:
(Ref: https://discuss.elastic.co/t/logstash-breaks-when-disabling-certain-x-pack-features/89511)

xpack.monitoring.enabled: false

ElasticSearch

Installation:
apt-get install elasticsearch

Start/Stop/Restart Elastic search:
/etc/init.d/elasticsearch {start|stop|restart}

To check if elasticsearch has been started:
ps aux | grep $(cat /var/run/elasticsearch/elasticsearch.pid)

Example of result(truncated):
elastic+ 10978 3.2 55.2 4622152 2319168 pts/3 Sl 15:44 0:10 /usr/lib/jvm/java-8-oracle/bin/java ........

The check the Elasticsearch log file:
tail -f /var/log/elasticsearch/elasticsearch.log

NOTE 1:
If you see the line:
[WARN ][o.e.b.BootstrapChecks ] [wJdCtOd] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
and the result of the following command is empty,

grep vm.max_map_count /etc/sysctl.conf

Solution:
Raise the max virtual memory areas vm.max_map_count to 262144 as follows:
Add the following line in the file /etc/sysctl.conf

vm.max_map_count=262144

And run the command:
sysctl -w vm.max_map_count=262144
OR
echo 262144 > /proc/sys/vm/max_map_count

ALSO Make sure the elasticsearch config file (/etc/elasticsearch/jvm.options) has the following entries:
-Xms2g
-Xmx2g

IMPORTANT:
if the following commands are failing it might be because some Virtual Servers are not allowing such changes in the kernel:
eg.
sysctl -w vm.max_map_count=262144
sysctl: permission denied on key ‘vm.max_map_count’
echo 262144 > /proc/sys/vm/max_map_count
-bash: /proc/sys/vm/max_map_count: Permission denied

Elastic search should be able to run anyway but might be limited in performance and may have other issues because of these limitations.
There is no known remedies to this for Strato VM servers.

NOTE 2:
If you see the line:
[WARN ][i.n.u.i.MacAddressUtil ] Failed to find a usable hardware address from the network interfaces; using random bytes: ……..

Solution:
No need to worry, the accuracy of the MAC address is not so important in this installation.

NOTE 3:
If you see the line:
[WARN ][o.e.b.BootstrapChecks ] [wJdCtOd] system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
If this problem occurs elasticsearch will start but not get initialised properly and most likely not function properly.

Solution:
If elasticsearch is accessed only in a protected environment, disabling this installation of system call filters should be no problem
by editing the file /etc/elasticsearch/elasticsearch.yml and adding the following line:
bootstrap.system_call_filter: false
Restart elasticsearch:
service elasticsearch restart

————————————————————————

X-Pack for elasticsearch

X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting, machine learning, and graph capabilities into one easy-to-install package.

Installation:
/usr/share/elasticsearch/bin/elasticsearch-plugin install x-pack

Results:
-> Downloading x-pack from elastic
[=================================================] 100%
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin requires additional permissions @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.io.FilePermission \\.\pipe\* read,write
* java.lang.RuntimePermission accessClassInPackage.com.sun.activation.registries
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.RuntimePermission setFactory
* java.security.SecurityPermission createPolicy.JavaPolicy
* java.security.SecurityPermission getPolicy
* java.security.SecurityPermission putProviderProperty.BC
* java.security.SecurityPermission setPolicy
* java.util.PropertyPermission * read,write
* java.util.PropertyPermission sun.nio.ch.bugLevel write
* javax.net.ssl.SSLPermission setHostnameVerifier
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: plugin forks a native controller @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
This plugin launches a native controller that is not subject to the Java
security manager nor to system call filters.

Continue with installation? [y/N]y
-> Installed x-pack

KIBANA

Install kibana package
apt install kibana
Install X-Pack for logstash
X-Pack is an Elastic Stack extension that bundles security, alerting, monitoring, reporting, machine learning, and graph capabilities into one easy-to-install package.
/usr/share/kibana/bin/kibana-plugin install x-pack
Change built-in users password
Ref: https://www.elastic.co/guide/en/x-pack/5.5/setting-up-authentication.html#reset-built-in-user-passwords
change passwords

curl -XPUT 'localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'
{
"password": "elasticpassword"
}
'

curl -XPUT 'localhost:9200/_xpack/security/user/kibana/_password?pretty' -H 'Content-Type: application/json' -d'
{
"password": "kibanapassword"
}
'

curl -XPUT 'localhost:9200/_xpack/security/user/logstash_system/_password?pretty' -H 'Content-Type: application/json' -d'
{
"password": "logstashpassword"
}
'

Update the Kibana server with the new password /etc/kibana/kibana.yml
elasticsearch.password: kibanapassword
Update the Logstash configuration with the new password /etc/logstash/logstash.yml
xpack.monitoring.elasticsearch.password: logstashpassword
Disable Default Password Functionality /etc/elasticsearch/elasticsearch.yml
xpack.security.authc.accept_default_password: false

Start/Stop/Restart kibana
service kibana {start|stop|restart}

04 Jul 17 TCP Proxying using socat

Introduction:
Lately I’ve had to create a pure bidirectional TCP Proxy for a project. For this there are lots of alternatives like haproxy, nginx, cat and socat and others. Because of the simplicity of the command I decided to use socat but will also show the command for cat as well.

The NCAT method:
The following command will us a pipe to transport the data in both directions. Only one client can be connected at one time.
cd /var/tmp
mkfifo fifo &>/dev/null
/bin/nc -l -p $frontend_port -s $frontend_addr <fifo | /bin/nc $backend_addr $backend_port >fifo

The SOCAT method(Best!):
Note: this method runs the command in a screen session but doesn’t need to if the process is only temporarily needed to be run.
/usr/bin/screen -d -m /usr/bin/socat -d -d -lmlocal2 \
TCP4-LISTEN:$frontend_port,bind=$frontend_addr,reuseaddr,fork,su=daemon \
TCP4:$backend_addr:$backend_port,bind=$backend_iface_addr

23 May 17 Disabling the admin security password confirmation in Jira and Confluence

Introduction:
Although in Jira and Confluence the WebSudo, requesting the confirmation of the administrator’s password, are neat security features if you are working in a company where the chances of someone fiddling around with your computer are high. BUT in a very small company, where this risk is almost none, this feature has proven very annoying for me. So I did some research to disable these features in both Jira and Confluence.

Assumptions:
Jira Version: 7.x
Confluence: 6.x

Methods:

In Jira:

– Edit the file /opt/atlassian/jira/atlassian-jira/WEB-INF/classes/jpm.xml
– Look for the property: jira.websudo.is.disabled and set all is values to true as follows:
.....
<property>
<key>jira.websudo.is.disabled</key>
<default-value>true</default-value>
<type>boolean</type>
<admin-editable>true</admin-editable>
<sysadmin-editable>true</sysadmin-editable>
</property>
......

In Confluence

– Edit the file /opt/atlassian/confluence/bin/setenv.sh
– Close to the end where there is a list of multiple components of the variable
CATALINA_OPTS=....
CATALINA_OPTS=....

– Add the following line after this list but before the line: export CATALINA_OPTS
CATALINA_OPTS="-Dpassword.confirmation.disabled=true ${CATALINA_OPTS}"

———-
Note: After these changes Jira and Confluence need to be restarted as follows:
service jira stop
service confluence stop
service jira start
service confluence start

16 May 17 Hardening the SSL security in Apache, Dovecot and Postfix

Introduction:

After having gotten a report from OpenVAS that my SSL security level of the mail server were medium, I looked for ways to improve this.
I found very good sites which helps me making these improvements:
https://weakdh.org/sysadmin.html
https://wiki.dovecot.org/SSL/DovecotConfiguration
https://bettercrypto.org/static/applied-crypto-hardening.pdf
Based on this site and extending to cover dovecot mail service here is the result:

Hardening Apache:

In /etc/apache2/mods-available/ssl.conf
Change the following parameters as follows:
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!DH+3DES:!RSA+3DES
SSLHonorCipherOrder on

Hardening Dovecot:

Note: you should have openssl >=1.0.0 dovecot >=2.1.x required, better dovecot >=2.2.x because of ECDHE support Dovecot tryies to use PFS by default, so besides the enabled SSL almost no actions are required change the log settings to see the cipher, grep for a login_log_format_elements in dovecot configs and add %k to it
eg:
login_log_format_elements = "user=< %u> method=%m rip=%r lip=%l mpid=%e %c %k"
Configure the allowed ciphers. Server side enforcement works only for dovecot >=2.2.6
In /etc/dovecot/conf.d/ssl.conf
Change some parameters as follows:
ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4
#only for dovecot >=2.2.6, enforce the server cipher preference
ssl_prefer_server_ciphers = yes
#disable SSLv2 and SSLv3
ssl_protocols = !SSLv2 !SSLv3

Add the following parameter:
ssl_dh_parameters_length = 2048
Delete the file /var/lib/dovecot/ssl-parameters.dat
and restart Dovecot service:
service dovecot restart
Dovecote seeing that the Diffie Hellman parameters are assigned to be 2048 bits long and that its file is just been deleted, will regenerate a new one in the background.

Hardening Postfix

In /etc/postfix/main.cf
Change or add the following configuration parameters:
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3
smtpd_tls_mandatory_ciphers=high
tls_high_cipherlist=EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CBC3-SHA, KRB5-DES, CBC3-SHA
smtpd_tls_dh1024_param_file=/etc/ssl/dh2048.pem

Generate a new Diffie Hellman parameters file as follows:
openssl dhparam -out /etc/ssl/dh2048.pem 2048

24 Feb 17 Whitelisting Hosts in Postfix/Amavis

Introduction:
I have an email server with very strong spam filtering and every now and then it does see the emails that I send from our own networks as SPAM.
In order to bypass the SPAM scanner for those networks without bypassing the virus scanning of Amavis I found these instructions in Internet at:
http://verchick.com/mecham/public_html/spam/bypassing.html#1

Allow clients on my internal network to bypass scanning by using the ‘MYNETS’ policy bank. You can use the built in ‘MYNETS’ policy bank to allow clients included in $mynetworks. Let’s assume you allow all (or most) clients on your internal network to send outbound mail through your spamfilter.
The IP addresses of these clients are included in Postfix’ $mynetworks in main.cf:
mynetworks = 127.0.0.0/8 !192.168.1.1 192.168.1.0/24
In /etc/amavis/conf.d/50-user @mynetworks determines which clients will use the ‘MYNETS’ policy bank:
@mynetworks = qw( 127.0.0.0/8 [::1] [FE80::]/10 [FEC0::]/10
!192.168.1.1 192.168.1.0/24 );

And you would configure the ‘MYNETS’ policy bank as desired:
Also added to /etc/amavis/conf.d/50-user
$policy_bank{'MYNETS'} = { # clients in @mynetworks
bypass_spam_checks_maps => [1], # don't spam-check internal mail
bypass_banned_checks_maps => [1], # don't banned-check internal mail
bypass_header_checks_maps => [1], # don't header-check internal mail
};

When using the “MYNETS’ policy bank, you must use *_send_xforward_command in master.cf which enables forwarding of the client’s IP address to amavisd-new.:
smtp-amavis unix - - n - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20

(or)
lmtp-amavis unix - - n - 2 lmtp
-o lmtp_data_done_timeout=1200
-o lmtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20

21 Jan 17 Mounting a remote directory using SSHFS in Debian Jessie

Introduction:
If you want to mount a directory on a remote server via Internet NFS can be quite a challenge to protect. A good solution would then be to use SSHFS. Here is a shot Howto for Debian Jessie.

Note: In Wheezy and in Jessie before I did an upgrade to the kernel 3.16.0-4-amd64, the following entry in /etc/fstab was working:
sshfs#root@remote.server.com:/remote_dir /local_dir fuse defaults 0 0
BUT, as soon as upgraded Jessie to the kernel 3.16.0-4-amd64, I could not boot any more and the system went into an emergency mode signalizing that I should give the root password or press Ctrl-D to continue. Ctrl-D brought to nowhere and the system just crashed. It was also suggested that I should give the command ‘journalctl -xb’ to find out what was wrong after I had given the root password. This command gave me the indication that ‘process /bin/plymouth could not be executed’. Well, the message is quite misleading since the error was that the new kernel was no more supporting the above older method of mounting a filesystem using SSHFS in /etc/fstab. Commenting this entry in /etc/fstab allowed me to boot and later to change the entry for a new one that worked which follows.

First install the needed package:
apt-get install sshfs
Then considering the two scenarios:
1 – User mount: Mounting a remote directory belonging to user ‘media’ using SSHFS and the ssh keys. User ‘media’ was configured in both servers to have the same UID.
2 – Root mount: Mounting a remote directory belonging to root using SSHFS and the ssh keys.

Scenario 1:(user mount)

On remote server run the command:
useradd -d /home/media/ -u 2017 -s /bin/bash media
passwd media (give any password, that will need to be deleted later anyway)
mkdir -p /home/media/share1
chown -R media: /home/media/share1

On local server run the commands:
useradd -d /home/media/ -u 2017 -s /bin/bash media
mkdir -p /home/media/share1
chown -R media: /home/media/share1
su - media
ssh-keygen -t rsa (press <Enter> to all questions)
ssh-copy-id media@remote.server.com (enter media user's temporary password of remote server)

Enter in /etc/fstab:
media@remote.server.com:/home/media/share1 /home/media/share1 fuse.sshfs noauto,x-systemd.automount,_netdev,user,idmap=user,follow_symlinks,identityfile=/home/media/.ssh/id_rsa,allow_other,default_permissions,uid=2017,gid=2017 0 0
Back on remote server, disable the user’s password using the command:
passwd -l media
———- End scenario 1 ———–

Scenario 2 (root mount)

ssh-copy-id root@remote.server.com (enter 'root' password of remote server)
Enter in /etc/fstab:
root@remote.server.com:/share2 /share2 fuse.sshfs noauto,x-systemd.automount,_netdev,user,idmap=user,follow_symlinks,identityfile=/root/.ssh/id_rsa,allow_other,default_permissions,uid=0,gid=0 0 0
———- End scenario 2 ———–
Then reboot the system
reboot
After reboot you won’t see yet any mount entry if you give the command ‘mount’. It will only appear after the first attempt to access the mount point in the local server. This mount is governed by systemd. You can’t quite control manually the mounting and unmounting of this new method since it’s controlled by systemd. I’m still looking for ways to manually mount/unmount this systemd controlled mount. Any suggestions is welcome.

19 Jan 17 Installing TeamPass in Debian Jessie

Introduction:
TeamPass is a very good Web application which can store securely Passwords for single person or teams. Here are the steps I used to install it in Debian Jessie. These instructions can also be used with no or minimal changes to install TeamPass in other Debian or Ubuntu systems.
These instruction are partly based on this site:
http://teampass.net/2013-12-31-installation-on-linux-server
and these
http://bourntech.com/blog/install-teampass-on-ubuntu-14-6lts/
https://github.com/nilsteampassnet/TeamPass/

Steps:
Create the user that will be used as owner of the TeamPass htdocs and Apache TeamPass requests processes.
useradd -d /opt/teampass/ -s /bin/false passwords
Prepare the teampass home directories
mkdir -p /var/www/teampass/fcgi/tmp
mkdir /var/www/teampass/logs
mkdir /var/www/teampass/auth
cd /var/www/teampass/
#Get the latest released software:
wget --no-check-certificate https://github.com/nilsteampassnet/TeamPass/archive/master.zip
unzip master.zip

Install the required packages:
apt-get install php5-mcrypt php5-mysqlnd php5-gd openssl apache2-suexec-custom apache2-mpm-prefork libapache2-mod-fcgid libapache2-mod-php5 php5-cgi mariadb-server
In order to allow Apache to modify files inside the TeamPass htdocs we use FCGI/suexec Modules.
a2enmod fcgid
a2enmod suexec
a2enmod ssl

Create the fcgi_wrapper script:
touch /var/www/teampass/fcgi/php-fcgi-starter
mcedit /var/www/teampass/fcgi/php-fcgi-starter

Content:
#!/bin/sh
export PHPRC=/var/www/teampass/fcgi/
export PHP_FCGI_CHILDREN=2
export PHP_FCGI_MAX_REQUESTS=500
exec /usr/bin/php5-cgi

Make it runnable but not for others:
chmod 750 /var/www/teampass/fcgi/php-fcgi-starter
Copy the php.ini from system to /var/www/teampass/fcgi/
cp /etc/php5/apache2/php.ini /var/www/teampass/fcgi/
Adapt the php.init to the site:
mcedit /var/www/teampass/fcgi/php.ini
Add the following 2 lines at the end:
upload_tmp_dir = /var/www/teampass/fcgi/tmp
session.save_path = /var/www/teampass/fcgi/tmp

And look for the configuration: max_execution_time and change its value from 30 to 60. Eg.
max_execution_time = 60
Create the Apache2 configuration:
Content of config file in /etc/apache2/sites-available/teampass.mydomain.com.conf:
# ============ https://teampass.mydomain.com ==================
<virtualhost *:443>
ServerName teampass.mydomain.com
DocumentRoot /var/www/teampass/TeamPass-master
SuexecUserGroup passwords passwords
<directory /var/www/teampass/TeamPass-master>
Options -Indexes +FollowSymLinks +ExecCGI
FCGIWrapper /var/www/teampass/fcgi/php-fcgi-starter .php
AddHandler fcgid-script .php
DirectoryIndex index.php
Require 192.168. granted
AuthType Basic
AuthName "Private area"
AuthUserFile /var/www/teampass/auth/web.auth
Require valid-user
Satisfy all
</directory>
SSLEngine On
SSLCertificateFile /etc/letsencrypt/live/teampass.mydomain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/teampass.mydomain.com/privkey.pem
SSLCACertificateFile /etc/letsencrypt/live/teampass.mydomain.com/chain.pem
ErrorLog /var/www/teampass/logs/error_log
CustomLog /var/www/teampass/logs/access_log combined
</virtualhost>

Create the first layer(BASIC) Authentication credentials for first user:
htpasswd -c /var/www/teampass/auth/web.auth username
Give the whole directory ownership to ‘passwords’ user
chown -R passwords: /var/www/teampass/
NOTE: Before you restart your Apache2 service, make sure the Certificate is been issue and installed in the directory: /etc/letsencrypt/live/teampass.mydomain.com/
You can use the instructions on this link to install LetsEncrypt software:
//tipstricks.itmatrix.eu/?s=letsencrypt&x=0&y=0

Enable Apache’s new configuration:
a2ensite teampass.mydomain.com
Restart Apache to activate it’s new configuration:
service apache2 restart
Prepare the suexec permissions files
echo "/var/www/teampass" >> /etc/apache2/suexec/www-data
echo "/var/www/teampass" > /etc/apache2/suexec/passwords
echo "TeamPass-master" >> /etc/apache2/suexec/passwords

IMPORTANT: We need to make sure that the cgi-script called by suexec is residing under the Server’s DocumentRoot for suexec to be allowed to run, therefore we installed the site under /var/www/teampass(which is located under the Server’s DocumentRoot(/var/www/) NOT meaning the VirtualHost’s DocumentRoot. A symlink is allowed here.

Preparing the MySQL database:

Create the new Database in MySQL:
Follow theses instructions:
1) Connect to mysql as root:
mysql -p -u root
PW: ******

2) Create the DB, user and user access rights:
CREATE DATABASE pwdb CHARACTER SET utf8 COLLATE utf8_bin;
GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP,ALTER,INDEX on pwdb.* TO 'pwuser'@'localhost' IDENTIFIED BY 'password';
flush privileges;

Quit Mysql:
quit;
3) Tip: To confirm if the permissions were granted successfully, log into the DB server with the PWDB DB user(pwuser) and run the command below:
SHOW GRANTS FOR 'pwuser'@'localhost';
4) Quit Mysql:
quit;
Installing TeamPass via the web interface:
In the browser:
https://teampass.mydomain.com/install/install.php
Fill-in the appropriate, paths, MySQL credentials and extra settings and save this configuration.
You are then ready to use TeamPass