msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

06 Apr 14 Switching from xm(xend) XenToolStack to XL XenToolStack in Delian Wheezy

Introduction:


While I upgraded my Xen DOM0 from Squeeze to Wheezy it was recommended to switch from the Xend(xm) Toolstack to XL Toolstack. Because I found very little info on how to do the switch. So here is a way do it on Wheezy.
Here we are assuming that you have installed Xen 4.1 Hypervisor on Debian Wheezy and you are still running the Xend ToolStack.
Since the Xend Toolstack will be rendered soon obsolete, it is therefore recommended to switch to the XL ToolStack.
Reference: http://wiki.xen.org/wiki/Network_Configuration_Examples_%28Xen_4.1%2B%29#Overview

Settings for Bridge networking on dual home: eth0 and eth1

Note: Unlike the Xend ToolStack, XL toolstack doesn’t create the bridges for eth0 and eth1, therefore they need to be created using the normal system network settings for them to be ready at boot time.
To make sure xend doesn’t try to configure the bridges, force xend to never try by reconfiguring the networking:
Edit /etc/xen/xend-config.sxp
(network-script dummy)
(vif-script vif-bridge)

INTERFACES
Edit the file: /etc/network/interfaces
Content: (make sure you replace the following example IPs etc. accordingly)
# The loopback network interface
auto lo
iface lo inet loopback
#
# eth0 and xenbr0 bridge
auto xenbr0
iface xenbr0 inet static
bridge_ports eth0
address 12.34.56.78
netmask 255.255.255.0
network 12.34.56.0
broadcast 12.34.56.255
gateway 12.34.56.254
bridge_stp off
post-up ethtool -K xenbr0 tx off
post-up ip link set xenbr0 promisc off
#
# eth1 and xenbr1 Bridge
auto xenbr1
iface xenbr1 inet static
bridge_ports eth1
address 192.168.0.1
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
bridge_stp off
post-up ethtool -K xenbr1 tx off

The command ifconfig will then show the eth0 and eth1 without IP and their respective Bridges (xenbr0,xenbr1) will have them.

Make sure xen Linux is listed as the default kernel when booting
mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
or
dpkg-divert --divert /etc/grub.d/09_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub

Switch to the XL Xen ToolStack
Edit /etc/default/xen
TOOLSTACK=xl
Edit /etc/xen/xl.conf and make sure the entries are as follows:
# automatically balloon down dom0 when xen doesn't have enough free memory to create a domain
autoballoon=1
# full path of the lockfile used by xl during domain creation
lockfile="/var/lock/xl"
# default vif script
vifscript="/etc/xen/scripts/vif-bridge"

If your DOMUs configurations are set to use pygrub as boot loader, then make sure the path to pygrub in the DOMU configuration file is correct as follows:
bootloader = '/usr/lib/xen-4.1/bin/pygrub'
In the same DOMU configuration file, make sure you are using the appropriate bridges with the network interfaces assignment for example:
vif = [ 'ip=12.34.56.18,mac=00:16:3E:D7:9C:F4,bridge=xenbr0' , 'ip=192.168.0.18,mac=00:16:3E:D7:9C:F6,bridge=xenbr1']
Finally, before we reboot the system we need to make sure we deactivate the xend(xm) toolstack and related features at boot time via:
update-rc.d xendomains defaults
update-rc.d xen defaults
/etc/init.d/xen restart
/etc/init.d/xendomains restart

Reboot
reboot
Start your DOMUs as usual with the command xl instead of xm.

Settings for Routing/Bridging networking on dual home: eth0 and eth1

Note: In the example above I’m using the Bridging method for both eth0 and eth1. In this present example I use routing for eth0(Internet connection) and bridging for eth1(internal private network). One might ask why use routing for eth0? The reason is mostly because of some type of routers/switches that the server provider uses makes it impossible to use bridging for eth0. The problem with some of those routers/switches is that, although they allow multiple IP addresses per network adapter, they allow only one MAC address per network adapter. For example Hetzner in Germany is using such routers/switches. This makes the use of bridging impossible for accessing the virtual machines via DOM0 from Internet. In this case the routing method is used for eth0. The other reason for using routing is also, besides the possible problems with the providers routers/switches, is the use of the redundancy software Heartbeat where two virtual machines share the same virtual IP. Heartbeat switches the IP from one VM to another, depending on the VM’s availability. In this case using bridging is also impossible because of some long refresh rates of the ARP tables of the switches in front of eth0. For example, if the MAC addr. is set for a certain IP and then Heartbeat gives that IP to another VM, then the MAC addr. for this IP will change but the ARP table of the switch will not follow until the switch refreshes its ARP table. This would result in downtime, which is exactly what heartbeat is supposed to avoid.

In this example below I use routing method for eth0 and bridging for eth1, consequently configure eth0 as a usual interface and create a bridge for eth1.
Xen XL toolstack will automatically create the proper vif* interfaces and routing entries for each VM while starting the VM.
To make sure xend doesn’t try to configure the bridges, force xend to never try by reconfiguring the networking:
Edit /etc/xen/xend-config.sxp
(network-script dummy)
(vif-script vif-route_eth0-bridge_eth1)

Edit the file: /etc/network/interfaces
Content: (make sure you replace the following example IPs etc. accordingly)
Here we use a very normal Network configuration without bridges.
# The loopback network interface
auto lo
iface lo inet loopback
#
# eth0
auto eth0
iface eth0 inet static
address 12.34.56.78
netmask 255.255.255.0
network 12.34.56.0
broadcast 12.34.56.255
gateway 12.34.56.254
#
# eth1 and xenbr1 Bridge
auto xenbr1
iface xenbr1 inet static
bridge_ports eth1
address 192.168.0.19
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
bridge_stp off
post-up ethtool -K xenbr1 tx off
post-up ip link set xenbr1 promisc off

Make sure xen Linux is listed as the default kernel when booting
mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
or
dpkg-divert --divert /etc/grub.d/09_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub

Switch to the XL Xen ToolStack
Edit /etc/default/xen
TOOLSTACK=xl
Edit /etc/xen/xl.conf and make sure the entries are as follows:
# automatically balloon down dom0 when xen doesn't have enough free memory to create a domain
autoballoon=1
# full path of the lockfile used by xl during domain creation
lockfile="/var/lock/xl"
# default vif script
vifscript="/etc/xen/scripts/vif-route_eth0-bridge_eth1"

Note: Here we use a script which will use routing for eth0 and bridging for eth1. Here we will create it.
touch /etc/xen/scripts/vif-route_eth0-bridge_eth1
chmod 755 /etc/xen/scripts/vif-route_eth0-bridge_eth1

Edit the file /etc/xen/scripts/vif-route_eth0-bridge_eth1.
Content:
#!/bin/sh
# Custom vif script which allows to combine routing for Internet and bridging for internal LAN
dir=$(dirname "$0")
IFNUM=$(echo ${vif} | cut -d. -f2)
if [ "$IFNUM" = "0" ] ; then
"$dir/vif-route" "$@"
else
"$dir/vif-bridge" "$@"
fi

PyGRUB
If your DOMUs configurations are set to use pygrub as boot loader, then make sure the path to pygrub in the DOMU configuration file is correct as follows:
bootloader = '/usr/lib/xen-4.1/bin/pygrub'
In the same DOMU configuration file, make sure you are using the appropriate MAC addresses with the network interfaces assignment for example:
vif = [ 'ip=12.34.56.18,mac=00:16:3E:D7:9C:F4' , 'ip=192.168.0.18,mac=00:16:3E:D7:9C:F6',bridge=xenbr1]
Setup the IP forwarding and ARP proxying in kernel:
Edit the file /etc/sysctl.conf
Either un-comment or add the following lines:
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
# ARP Proxying
net.ipv4.conf.eth0.proxy_arp = 1

To make this change take effect immediately run:
sysctl -p /etc/sysctl.conf

Finally, before we reboot the system we need to make sure we activate the proper toolstack and related features at boot time via:
update-rc.d xendomains defaults
update-rc.d xen defaults
/etc/init.d/xen restart
/etc/init.d/xendomains restart

Reboot
reboot
Start your DOMUs as usual with the command xl instead of xm.

20 Jan 14 Creating a XEN machine and Installing Group Office in Debian Wheezy

Introduction

In this Tutorial I will explain the steps I did to create a Xen Virtual Machine with minimal packages and then install the latest Group Office Web based Collaboration software. You’ll need to be fluent in Linux and Xen because I don’t explain much here.

Note: My hypervisor is Xen 4.0 in Debian Squeeze with xen-utils-4.0 package installed. I also use fictive domain(myserver.com) names and IP addresses just as example.

Creating the Xen Virtual Machine

This virtual machine will be created with the xen tools which bootstraps the creation of the VM.
Bootstrapping:
mkdir -p /virtual/xen/
cd /virtual/xen/
xen-create-image --dir=. --dist=wheezy --hostname=mail.myserver.com --size=20Gb --swap=2048Mb --ip=87.176.102.167 --gateway=87.176.102.254 --netmask=255.255.255.0 --memory=4096Mb --arch=amd64 --role=udev

Install the kernel and pyGrub
– Put the produced disk.img and swap.img in the proper path.
eg. in /virtual/xen/MAIL/
Mount the disk image in loop
mkdir /mnt/MAIL
mount /virtual/xen/MAIL/disk.img /mnt/MAIL -o loop,rw

Mount /sys, /proc, /dev and chroot to it
mount /proc /mnt/MAIL/proc -o bind
mount /sys /mnt/MAIL/sys -o bind
mount /dev /mnt/MAIL/dev -o bind
chroot /mnt/MAIL

Install the grub-legacy in VM
apt-get update
apt-get install grub-legacy linux-image-3.2.0-4-amd64 mc
mkdir /boot/grub
mcedit /boot/grub/menu.lst
CONTENT:
#----------------
default 0
timeout 2
#
title Debian GNU/Linux
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro
initrd /initrd.img
#
title Debian GNU/Linux (recovery mode)
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro single
initrd /initrd.img
#-------------

Leave chroot and unmount all.
exit
umount /mnt/MAIL/dev
umount /mnt/MAIL/sys
umount /mnt/MAIL/proc
umount /mnt/MAIL/

Adjust the VM xen configuration(/etc/xen/mail.server.com.cfg) as follows:
Replace the older kernel and initrd lines in the Xen DOMu configuration file as follows:
Example:
REPLACE:
kernel = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk = '/boot/initrd.img-2.6.32-5-xen-amd64'
WITH:
bootloader = '/usr/lib/xen-default/bin/pygrub'

Adjust the paths of the disks properly:
Example:
disk = [
'file:/virtual/xen/MAIL/disk.img,xvda2,w',
'file:/virtual/xen/MAIL/disk.swp,xvda1,w',
]

Test the pyGRUB configuration with the VM disk
Note: A GRUB menu should appear for a few seconds and then disappear with an error message. Ignore the error message. Most important is that the Grub menu appears.
/usr/lib/xen-default/bin/pygrub /virtual/xen/MAIL/disk.img
Start the VM
The Grub menu should appear and start booting.
xm create /etc/xen/mail.server.com.cfg -c

Installing Group-Office


Login as root and configure APT with the Group Office repositories
(REF: https://www.group-office.com/wiki/Installing_on_Debian_or_Ubuntu)
apt-get update, apt-get upgrade
echo -e "\n## Group-Office repository\ndeb http://repos.groupoffice.eu/ fivezero main" | tee -a /etc/apt/sources.list
gpg --keyserver hkp://keyserver.ubuntu.com:11371 --recv-keys 01F1AE44
gpg --export --armor 01F1AE44 | apt-key add -
apt-get update

Install Group Office
apt-get install groupoffice-mailserver postfix postfix-mysql dovecot-mysql dovecot-managesieved dovecot-sieve dovecot-lmtpd rsync mc
– Setting root password to MySQL server.
– Setting the domain name.

Note from Group Office before start of installation.
After installation is completed launch your browser and go to http://localhost/groupoffice/
or replace localhost with the hostname / IP of this machine.
The default login is username: admin and password: admin.
Enjoy Group-Office!

Setting root password to MySQL server:
Setting MySQL password of user:groupoffice-com DB groupofficecom :

Now some undesired installation features messages will appear:

[FAIL] Clamav signatures not found in /var/lib/clamav ... failed!
[FAIL] Please retrieve them using freshclam ... failed!
[FAIL] Then run '/etc/init.d/clamav-daemon start' ... failed!

To fix that:
apt-get -f install
freshclam
/etc/init.d/clamav-daemon start

All looking good now,
In Browser, try to login with your ‘admin’ password at:
http://mail.myserver.com/groupoffice

HINT about domains:
If you configure more domains in the admin web interface under ‘Email Domains’ menu item and try to create new users, only the original domain is offered to select as possible mailboxes for the new users. The newly configured domains are not listed. To remedy to that, you need to enter all of the domains this system may use into both GroupOffice and Amavis the configuration file:
IN /etc/groupoffice/config.php
$config['serverclient_domains']='domain1.com,domain2.com';

IN /etc/amavis/conf.d/05-domain_id
@local_domains_acl = ( ".$mydomain" , "domain1.com", "domain2.com")

Recommendation:
In order to raise your mail server’s general acceptance from large mailing servers like AOL, GMX, Yahoo, etc. it is recommended to:
– Configure your domain in DNS concerning the SPF1 and SPF2
– Configure in your mail server and DNS to send DKIM token.
See DKIM installation at: http://tipstricks.itmatrix.eu/?p=1494 for DKIM installation.
– Configure Postfix to use RBL SPAM filtering(see instructions below)

SPAM Reduction


This server is already providing some anti-spam protection but in some cases extra filter might need to be installed.

Add some more RBL SPAM Filtering

Note: In my mail server, almost every day about 800 to 2000 Spams are blocked using the RBL filtering method. So I do recommend it since its also quite simple as well.
Edit your Postfix main configuration file /etc/postfix/main.cf and replace the existing configuration with the following one. It contains the same configuration as the original except it adds to the list of RBL servers.
Postfix RBL settings:
smtpd_recipient_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination,
reject_invalid_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_rhsbl_sender dsn.fc-ignorant.org,
check_recipient_access hash:/etc/postfix/spam_rec_addr,
check_client_access hash:/etc/postfix/rbl_whitelist,
reject_rbl_client abuse.rfc-ignorant.org,
reject_rbl_client blackholes.brainerd.net,
reject_rbl_client bl.deadbeef.com,
reject_rbl_client dnsbl.antispam.or.id,
reject_rbl_client korea.services.net,
reject_rbl_client l1.spews.dnsbl.sorbs.net,
reject_rbl_client l2.spews.dnsbl.sorbs.net,
reject_rbl_client postmaster.rfc-ignorant.org,
reject_rbl_client query.bondedsender.org,
reject_rbl_client relays.bl.kundenserver.de,
reject_rbl_client relays.nether.net,
reject_rbl_client sbl.spamhaus.org,
reject_rbl_client zen.spamhaus.org,
reject_rbl_client ix.dnsbl.manitu.net,
reject_rbl_client spamguard.leadmon.net,
reject_rbl_client tr.countries.nerd.dk,
reject_rbl_client unsure.nether.net,
reject_rbl_client whois.rfc-ignorant.org,
reject_rbl_client l1.bbfh.ext.sorbs.net,
reject_rbl_client l2.bbfh.ext.sorbs.net,
reject_rbl_client psbl.surriel.com,
reject_rbl_client b.barracudacentral.org,
reject_rbl_client cbl.abuseat.org,
permit

# Allows to add a SPAM blacklist if needed (/etc/postfix/spam_addr)
smtpd_reject_unlisted_sender = yes
smtpd_sender_restrictions =
permit_mynetworks,
permit_sasl_authenticated,
reject_unknown_sender_domain,
reject_non_fqdn_sender,
check_sender_access hash:/etc/postfix/spam_addr
permit

# Allows to set regex rules to refuse certain know SPAM content in (/etc/postfix/spam_body_regex)
body_checks = regexp:/etc/postfix/spam_body_regex

Raising server delivery acceptance rate with DKIM

Ref: See instruction in http://tipstricks.itmatrix.eu/?p=1494

Raising security with TLS mail delivery

Ref: http://tipstricks.itmatrix.eu/?p=855
This feature allows postfix to send emails to remote mail servers using TLS encryption if the remote email server does support TLS transport otherwise clear text as usual.

Edit the file: /etc/postfix/main.cf and at the end enter the following:
smtp_tls_security_level = may

OPTIONAL: Enable DKIM verification in Amavis

This verification will warn you if a mail has been received which failed the DKIM verification.
Edit the file /etc/amavis/conf.d/60-groupoffice_defaults
Add the following line:
# Activating warnings for failed DKIM checked emails
$enable_dkim_verification = 1;

OPTIONAL: Enable the addition of ‘*****SPAM*****’ in header of suspicious emails.

Edit the file: /etc/amavis/conf.d/60-groupoffice_defaults
Add the following line:
$sa_spam_subject_tag = '***SPAM*** ';
You can then use this extra Subject tag to filter your mails and send them automatically in another directory like in ‘Spam’ directory.

OPTIONAL: Enable the Bayes Spam and Ham learning


For this we need to feed Spamassassin some Spam(Bad) and Ham(good) emails.
In this above configuration file the path of the files where Spamassassin learns is set to /home/spamd which doesn’t exist.
I don’t quite know how SA will discern the difference between Ham and spam So I’m doing it another way.
In order to feed it some spam mails each user should contribute to it as follows:
– The users create two new mail folder called exactly ‘Spam’ and ‘Ok’
– Then each time the user receives a definite SPAM email that is NOT tagged *****SPAM*****, he drops the email into his ‘Spam’ folder and forget about it.
– Each time he sees that a good mail has been erroneously tagged *****SPAM***** he drops a COPY a copy of the email into his ‘Ok’ folder and forgets about it.
The following configurations will ensure the following:
– The emails gathered in user’s ‘Spam’ and ‘Ok’ directories will be harvested by a cron job and be added automatically to /home/SA/spam(BAD) or /home/SA/ham(Good) directories respectively for sa-learn to learn from them.

We will create the directories and assign full access to the user ‘spamd’
mkdir -p /home/SA/spam
mkdir -p /home/SA/ham
chown spamd: /home/SA/spam /home/SA/ham

– SpamAssassin will regularly learn from it and offer a continuous increasing accuracy in detecting spams.
System cron job to harvest each day the user’s Spam mails and feed SA learning directory:
0 0 * * * /root/bin/SA-learn.sh

Creating the script:
touch /root/bin/SA-Learn.sh
chmod 755 /root/bin/SA-Learn.sh
Content of script /root/bin/SA-Learn.sh
#!/bin/bash
# make sure the lock file can be written in /home/spamd/
mkdir -p /home/spamd
chown -R spamd: /home/spamd
#
# Purpose: Feeds SA to learn the SPAM emails and GOOD emails
# Harvest the SPAM emails from users and deposit them in spam directory
for spamdir in $(find /home/vmail/ -type d -name '.Spam') ; do rsync -au $spamdir/cur/ /home/SA/spam/; done
#
# Harvest the HAM emails from users and deposit them in ham directory
for hamdir in $(find /home/vmail/ -type d -name '.Ham') ; do rsync -au $hamdir/cur/ /home/SA/ham/; done
#
# Now tell SA to learn from them
/usr/bin/sa-learn --spam -u spamd --dir /home/SA/spam/* -D
/usr/bin/sa-learn --ham -u spamd --dir /home/SA/ham/* -D
#
# Then deleted the mails it learned from to prevent relearning the same thing and accumulating old mails
rm -r /home/SA/spam/* /home/SA/ham/*
# We let the users delete their own spam and ham mails.
# eof

IMPORTANT: In order for the spam filtering/Dovecot sieve to work you have to make sure that the following line is disabled or not present in /etc/postfix/main.cf
#transport_maps = proxy:mysql:/etc/postfix/mysql_virtual_transports.cf
If present and enabled this above line overwrites the setting of transport agent and prevents postfix from using ‘dovecot’ as local transport by using ‘virtual’ instead. It’s been fixed in the GroupOffice version 5.0.44.

Enabling DNS White List (DNSWL) in Postfix


Resources: http://www.dnswl.org
DNSWL.org provides a Whitelist of known legitimate email servers to reduce the chances of false positives while spam filtering. To enable it edit the file /etc/postfix/main.cf and add the following line right before the postgrey line as follows:
smtpd_recipient_restrictions =
......
permit_dnswl_client list.dnswl.org,
(postgrey line below)
check_policy_service inet:127.0.0.1:10023,
permit

OPTIONAL:
To force using TLS for delivering to selected destinations and fail sending the mail if the destination server doesn’t support it.
smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
Content of /etc/postfix/tls_policy
example.com encrypt
.example.com encrypt

Hash the list:
postmap /etc/postfix/tls_policy

Allowing roaming SMTP use with SASL authentication

NOTE: Because the users’ credentials are stored in GroupOffice MySQL database we need to do the special authentication chain via dovecot which is configured to read Group Office database and its users data:
Configure SASL authentication
postconf -e 'smtpd_sasl_auth_enable = yes'
postconf -e 'smtpd_sasl_security_options = noanonymous'
postconf -e 'broken_sasl_auth_clients = yes'
postconf -e 'smtpd_sasl_type = dovecot'
postconf -e 'smtpd_sasl_path = private/auth'

Edit the file /etc/dovecot/conf.d/10-master.conf and enter inside the section ‘service auth {‘ insert the following lines as follows:

# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
group = postfix
}

PROBLEM: After I upgraded GroupOffice to version 5.0.55 I was no more capable to login my mail accounts.
See solution in http://www.dovecot.org/list/dovecot/2012-June/066444.html
The /var/log/mail.log said:
..... inbox=yes namespace missing
The solution is:
Edit the file /etc/dovecot/conf.d/15-mailboxes.conf and right under the line:
namespace inbox {
Insert the line
inbox=yes

Raising the SMTP security with TLS encryption

This generates a self-signed certificate. It is strongly recommended to buy a proper CA signed certificate for that purpose especially if your mail clients are not very computer literates. The security warning messages appearing in their mail clients because of self-signed certificates might scare them and lose trust in your service.
Generating the self-signed certificate:
mkdir -p /etc/ssl/mailserver/
cd /etc/ssl/mailserver/
openssl genrsa 1024 > mail-key.pem
chmod 400 mail-key.pem
openssl req -new -x509 -nodes -sha1 -days 365 -key mail-key.pem > mail-cert.pem

Enter the information required for the self signed certificate.
IMPORTANT: Enter your host name when ‘Common Name’ is asked.
Configuring postfix for TLS
Run the commands:
postconf -e 'smtpd_use_tls = yes'
postconf -e 'smtpd_tls_session_cache_timeout = 3600s'
postconf -e 'smtpd_tls_cert_file = /etc/ssl/mailserver/mail-cert.pem'
postconf -e 'smtpd_tls_key_file = /etc/ssl/mailserver/mail-key.pem'
postconf -e 'smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache'
postconf -e 'smtpd_tls_security_level = may'
postconf -e 'smtpd_tls_loglevel = 0'
postconf -e 'tls_random_source = dev:/dev/urandom'
postconf -e '#smtpd_tls_CAfile = /etc/postfix/cert/cacert.pem'

IMPORTANT NOTE: The intermediate CA and Key of certificate MUST be included in the certificate file (CRT) if you dont specify it in smtpd_tls_CAfile and smtpd_tls_key_file.

You must also make sure the “permit_sasl_authenticated” is present in the “smtpd_recipient_restrictions” configuration option. Edit this option in /etc/postfix/main.cf and add it right after “permit_mynetworks”.

Edit the file /etc/postfix/master.cf and add the following lines:
# Added to allow postfix to also listen to port 587(submission) well as port 465(smtps)
587 inet n - - - - smtpd
465 inet n - - - - smtpd

Adding extra postfix server security


Recommendation for better security by OpenVAS
postconf -e 'disable_vrfy_command=yes'

Restart postfix and dovecot

/etc/init.d/postfix restart
/etc/init.d/dovecot restart

APACHE2 Configuration

Configuring Redirection of ALL HTTP requests to HTTPS

Commands:
a2enmod ssl rewrite
a2ensite default-ssl

Edit the file: /etc/apache2/sites-available/default-ssl
and add the following lines at the very end after </IfModule>.
# Redirecting all HTTP to HTTPs
<IfModule mod_rewrite.c>
<IfModule mod_ssl.c>
RewriteEngine on
RewriteCond %{HTTPS} !^on$ [NC]
RewriteRule . https://%{HTTP_HOST}%{REQUEST_URI} [L]
</IfModule>
</IfModule>

Edit the file: /etc/apache2/sites-available/default
and add the following lines after LogLevel warn.
# Redirecting all HTTP to HTTPs
<IfModule mod_rewrite.c>
<IfModule mod_ssl.c>
RewriteEngine on
RewriteCond %{HTTPS} !^on$ [NC]
RewriteRule . https://%{HTTP_HOST}%{REQUEST_URI} [L]
</IfModule>
</IfModule>

Installing separate WebMail Interfaces

Configuration of RoundCube and Apache

Install Roundcube WebMail interface
apt-get install roundcube roundcube-plugins roundcube-plugins-extra
Prepare configuration for Roundcube and Apache
Edit file: /etc/roundcube/apache.conf
Uncomment the following 2 lines as follows:
Alias /roundcube/program/js/tiny_mce/ /usr/share/tinymce/www/
Alias /roundcube /var/lib/roundcube

Configuration of Squirrelmail and Apache

Installing Squirrelmail WebMail interface
apt-get install squirrelmail squirrelmail-decode
ln -s /etc/squirrelmail/apache.conf /etc/apache2/conf.d/squirrelmail

Optional:
For changing configuration of squirrelmail, run the command:
/usr/sbin/squirrelmail-configure

ADD authentication security to admin sites

Edit the file /etc/apache2/sites-available/default-ssl
and add the following lines at the very end:
# Authentication for private areas
<LocationMatch (/awstats|/awstats-icon|/mailgraph|/queuegraph|/phpmyadmin)>
AuthName "Private Area"
AuthType Basic
AuthUserFile /etc/apache2/web.auth
Require valid-user
</LocationMatch>

Run:
touch /etc/apache2/web.auth
For each admin user you need to create a password with this command
htpasswd /etc/apache2/web.auth <username>

Installing AWSTATS, MAILGRAPH and QUEUEGRAPH for Mail stats

As default Awstats creates a full new report at: 03:10 Hrs each day
it also refreshes the data every 10 minutes.

Installation:
apt-get install awstats mailgraph queuegraph
chown www-data. /var/lib/awstats
chmod o+r /var/log/mail.log

Edit /etc/awstats/awstats.local
Add the following lines:
# You can overrides config directives here.
# This is particularly useful for users with several configs for
# different virtual servers, who want to reuse common parameters.
# Also, this file is not updated with each new upstream release.
LogFile="perl /usr/share/doc/awstats/examples/maillogconvert.pl standard < /var/log/mail.log |"
LogType=M
LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"
LevelForBrowsersDetection=0
LevelForOSDetection=0
LevelForRefererAnalyze=0
LevelForRobotsDetection=0
LevelForWormsDetection=0
LevelForSearchEnginesDetection=0
LevelForFileTypesDetection=0
ShowMenu=1
ShowSummary=HB
ShowMonthStats=HB
ShowDaysOfMonthStats=HB
ShowDaysOfWeekStats=HB
ShowHoursStats=HB
ShowDomainsStats=0
ShowHostsStats=HBL
ShowAuthenticatedUsers=0
ShowRobotsStats=0
ShowEMailSenders=HBML
ShowEMailReceivers=HBML
ShowSessionsStats=0
ShowPagesStats=0
ShowFileTypesStats=0
ShowFileSizesStats=0
ShowBrowsersStats=0
ShowOSStats=0
ShowOriginStats=0
ShowKeyphrasesStats=0
ShowKeywordsStats=0
ShowMiscStats=0
ShowHTTPErrorsStats=0
ShowSMTPErrorsStats=1
SiteDomain=mail.myserver.com
LoadPlugin="geoipfree"

Create the a Apache configuration file: /etc/apache2/conf.d/awstats
and add this following content:
# Configuration for email-AWSTATS, MAILGRAPH and QUEUEGRAPH
Alias /awstats /usr/lib/cgi-bin/
Alias /awstats-icon/ /usr/share/awstats/icon/
Alias /mailgraph /usr/lib/cgi-bin/mailgraph.cgi
Alias /queuegraph /usr/lib/cgi-bin/queuegraph.cgi
Alias /queuegraph.cgi /usr/lib/cgi-bin/queuegraph.cgi
#
<Directory /usr/lib/cgi-bin/>
Options +execCGI
AddHandler cgi-script .pl .cgi
DirectoryIndex awstats.pl
</Directory>

Restart Apache service

service apache2 restart

List of URLs for this mail server:

Group Office https://mail.myserver.com/groupoffice
Roundcube Webmail https://mail.myserver.com/roundcube
Squirrelmail Webmail https://mail.myserver.com/squirrelmail
Mail stats(Awstats) https://mail.myserver.com/awstats
Mail traffic graph https://mail.myserver.com/mailgraph
Mail queues graph https://mail.myserver.com/queuegraph

Group Office forum and wikis addresses

https://www.group-office.com/
https://www.group-office.com/forum/
https://www.group-office.com/wiki/
https://www.group-office.com/wiki/Mailserver

15 Oct 13 Configuring Debian Wheezy Xen 4.1 Hypervisor

Introduction:


When I upgraded from Debian Lenny Xen 3.2.1 to Squeeze Xen 4.0, except for the DOMu hard disks names (from /dev/hdax to /dev/xvdax), I had very little changes to do and all went pretty well. Lately I wanted to upgrade from Squeeze Xen 4.0 to Wheezy Xen 4.1 and I had a few very unexpected and unwanted changes to do, here is what I ended up with which works.
Note: The difficulties may be due to some hardware incompatibility, I don’t know (eth0 & eth1: e1000e Intel(R) PRO/1000 Network). The following solution I finally got is dirty and I would appreciate if someone would help me clarify what is really going on and find a better and more elegant solution.

Constellation:


Here is what I needed:
eth0: connected to internet
eth1: connected to internal LAN (private LAN between hardware servers and the virtual machines)
In Squeeze the real interface was called ‘peth0’ and the bridge was called ‘eth0’
The same was the case for peth1 and eth1.
In Wheezy that worked for peth0 and eth0 but not for peth1 and eth1. The new Xen scripts(/etc/xen/scripts/network-bridge) is allowing to build only one bridge. I still don’t know why. If I commented that line, the script tried to build the second bridge for eth1 but it didn’t work.
Here is the code that only allows one bridge(eth0) to be created:
/etc/xen/scripts/network-bridge
Line 219:
if [ `brctl show | wc -l` != 1 ]; then
return
fi

Here is what I had used in Squeeze:
/etc/xen/xend-config.sxp
# Using the (self-made) eth0-eth1 network script wrapper
(network-script network-bridge-eth0-eth1)

/etc/xen/scripts/network-bridge-eth0-eth1
#!/bin/sh
# needs to get used in the xen-tools to creat 2 bridges: eth0 and eth1
/etc/xen/scripts/network-bridge "$@" netdev=eth0
sleep 4
/etc/xen/scripts/network-bridge "$@" netdev=eth1

Configuration in Wheezy


Extra problem:
When I started a virtual machine using only one bridge (eth0), it didn’t start and the following error showed up: ‘File already exists. Bringing up eth0 failed’ or something like that.
Someone suggested to add the following line(which works) to:
/etc/network/interfaces
pre-up ip addr del 98.184.49.14/24 dev eth0 2> /dev/null || true
From this point I had a properly working ‘peth0’ as physical interface and eth0 as bridge, but could not configure the eth1 the same way at all. I really tried many different configurations including creating the bridge by hand with brctl commands then adding the bridge in the /etc/network/interfaces etc. to no avail. Here is the only combination which worked so far, which is as I wrote previously, very dirty and not elegant and would hope to get to know a better way.

The solution:


The result is the following:

Physical Interface Bridge
================== ======
peth0 eth0
eth1 xenbr1

Here I will only show the content of the files which are concerned:
/etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
#
# The primary network interface
auto eth0
iface eth0 inet static
address 98.184.49.14
netmask 255.255.255.0
network 98.184.49.0
broadcast 98.184.49.255
gateway 98.184.49.254
#
pre-up ip addr del 98.184.49.14/24 dev eth0 2> /dev/null || true
#
iface eth1 inet manual
#
iface xenbr1 inet manual
address 192.168.0.5
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
bridge_ports eth1
bridge_maxwait 0

Note: in the above file the Internet IP is just an example and not the real IP of my server.

/root/bin/start_Xen_env.sh
#!/bin/bash
# Constants
eth1_IP=$(grep -i flash2.srv /etc/hosts | awk '{print $1}')
/etc/xen/scripts/network-bridge start netdev=eth0
/sbin/ifup eth1
/sbin/ifup xenbr1
/sbin/ifconfig eth1 $eth1_IP
/sbin/ifconfig xenbr1 up
route del -net 192.168.0.0 netmask 255.255.255.0 dev eth1
route add -net 192.168.0.0 netmask 255.255.255.0 dev xenbr1

Example of network configuration of a DOMu Xen configuration:

.....
vif = [ 'ip=98.184.53.164,mac=00:16:3E:78:1C:64,bridge=eth0' , 'ip=192.168.0.164,mac=00:16:3E:D7:9C:64,bridge=xenbr1' ]
.....

Process:


Here is the sequence of event that needs to happen after reboot before I can start any DOMUs.
– After reboot the eth0 interface is configured as normal interface connected to Internet
– eth1 and xenbr1 interfaces are configured but not activated.
– I login as root to the server through eth0 and run the script /root/bin/start_Xen_env.sh manually:
These steps prepare the Xen networking environment which I need to be able to start DOMUs.
Note: for some strange reason which I would like to know why, if I ran this above script from root crontab as follows, it failed to create the peth0.
@reboot /bin/sleep 10 ; /root/bin/start_Xen_env.sh
If I run the same script by hand after a reboot then all goes well.

After I ran the script, here is the result of the output of the brctl and ifconfig commands:
flash2 ~ # brctl show
bridge name bridge id STP enabled interfaces
eth0 8000.00259033cc14 no peth0
xenbr1 8000.00259033cc15 no eth1

flash2 ~ # ifconfig
eth0 Link encap:Ethernet HWaddr 00:25:90:33:cc:14
inet addr:98.184.49.14 Bcast:98.184.49.255 Mask:255.255.255.0
inet6 addr: fe80::225:90ff:fe33:cc14/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:209 errors:0 dropped:4 overruns:0 frame:0
TX packets:44 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:11687 (11.4 KiB) TX bytes:5364 (5.2 KiB)
.
eth1 Link encap:Ethernet HWaddr 00:25:90:33:cc:15
inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:320 (320.0 B) TX bytes:314 (314.0 B)
Interrupt:17 Memory:fb6e0000-fb700000
.
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1544 errors:0 dropped:0 overruns:0 frame:0
TX packets:1544 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:102145 (99.7 KiB) TX bytes:102145 (99.7 KiB)
.
peth0 Link encap:Ethernet HWaddr 00:25:90:33:cc:14
inet6 addr: fe80::225:90ff:fe33:cc14/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:436462 errors:0 dropped:11264 overruns:0 frame:0
TX packets:4791 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:29190641 (27.8 MiB) TX bytes:1020853 (996.9 KiB)
Interrupt:16 Memory:fb5e0000-fb600000
.
xenbr1 Link encap:Ethernet HWaddr 00:25:90:33:cc:15
inet6 addr: fe80::225:90ff:fe33:cc15/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:5 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:230 (230.0 B) TX bytes:238 (238.0 B)

Any improvement or suggestions regarding this issue is very welcome.

09 Jul 13 Basics of Linux Logical Volume Manager

Concept:

Physical Volume(PV): Real partition which can be assigned to a pool of resource to create/manage logical volumes
Logical Volume Group(LG): Pool of resources (partitions) which can be used to create/manage Logical volumes
Logical Volumes(LV): Logical Partitions which can be formatted and used just like a normal partition. The difference is that these Logical Partitions can be easily created and resized at will, as long as there is enough resources in the Logical Volume Group (resources pool)

Basic installation instructions:

Install The logical Volume Manager package
apt-get install lvm2
Start the LVM
This will simply load the kernel module ‘dm_mod’ and do some LVM environment checks.
No daemons will be started.
/etc/init.d/lvm2 start
OR probably better for Ubuntu server:
echo 'dm_mod' >> /etc/modules
modprobe dm_mod

Scenario: I have a partition(/dev/sda3) which I want to assign into a pool of resources which will be used to create flexible logical volumes.

STEPS:

Assign a Physical Volume to be used by LVM
pvcreate /dev/sda3
pvdisplay

Create a Logical Volume Group (data1)
This will be a pool of assigned Physical Volumes which can be used to create/resize etc Logical Volumes
vgcreate data1 /dev/sda3
vgdisplay

Create the logical volume(275 GB)
This volume can be used as a normal partition in system
lvcreate -n data -L 275g data1
lvdisplay

Format the Volume
mkfs.ext3 /dev/data1/data
Mount and use the volume like a normal partition
mkdir /mnt/lvm01
mount /dev/data1/data /mnt/lvm01

You can take a look at the various commands to control the Logical Volumes
man -k '^pv'
man -k '^vg'
man -k '^lv'

That’s it for the minimum.

Resizing Volumes

Here are some other useful commands for resizing the volumes.
Assuming that we now want to add 20GB of space(total 295GB) to the existing Volume(/dev/data1/data)

Steps:

Unmount the existing volume
umount /dev/data1/data
Add an extra partition(/dev/sdb1) to the Logical Volumes Group (data1)
vgextend data1 /dev/sdb1
Resize our volume to 295GB
lvextend -L295G /dev/data1/data
Resize the file system to take possession of the new space
resize2fs /dev/data1/data
Remount the Volume
mount /dev/data1/data /mnt/lvm01
Check the new space
df -h /mnt/lvm01

Note: There is a good extention to this article which shows how to resize volumes etc. It is in German but can be deducted easily:
http://www.server-wissen.de/linux-debian/lvm-einrichten-eines-logical-volumes/

16 Mar 13 Installing an Ubuntu 12.0.4 LTS as Xen DOMu in Debian Squeeze hypervisor

Lately I needed to install Zimbra 8.0.3 which only installs easily in an Ubuntu 10.0.4 or 12.0.4 LTS system.
So I decided for that to install an Ubuntu 12.0.4 LTS as Xen DOMu in a Debian Squeeze Xen Hypervisor and here is how I did it.

The following commands can be put into a runnable bash script. Adapt the constants below to your system and simply run it, or simply(recommended) cut and paste the commands in bash terminal. Then afterwards (indicated below)inside the chroot the commands need to be done manually.

#!/bin/bash
# Settings some user changeble constants
virtbase=/virtual/xen
tempdir=/virtual/temp
VM=UBUNTU
#
# Installing required packages
apt-get update
apt-get install gnupg
#
# Creating, formating and mounting an empty 10GB VM
mkdir -p $virtbase/$VM
dd if=/dev/zero of=$virtbase/$VM/disk.img bs=1GB count=10
mkfs.ext3 $virtbase/$VM/disk.img
mkdir -p /mnt/$VM
mount $virtbase/$VM/disk.img /mnt/$VM -o loop,rw
#
# create and format a 2GB swap file
dd if=/dev/zero of=$virtbase/$VM/swap.img bs=1GB count=2
mkswap $virtbase/$VM/swap.img
#
# Creating a temporary ubuntu bootstrap environment
mkdir -p $tempdir
cd $tempdir
#
# Getting the ubuntu bootstrapper and keys
wget "http://de.archive.ubuntu.com/ubuntu/pool/main/d/debootstrap/debootstrap_1.0.40~precise1_all.deb"
ar -xf debootstrap_1.0.40~precise1_all.deb
tar xzf data.tar.gz
tar xzf control.tar.gz
wget "http://archive.ubuntu.com/ubuntu/pool/main/u/ubuntu-keyring/ubuntu-keyring_2011.11.21.tar.gz"
tar xzf ubuntu-keyring_2011.11.21.tar.gz
#
# Bootstrapping Ubuntu system into the mounted empty VM
DEBOOTSTRAP_DIR=$tempdir/usr/share/debootstrap usr/sbin/debootstrap --arch amd64 --keyring=ubuntu-keyring-2011.11.21/keyrings/ubuntu-archive-keyring.gpg precise /mnt/$VM http://de.archive.ubuntu.com/ubuntu/
#
# Chroot to the new VM to prepare it
mount /sys /mnt/$VM/sys -o bind
mount /proc /mnt/$VM/proc -o bind
chroot /mnt/$VM/

From this point you need to cut-and-paste the commands in the console manually

Note: Ignore minor errors during apt-get install because we are in chroot and many things will not behave as normal but install properly.

Expand the repositories to get more packages to be available
sed -i 's/precise main/precise main restricted multiverse universe/' /etc/apt/sources.list
apt-get update

Remove server-useless resolvconf
apt-get remove resolvconf
Give a password to root user
passwd
Install the kernel and old grub(for virtual booting), ssh service and and some useful programs
apt-get install linux-image-virtual grub nano mc ssh fail2ban less manpages libgmp3c2 libperl5.14 sysstat sqlite3
Stop fail2ban
/etc/init.d/fail2ban stop
Add the correct mounts in /etc/fstab
echo "/dev/xvda1 / ext3 noatime,nodiratime,errors=remount-ro,usrquota,grpquota 0 1" > /etc/fstab
echo "/dev/xvda2 none swap sw 0 0" >> /etc/fstab

Create the grub config file
mkdir -p /boot/grub
nano /boot/grub/menu.lst

Content:
default 0
timeout 2
#
title Debian GNU/Linux
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro
initrd /initrd.img
#
title Debian GNU/Linux (recovery mode)
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro single
initrd /initrd.img

Here we need to configure the server name, and network addresses
Constants:(Create a script (/root/loadvars.sh) with the following content and fill in those Variables definitions accordingly.)
InternetIP=xx.200.75.211
LANIP=192.168.100.211
Gateway=xx.200.75.209
Broadcast=xx.200.75.223
Netmask=255.255.255.240
Hostname=ubuntu.myserver.com
DNS1=192.168.100.1
DNS2=xx.187.164.20
DNS3=xx.201.0.34
search="srv mysqrver.com"

Call/load the script which will set the proper variables for the processes below:
. /root/loadvars.sh

Configure the server name
echo "$Hostname" > /etc/hostname
echo "$Hostname" > /etc/mailname

Configure /etc/hosts
echo "$InternetIP $Hostname $(echo $Hostname | cut -d. -f1)" >>/etc/hosts
echo "$LANIP $(echo $Hostname | cut -d. -f1).srv" >>/etc/hosts

Configure the resolver
cat > /etc/resolv.conf << EOF
nameserver $DNS1
nameserver $DNS2
nameserver $DNS3
search $search
EOF

Configure the netwwork
cat > /etc/network/interfaces << EOF
# /etc/network/interfaces
auto lo
iface lo inet loopback
#
auto eth0
iface eth0 inet static
address $InternetIP
netmask $Netmask
broadcast $Broadcast
gateway $Gateway
#
auto eth1
iface eth1 inet static
address $LANIP
netmask 255.255.255.0
EOF

Leave chroot and unmount all of the VM
exit
umount /mnt/UBUNTU/sys
umount /mnt/UBUNTU/proc
umount /mnt/UBUNTU

Configure the xen configuration file for UBUNTU
Note: Replace the {InternetIP} and {LANIP} with the appropriate ones.
nano $virtbase/$VM/${VM}.cfg
Content:
bootloader = '/usr/lib/xen-default/bin/pygrub'
memory = '2500'
root = '/dev/xvda1 ro'
disk = [
'file:$virtbase/UBUNTU/disk.img,xvda1,w',
'file:$virtbase/UBUNTU/swap.img,xvda2,w',
]
name = 'UBUNTU'
vif = [ 'ip={InternetIP},mac=00:16:3E:3D:6B:11,bridge=eth0' , 'ip={LANIP},mac=00:16:3E:D7:9C:11,bridge=dummy0']
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
vnc = 0
vncunused = 0
extra = "console=hvc0"
vcpus = 1
cpus ="1"

Create a symlink of the xen config file
ln -s $virtbase/$VM/${VM}.cfg /etc/xen/${VM}.cfg
Start the VM
xm create /etc/xen/${VM}.cfg
Connect to the VM via SSH
ssh root@$LANIP
(Give the root password you configured with passwd previoulsy)

07 Mar 13 Using pyGRUB on Squeeze/Wheezy to boot a domU kernel

This adapded article is based on the following Debian Wiki article:
http://wiki.debian.org/PyGrub

In this article I assume:
– The reader is familiar with Linux and Xen Hypervisor
– The DOMu system partition is using a file image instead of a physical partition.

Introduction:
This method offers the advantage of loading the kernel which is installed in the DOMu. For this we use a Python script(/usr/lib/xen-default/bin/pygrub) which is located on DOM0. It understands the EXT3 filesystem of DOMu, loads and starts the Kernel(vmlinuz) and Ramdisk(initrd.img) files from DOMu defined in its Grub legacy configuration file(/boot/grub/menu.lst).

Prerequisites:
– The DOMu filesystem is EXT3
– The disk devices are using the format ‘xvda1/2/3…’
– The system image is the first listed in the DOMu xen configuration file.
Example:
root = '/dev/xvda1 ro'
disk = [
'file:/xen/VMS/disk.img,xvda1,w',
'file:/xen/VMS/disk.swp,xvda2,w',
]

On the DOMu:

Start the DOMu as done before and:
Make sure in the DOMu /etc/fstab that the mount points refer to /dev/xvd{a,b}1, /dev/xvd{a,b}2, …..
Example:
/dev/xvda1 / ext3 noatime,nodiratime,errors=remount-ro,usrquota,grpquota 0 1
/dev/xvda2 none swap sw 0 0

For Debian Squeeze DOMu install the Debian grub-legacy and latest kernel packages:
aptitude install grub-legacy linux-image-2.6-xen-amd64
For Debian Wheezy DOMu use this command:
aptitude install linux-image-amd64 grub-legacy
Create the pyGrub configuration file based on the system storage device (/dev/xvda1):
mkdir /boot/grub
vim /boot/grub/menu.lst

Content:
default 0
timeout 2
#
title Debian GNU/Linux
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro
initrd /initrd.img
#
title Debian GNU/Linux (recovery mode)
root (hd0,0)
kernel /vmlinuz root=/dev/xvda1 ro single
initrd /initrd.img

Stop DOMu.
halt

IMPORTANT: After every kernel update in the DOMU Debian tries to overwrite this file.
After each Kernel update issue the following commands:

mv /boot/grub/menu.lst /boot/grub/menu.lst.debian
mv /boot/grub/menu.lst~ /boot/grub/menu.lst

Reason: After a kernel upgrade the install script recreates its own grub.lst file which is not compatible with pygrub.

In DOM0:


Check that grub was properly installed on the domU. For DOM0 in Debian Squeeze with the command:
/usr/lib/xen-default/bin/pygrub /xen/VMS/disk.img
If your DOM0 is Debian Wheezy, then use this command instead:
/usr/lib/xen-4.1/bin/pygrub /xen/VMS/disk.img

This should give you a Grub boot menu as if the system will start but will kick out a couple of sec later with an error message. Ignore the error message, the presence of the boot menu was the indicator that everything is ready in the DOMu.

Replace the older kernel and ramdisk lines in the Xen DOMu configuration file as follows:
Example:
REPLACE:
kernel = '/boot/vmlinuz-2.6.32-5-xen-amd64'
ramdisk = '/boot/initrd.img-2.6.32-5-xen-amd64'

WITH:
bootloader = '/usr/lib/xen-default/bin/pygrub'
For Debian Wheezy, use the following entry instead.
bootloader = '/usr/lib/xen-4.1/bin/pygrub'

That was it. Now start the DOMu.

02 Aug 12 Creating a new Xen DOMU Debian Squeeze image file(bootstrapping)

Here is a short example with many options given which creates a new Debian Squeeze Xen 4.0x image file.

The command will create a 25GB Debian Squeeze Image file in the LVM:’vservers’ which will run the 32 bit kernel 2.6.26-2-xen-686 with 2GB of RAM and 1GB of SWAP file with root password ‘changeme’.

Note: remember to replace the values enclosed inside {….}. Then delete these { and } as well.
xen-create-image --hostname= --size=25Gb --swap=1024Mb --ip={xxx.xxx.xxx.xxx} --netmask={xxx.xxx.xxx.xxx} \
--gateway={xxx.xxx.xxx.xxx} --force --lvm=vservers --memory=2048Mb --arch=i386 \
--kernel=/boot/vmlinuz-2.6.26-2-xen-686 --debootstrap --dist=squeeze \
--initrd=/boot/initrd.img-2.6.26-2-xen-686 \
--mirror=http://ftp2.de.debian.org/debian/ --passwd "changeme"

15 Jun 11 Create a mixed (routed & bridged) private VLAN for Xen Virtual Machines

The most common way of configuring the networking in Xen environment is by using bridges.
In the case of servers rented at Hetzner provider (Germany) this would not work because the infrastructure is allowing only one MAC address per server. It allows for multiple IPs but only one MAC address.
To circumvent this situation we are then obliged to use the ‘Routed’ networking for eth0 explained in:
http://tipstricks.itmatrix.eu/?p=462. This explained method uses also ‘routing’ method for internal VLAN.
This works for Xen 3.2.1 environment BUT!! if you installed Xen 4.0.x from Debian Squeeze the routing method for the internal LAN doesn’t work as with Xen 3.2.1. (At least I could not get it to work).
One solution is to do a mixed network environment:
ETH0 in ‘Routed’ (needed in Hetzner servers)
DUMMY0 in Bridged (Needed in Xen 4.0.x)
This will create a bridge in Xen DOM0 and make available an interface(eth1) in all the DOMx (in our case ‘vsystem1’).
(This howto was inspired from the link: http://wiki.xensource.com/xenwiki/XenNetRoutingWithPrivateNetwork)

NOTE:
If you want to do the same with a real interface eg. eth1 the just omit the step 1 below and replace the word ‘dummy0’ with ‘eth1’

Building a dummy interface and bridge in DOM0

1) Add the dummy interface driver to the auto-loaded moludes
echo dummy >> /etc/modules
2) Configure the network interface:
/etc/network/interfaces
auto dummy0
iface dummy0 inet static
address 192.168.100.1
netmask 255.255.255.0

3) Bring up the dummy0 interface:
ifup dummy0
4) Create a network settings wrapper:
/etc/xen/scripts/network-route-eth0_bridge-dummy0
#!/bin/sh
dir=$(dirname "$0")
"$dir/network-route" "$@" netdev=eth0
"$dir/network-bridge" "$@" netdev=dummy0
echo 1 >/proc/sys/net/ipv4/ip_forward

5) Set the running rights to the script
chmod 755 /etc/xen/scripts/network-route-eth0_bridge-dummy0
6) Instead of using the default network-script use the above new wrapper script:
/etc/xen/xend-config.sxp
(network-script network-route-eth0_bridge-dummy0)
7) Create manually the bridge for the dummy0 interface for now instead of booting.
(Because of the wrapper script it will be created automatically at boot-up)
Run the command:
/etc/xen/scripts/network-bridge start netdev=dummy0 antispoof=no
You should get the following message and then the normal shell prompt:
'Waiting for pdummy0 to negotiate link.'
8)Check if the new bridge is present:
ifconfig pdummy0
Good example of result:
pdummy0 Link encap:Ethernet HWaddr d2:1b:97:ac:b0:74
inet6 addr: fe80::d01b:97ff:feac:b074/64 Scope:Link
UP BROADCAST RUNNING NOARP PROMISC MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:888 (888.0 B)

Create the vif script which will set the vifxx.0 as ‘routed’ and vifxx.1 as bridged.(Used to connect the network DOMu networking)
9) Create a network vifxx settings wrapper:
/etc/xen/scripts/vif-route-eth0_bridge-dummy0
#!/bin/sh
# Custom vif script which allows to combine routing for Internet and bridging for internal LAN
dir=$(dirname "$0")
IFNUM=$(echo ${vif} | cut -d. -f2)
if [ "$IFNUM" = "0" ] ; then
"$dir/vif-route" "$@"
else
"$dir/vif-bridge" "$@"
fi

10) Set the running rights to the script
chmod 755 /etc/xen/scripts/vif-route-eth0_bridge-dummy0
11) Instead of using the default ‘vif-script’ use the above new wrapper script:
/etc/xen/xend-config.sxp
(vif-script vif-route-eth0_bridge-dummy0)
Modify the file /etc/xen/scripts/vif-common.sh
Around line 135 replace the following line
ip addr show "$1" | awk "/^.*inet.*$1\$/{print \$2}" | sed -n '1 s,/.*,,p'
with
ip -4 -o addr show primary dev $1 | awk '$3 == "inet" {print $4; exit}' | sed 's#/.*##'

DOMU network configuration


1) Configure your Xen Domx virtual machine for eth1:
(It should be written all in one line)
/etc/xen/vsystem1.cfg
vif = [ 'ip=91.184.57.186,mac=00:16:3E:D7:9C:F4,bridge=eth0' , 'ip=192.168.100.100,mac=00:16:3E:D7:9C:F6,bridge=dummy0']

2) Bring down your vsystem1 DOMx machine
xm shutdown vsystem1
3) Mount the virtual disk in loop (for configuring the eth1 interface in it)
(Here we are assuming the virtual disk is /Xen/domains/vsystem1/disk.img)
mkdir /mnt/vsystem1
mount -o loop,rw /Xen/domains/vsystem1/disk.img /mnt/vsystem1

4) Configure the eth1 in the virtual disk
vim /mnt/vsystem1/etc/network/interfaces
Add the following lines and save the file:
auto eth1
iface eth1 inet static
address 192.168.100.100
netmask 255.255.255.0

5) Unmount the virtual disk
umount /mnt/vsystem1
6) Start the virtual machine
xm create /etc/xen/vsystem1.cfg -c
7) Login as root and check that the eth1 is configured
ifconfig eth1

NOTE:
To configure more virtual machines to use eth1 repeat the above steps 9 to 15 for each virtual machine.

NOT TO FORGET:
– You’ll need to configure your firewall in the DOM0 to forward the packets from one machine to another
– Do change the MAC address for each virtual machine you configure this way
– Set the ip_forwarding in the kernel of DOM0
echo 1 >/proc/sys/net/ipv4/ip_forward

06 Jun 11 Installing Xen 4.x in Debian Squeeze (6.0)

Here are the minimum steps necessary to install Xen Hypervisor 4.x on Debian Squeeze.
Installing the hypervisor and tools related
aptitude install xen-linux-system-2.6-xen-amd64 xen-hypervisor-4.0-amd64 linux-image-xen-amd64 xen-qemu-dm-4.0
Change some system configuration
Edit /etc/modules and add the following entry. To make sure the module ‘loop’ can create 64 loop connections.
The entry ‘loop’ might already be there, then just add its parameters.
# Making sure enough free loop devices are created
loop max_loop=64

To make sure the ’4gb seg fixup errors’ message don’t flood the syslog
echo 'hwcap 0 nosegneg' > /etc/ld.so.conf.d/libc6-xen.conf && ldconfig
Create the forgotten Xen script file in scripts directory.
touch /etc/xen/scripts/hotplugpath.sh
chmod 755 /etc/xen/scripts/hotplugpath.sh
Content of /etc/xen/scripts/hotplugpath.sh
SBINDIR="/usr/sbin"
BINDIR="/usr/bin"
LIBEXEC="/usr/lib/xen/bin"
LIBDIR="/usr/lib"
SHAREDIR="/usr/share"
PRIVATE_BINDIR="/usr/lib/xen/bin"
XENFIRMWAREDIR="/usr/lib/xen/boot"
XEN_CONFIG_DIR="/etc/xen"
XEN_SCRIPT_DIR="/etc/xen/scripts"

If you are installing Xen in a Hetzner server it is recommended to do the following changes in kernel options to insure a proper reboot:
Edit the file /etc/default/grub and replace the line:
GRUB_CMDLINE_LINUX_DEFAULT="nomodeset"With:
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 acpi=ht"

Making sure the system starts with the hypervisor (normally not the case and xend can’t start)
mv /etc/grub.d/10_linux /etc/grub.d/50_linux
update-grub2

Making sure xend doesn’t start too early, otherwise it doesn’t create its necessary bridge(s) (peth0…)
mv /etc/rc2.d/S01xend /etc/rc2.d/S20xend
reboot

NOTE: In the DOMUs you can delete the following init symlinks:
(They are not needed and only brings errors at boot time)
/etc/rc2.d/S12acpid
/etc/rcS.d/S08hwclockfirst.sh

14 Dec 10 Assigning CPUs to Xen virtual Machines

Introduction:
I’m renting a 8 CPU machine at Hetzner and I wanted to improve the performance.
At the beginning I was distributing many CPUs to the virtual machines which needed it and less cpus to others.
This included that some cpus would be shared by more than one Xen virtual machine.
I noticed that the performance was quite sluggish.
After having read about it in Internet I opted to assign 1 or maximum 2 CPUs per machine including DOM0
and no CPUs would be shared by the machines that need performance.
At the end this really improved the performance significantly because Xen has less CPU switching to do.

So here is the plan:
DOM0 : CPU 0 & 1
DOMU1 : CPU 2 & 3
DOMU2 : CPU 4 & 5
DOMUx : Shared CPUs 6 & 7 (some machines might get one CPU and others 2 CPUs

The principle is also that we assign vCPUs to Physical CPUs.
I have seen in Internet some ways to assign CPUs to DOM0 but somehow the following method doesn’t seem to work.
in /boot/grub/menu.lst
dom0_max_vcpus=2 dom0_vcpus_pin

Distributing CPUs for DOM0 and DOMUs


Manually assign a vCPU to a physical CPU
Syntax:
xm vcpu-pin ID VCPU CPU
eg.
xm vcpu-pin 14 0 7
Assigns vCPU 0 to physical CPU 7 for the DOMU-ID 14

So here is my method for the DOM0.
Note: Unfortunately this has to be done every time I boot, therefore putting the following content in a system init script which starts after the ‘xend’ daemon has started could be helpful.
DOM0
xm vcpu-pin 0 0 0
xm vcpu-pin 0 1 0
xm vcpu-pin 0 2 0
xm vcpu-pin 0 3 0
xm vcpu-pin 0 4 1
xm vcpu-pin 0 5 1
xm vcpu-pin 0 6 1
xm vcpu-pin 0 7 1

This assigns only the first 2 physical CPUs to 8 vCPUs used by DOM0


Command for listing the cpu distribution:
xm vcpu-list
DOMUs
Assigning CPUs to DOMUs can be done by hand as seen above, but I recommend doing it in their respective Xen configuration files as follows:

Edit the DOMu configuration file: eg. /etc/xen/domu_test.cfg
Add this content
vcpus = 2
cpus = ["4","5"]

This would assign 2 CPUs to the DOMu which will be CPU 2 and 3.

After these configuration changes the DOMU has to be shut down and then restarted.
A ‘Reboot’ command from DOM0 or inside the DOMU will not work.