IT Tips and Tricks

Introduction:

While I upgraded my Xen DOM0 from Squeeze to Wheezy it was recommended to switch from the Xend(xm) Toolstack to XL Toolstack. Because I found very little info on how to do the switch. So here is a way do it on Wheezy.
Here we are assuming that you have installed Xen 4.1 Hypervisor on Debian Wheezy and you are still running the Xend ToolStack.
Since the Xend Toolstack will be rendered soon obsolete, it is therefore recommended to switch to the XL ToolStack.
Reference: http://wiki.xen.org/wiki/Network_Configuration_Examples_%28Xen_4.1%2B%29#Overview

Settings for Bridge networking on dual home: eth0 and eth1

Note: Unlike the Xend ToolStack, XL toolstack doesn’t create the bridges for eth0 and eth1, therefore they need to be created using the normal system network settings for them to be ready at boot time.
To make sure xend doesn’t try to configure the bridges, force xend to never try by reconfiguring the networking:
Edit /etc/xen/xend-config.sxp
(network-script dummy)
(vif-script vif-bridge)
INTERFACES
Edit the file: /etc/network/interfaces
Content: (make sure you replace the following example IPs etc. accordingly)
# The loopback network interface
auto lo
iface lo inet loopback
#
# eth0 and xenbr0 bridge
auto xenbr0
iface xenbr0 inet static
bridge_ports eth0
address 12.34.56.78
netmask 255.255.255.0
network 12.34.56.0
broadcast 12.34.56.255
gateway 12.34.56.254
bridge_stp off
post-up ethtool -K xenbr0 tx off
post-up ip link set xenbr0 promisc off
#
# eth1 and xenbr1 Bridge
auto xenbr1
iface xenbr1 inet static
bridge_ports eth1
address 192.168.0.1
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
bridge_stp off
post-up ethtool -K xenbr1 tx off

The command ifconfig will then show the eth0 and eth1 without IP and their respective Bridges (xenbr0,xenbr1) will have them.

Make sure xen Linux is listed as the default kernel when booting
mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
or
dpkg-divert --divert /etc/grub.d/09_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub
Switch to the XL Xen ToolStack
Edit /etc/default/xen
TOOLSTACK=xl
Edit /etc/xen/xl.conf and make sure the entries are as follows:
# automatically balloon down dom0 when xen doesn't have enough free memory to create a domain
autoballoon=1
# full path of the lockfile used by xl during domain creation
lockfile="/var/lock/xl"
# default vif script
vifscript="/etc/xen/scripts/vif-bridge"
If your DOMUs configurations are set to use pygrub as boot loader, then make sure the path to pygrub in the DOMU configuration file is correct as follows:
bootloader = '/usr/lib/xen-4.1/bin/pygrub'
In the same DOMU configuration file, make sure you are using the appropriate bridges with the network interfaces assignment for example:
vif = [ 'ip=12.34.56.18,mac=00:16:3E:D7:9C:F4,bridge=xenbr0' , 'ip=192.168.0.18,mac=00:16:3E:D7:9C:F6,bridge=xenbr1']
Finally, before we reboot the system we need to make sure we deactivate the xend(xm) toolstack and related features at boot time via:
update-rc.d xendomains defaults
update-rc.d xen defaults
/etc/init.d/xen restart
/etc/init.d/xendomains restart

Reboot
reboot
Start your DOMUs as usual with the command xl instead of xm.

Settings for Routing/Bridging networking on dual home: eth0 and eth1

Note: In the example above I’m using the Bridging method for both eth0 and eth1. In this present example I use routing for eth0(Internet connection) and bridging for eth1(internal private network). One might ask why use routing for eth0? The reason is mostly because of some type of routers/switches that the server provider uses makes it impossible to use bridging for eth0. The problem with some of those routers/switches is that, although they allow multiple IP addresses per network adapter, they allow only one MAC address per network adapter. For example Hetzner in Germany is using such routers/switches. This makes the use of bridging impossible for accessing the virtual machines via DOM0 from Internet. In this case the routing method is used for eth0. The other reason for using routing is also, besides the possible problems with the providers routers/switches, is the use of the redundancy software Heartbeat where two virtual machines share the same virtual IP. Heartbeat switches the IP from one VM to another, depending on the VM’s availability. In this case using bridging is also impossible because of some long refresh rates of the ARP tables of the switches in front of eth0. For example, if the MAC addr. is set for a certain IP and then Heartbeat gives that IP to another VM, then the MAC addr. for this IP will change but the ARP table of the switch will not follow until the switch refreshes its ARP table. This would result in downtime, which is exactly what heartbeat is supposed to avoid.

In this example below I use routing method for eth0 and bridging for eth1, consequently configure eth0 as a usual interface and create a bridge for eth1.
Xen XL toolstack will automatically create the proper vif* interfaces and routing entries for each VM while starting the VM.
To make sure xend doesn’t try to configure the bridges, force xend to never try by reconfiguring the networking:
Edit /etc/xen/xend-config.sxp
(network-script dummy)
(vif-script vif-route_eth0-bridge_eth1)

Edit the file: /etc/network/interfaces
Content: (make sure you replace the following example IPs etc. accordingly)
Here we use a very normal Network configuration without bridges.
# The loopback network interface
auto lo
iface lo inet loopback
#
# eth0
auto eth0
iface eth0 inet static
address 12.34.56.78
netmask 255.255.255.0
network 12.34.56.0
broadcast 12.34.56.255
gateway 12.34.56.254
#
# eth1 and xenbr1 Bridge
auto xenbr1
iface xenbr1 inet static
bridge_ports eth1
address 192.168.0.19
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
bridge_stp off
post-up ethtool -K xenbr1 tx off
post-up ip link set xenbr1 promisc off

Make sure xen Linux is listed as the default kernel when booting
mv /etc/grub.d/20_linux_xen /etc/grub.d/09_linux_xen
or
dpkg-divert --divert /etc/grub.d/09_linux_xen --rename /etc/grub.d/20_linux_xen
update-grub
Switch to the XL Xen ToolStack
Edit /etc/default/xen
TOOLSTACK=xl
Edit /etc/xen/xl.conf and make sure the entries are as follows:
# automatically balloon down dom0 when xen doesn't have enough free memory to create a domain
autoballoon=1
# full path of the lockfile used by xl during domain creation
lockfile="/var/lock/xl"
# default vif script
vifscript="/etc/xen/scripts/vif-route_eth0-bridge_eth1"

Note: Here we use a script which will use routing for eth0 and bridging for eth1. Here we will create it.
touch /etc/xen/scripts/vif-route_eth0-bridge_eth1
chmod 755 /etc/xen/scripts/vif-route_eth0-bridge_eth1
Edit the file /etc/xen/scripts/vif-route_eth0-bridge_eth1.
Content:
#!/bin/sh
# Custom vif script which allows to combine routing for Internet and bridging for internal LAN
dir=$(dirname "$0")
IFNUM=$(echo ${vif} | cut -d. -f2)
if [ "$IFNUM" = "0" ] ; then
"$dir/vif-route" "$@"
else
"$dir/vif-bridge" "$@"
fi

PyGRUB
If your DOMUs configurations are set to use pygrub as boot loader, then make sure the path to pygrub in the DOMU configuration file is correct as follows:
bootloader = '/usr/lib/xen-4.1/bin/pygrub'
In the same DOMU configuration file, make sure you are using the appropriate MAC addresses with the network interfaces assignment for example:
vif = [ 'ip=12.34.56.18,mac=00:16:3E:D7:9C:F4' , 'ip=192.168.0.18,mac=00:16:3E:D7:9C:F6',bridge=xenbr1]
Setup the IP forwarding and ARP proxying in kernel:
Edit the file /etc/sysctl.conf
Either un-comment or add the following lines:
# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1
# ARP Proxying
net.ipv4.conf.eth0.proxy_arp = 1
To make this change take effect immediately run:
sysctl -p /etc/sysctl.conf

Finally, before we reboot the system we need to make sure we activate the proper toolstack and related features at boot time via:
update-rc.d xendomains defaults
update-rc.d xen defaults
/etc/init.d/xen restart
/etc/init.d/xendomains restart

Reboot
reboot
Start your DOMUs as usual with the command xl instead of xm.