msgbartop
MAC OS X, Linux, Windows and other IT Tips and Tricks
msgbarbottom

30 Oct 16 Monitoring Linux server with iPhone/iPad

Introduction:

Although Apple doesn’t have too many apps that support Linux admins, here is one that just came back on the market with a rebound on 26 Oct. 2016 with a new look, features and bug fixes: The iStat3 Server for Linux and iStat3 for iOS made by Bjango PTY Ltd. This app will display live the following characteristics of a Linux server.
– Uptime
– CPU usage
– System Load
– Disk space and disk activity
– Network traffic load
– Processes list(top)
– Sensors: Memory and CPU temperature

Read more about it on https://bjango.com/ios/istat/

In order for the iOS app to get this information from the Linux servers, it needs a connection to its colleague the iStat3 server, which is an agent running in each targeted Linux server. The agent is a daemon which runs in the background and listens on a standard port 5109(configurable). Since there are so many different Linux distributions the agent needs to be compiled in each targeted Linux server. In order to facilitate this process I wrote this article.

Note: I only mention the steps for Debian 6/7/8 and Ubuntu 12.x/14.x/16.x

Steps:

Installing the needed packages:
apt-get update && apt-get install build-essential g++ autoconf libxml2-dev libssl-dev libsqlite3-dev fancontrol libsensors4:amd64 libsensors4-dev lm-sensors
Download the software:
wget https://download.bjango.com/istatserverlinux -O istatserver-linux_3.0.tar.gz
or if changed address or not available
wget http://public.itmatrix.eu/istatserver-linux_3.0.tar.gz
Compiling and installing the software:
tar fvxz istatserver-linux_3.0.tar.gz
cd istatserver-3.01
./configure && make && make install

Configuring the istatserver:
Here you mostly need to modify the 5 digit server_code.
vim /usr/local/etc/istatserver/istatserver.conf

Extra preparations for Debian 6/7 or Ubuntu 12.x/14.x which are using the SysV init

Getting the start script from my repos:
wget http://public.itmatrix.eu/istatserver -O /etc/init.d/istatserver
chmod 755 /etc/init.d/istatserver
update-rc.d istatserver defaults
service istatserver start ; sleep 1 ; ps aux | grep -v grep | grep istat

Result should be:
istat 17891 0.0 0.2 42108 2332 ? R 18:39 0:00 /usr/local/bin/istatserver -d

Extra preparations for Debian 8 or Ubuntu 16.x which are using the Systemd init

vim /etc/systemd/system/istatserver.service
istatserver.service file content:
[Unit]
Description=istatserver server daemon
Documentation=man:istatserver(8)
After=network.target
#
[Service]
Type=simple
EnvironmentFile=/etc/default/istatserver
ExecStart=/usr/local/bin/istatserver $OPTIONS
ExecReload=/bin/kill -HUP $MAINPID
KillMode=control-group
Restart=on-failure
RestartSec=30s
#
[Install]
WantedBy=multi-user.target

touch /etc/default/istatserver
systemctl daemon-reload
systemctl enable istatserver.service
service istatserver start ; sleep 1 ; ps aux | grep -v grep | grep istat

Result should be:
istat 1507 43.0 0.0 118844 7120 ? Ssl 19:02 0:00 /usr/local/bin/istatserver

General Note:

Makes sure your firewall is allowing in the port 5109(or whatever the port you are using).
I’m using ufw, so for example the command would be:
ufw allow from any to any port 5109
Result:
Rule added
Rule added (v6)

UPGRADING from ISTATD to ISTATSERVER:

In case you had already the older version of this agent(istatd) running here are the steps to stop using it:
ps aux | grep istat
killall istatd ; sleep 2 ; killall istatd
update-rc.d -f istatd remove

Getting the iPad/iPhone APP:

Concerning the iOS app, you need to buy it on Apple store and its name is: iStat 3 from Bjango PTY Ltd.
This app allows to monitor multiple Linux servers with very pretty graphs.
If you have a Mac you can also buy the similar APP called iStat from Apple Store. It displays the exact same thing as with iPad and adds a few small extra features.
screen-shot-2016-10-31-at-19-41-18

15 Apr 16 Installing Webmin in Debian 8(Jessie)

These instructions are a ‘Plagiat’ of the site:
http://www.christophe-casalegno.com/2015/07/14/how-to-install-webmin-on-debian-8/

To install webmin on Debian 8 just follow this instructions :
cd /root
wget http://www.webmin.com/jcameron-key.asc
apt-key add jcameron-key.asc
echo "deb http://download.webmin.com/download/repository sarge contrib" >> /etc/apt/sources.list
echo "deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib" >> /etc/apt/sources.list
apt-get update
apt-get -y install webmin

If it’s too long for you, you can also just do this :
wget http://www.christophe-casalegno.com/tools/install_webmin.sh
chmod +x install_webmin.sh
./install_webmin.sh

13 Feb 16 Verifying the validity of an NFS mount

Introduction:
Every now and then if an NFS mount is no more connected to the server or something goes wrong with the NFS connection, running the command ‘ls mountpoint’ hangs the terminal till I press CTRL-C. So I tried to figure out a script that will be run as cron job and will tell me when an NFS mount has gone wrong. I had to revert to unorthodox tricks since doing a simple command ‘stat mountpoint &’ within the script would also hang the script. So I use the command ‘at now’ which runs the command independently for the script that initiated it. Here is an example of such script.

#!/bin/bash
# Name: MOUNT_CHECK.sh
# Purpose: Checks the health of the NFS mountpoint given by argument
# it kills the at/stat process and exits with an exit code 2 if the timeout has expired.
#-------------------------------------------------------------------
startdelay=3
timeout=10
# processes to be excluded in the 'ps | grep' test
excludes="openvpn|istatd|rpc.statd"
if [ $# -ne 1 ]; then
echo "ERROR: Needs mountpoint as argument"
echo "Usage: MOUNT_CHECK.sh MountPoint"
exit 2
fi
#
echo "/usr/bin/stat $1" | /usr/bin/at now
sleep $startdelay
while (ps ax | egrep -v "grep|$excludes" | grep -q stat); do
let count=${count}+1
sleep 1
if [ $count -ge $timeout ]; then
kill $(pidof stat)
#echo "Mountpoint $1 : FAILED to connect before timeout of $timeout sec."
exit 2
fi
done

18 Jan 16 Reporting SMART status of RAID disks

Reference site: http://www.cyberciti.biz/faq/linux-checking-sas-sata-disks-behind-adaptec-raid-controllers/

Note: Although Hardware RAID controllers made by other hardware manufacturers here I use Adaptec as an example:

Install the software:
apt- get install smartmontools
Curious which company the RAID controller is from?
Find out which RAID controller you have:
lspci | grep 'RAID'
Result: 01:00.0 RAID bus controller: Adaptec Device 028b (rev 01)
# Check out if the controller is supported and which devices it sees:
smartctl --scan
Example output:
/dev/sda -d scsi [SCSI]
/dev/sdb -d scsi [SCSI]

Check the SMART overall-health test of the drives :
smartctl -d scsi -H /dev/sda | grep 'SMART'
smartctl -d scsi -H /dev/sdb | grep 'SMART'

Result example:
/dev/sda: SMART Health Status: OK
/dev/sdb: SMART Health Status: OK

Checking Individual drives behind the RAID controller
The individual drives behind the controller are usually named sequentially according to the order of the simulated drives:
eg.
/dev/sda (2 drives behind controller): /dev/sg1 /dev/sg2
/dev/sdb (2 drives behind controller): /dev/sg3 /dev/sg4

Commands for doing those checks:
smartctl -d scsi --all -T permissive /dev/sg1
smartctl -d scsi --all -T permissive /dev/sg2
smartctl -d scsi --all -T permissive /dev/sg3
smartctl -d scsi --all -T permissive /dev/sg4

Create a script that will be run by cron regularly and send the results by email:
Script:
#!/bin/bash
# Name: SMART-report.sh
# Purpose: Sends report of SMART status of RAID hard disks
# Syntax: SMART-report.sh
#--------------------------------------------------------
(. ~/.bashrc
echo -n "/dev/sda: "
smartctl -d scsi -H /dev/sda | grep 'SMART'
echo -n "/dev/sdb: "
smartctl -d scsi -H /dev/sdb | grep SMART
echo "Individual drives behind the RAID controller";echo
echo "============== /dev/sda ===> /dev/sg1 ============="
smartctl -d scsi --all -T permissive /dev/sg1 | grep 'SMART';echo
echo "============== /dev/sda ===> /dev/sg2 ============="
smartctl -d scsi --all -T permissive /dev/sg2 | grep 'SMART';echo
echo "============== /dev/sdb ===> /dev/sg3 ============="
smartctl -d scsi --all -T permissive /dev/sg3 | grep 'SMART';echo
echo "============== /dev/sdb ===> /dev/sg4 ============="
smartctl -d scsi --all -T permissive /dev/sg4 | grep 'SMART'
) | mail -s "SMART Result of $(hostname -f)" user@my-email.com

14 Jan 16 Preventing a bash script from running concurrently

Introduction: In order to prevent a bash script instance from running more than once concurrently, here is a small tip on how to write the script.

Script template:
#!/bin/bash
# Prevents that an instance of the script starts while another instance of it is still running
scriptname=$(basename $0)
lockfile="/tmp/${scriptname}.lock"
if [ -e $lockfile ]; then exit 1 ; fi
touch $lockfile.lock
# Delete lock file if CTRL-C typed from the keyboard
trap 'rm $lockfile ; exit' SIGINT SIGQUIT
#----------------------------------
# ############ Put your script code here #####################
#----------------------------------
# delete the lock file
rm $lockfile
# .eof

16 Jun 15 Install TeamViewer in Debian Wheezy

Teamviewer is a very good and stable remote desktop with many clients software form almost any platform. Here I explain how I got TeamViewer to run on a headless Debian Wheezy server.
Reference: https://www.teamviewer.com/en/help/363-Wie-installiere-ich-TeamViewer-auf-meiner-Linux-Distribution.aspx#multiarch

Steps:
– Install the VNC desktop on the Debian Server for a particular user as per the instructions shown here:
//tipstricks.itmatrix.eu/installing-linux-remote-terminal-using-vnc-on-a-debian-server/
Note: this above VNC server will not be directly conecte to via VNC as remote desktop here, but as an Xserver based virtual desktop screen for TeamViewer server to mirror it to TeamViewer clients. At start we will need the VNC connection to the first time start of TeamViewer. Afterwards no more. Therefore the port 5901 should be afterwards blocked by a Firewall.

– Install TeamVierwer i386(32 Bit) in Wheezy. Because of problems with dependencies the 32 bit version of TeamViewer needs to be installed as follows:

– Install the i386 MultiArch environment and dependencies packages:
dpkg --add-architecture i386
apt-get update
apt-get install links:i386
apt-get install libasound2-plugins:i386 glibc-doc:i386 locales:i386 ia32-libs lib32z1 lib32asound2 libc6-i386 ia32-libs-i386

– Install TeamViewer
wget http://download.teamviewer.com/download/teamviewer_i386.deb
dpkg -i teamviewer_i386.deb
teamviewer daemon enable

…. not finished!! To be continued 🙂

04 May 15 Using CURL for sending crafted HTTP POST authenticated queries

CHALLENGE:
I came across a situation where I needed to send an HTTP request using the POST method with some POST data but after I have authenticated with name and password.

SOLUTION:(using curl tool)
The trick here is to preserve the SESSIONID of the authenticated response for the second POST request.

EXAMPLE:
I needed to go into my account in domain-hoster.net and request the CSV file which lists all my registered domains.

COMMANDS:
curl -v --user-agent "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0" -c cookies.txt -d "username=myuser&password={html_encoded_password}" http://login.domain-hoster.net/index/login
curl -v --user-agent "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:37.0) Gecko/20100101 Firefox/37.0" -b cookies.txt -d "orderField=&orderDir=&name=&state=&owner=&sedo=&lock=&date_expire=&renewal=&itemsPerPage=&csv=CSV" http://login.domain-hoster.net/domain

In the above example I simulate a Firefox Browser (–user-agent), save the cookies (includes the SESSIONID) in the file cookies.txt and use it in my second POST request to get the content of the requested CSV file into the terminal.

IMPORTANT NOTE: The password must be in proper HTML-encoded format to be accepted. This is applicable for any chars. that is not a-z or A-Z. There are many ways to convert the password in HTML-Encoded format. The most reliable way I found, is to manually login with a proper browser with name and password and look at the request headers using a browser plugin that lets you see the headers contents. The password will then be shown properly in the header.
Examples of password characters and their HTML-Encoded equivalents:
& = %26
! = %21, etc.
So a password like: Tw&Ui8vH!
would look like this: Tw%26Ui8vH%21

31 Mar 15 Monitoring latency time of http requests

Here is a simple but useful command which shows the latency time of http requests. You can adjust the delay between repeats as well as the URL being queried.
Reference: http://www.shellhacks.com/en/Check-a-Website-Response-Time-from-the-Linux-Command-Line

host="www.google.de"; delay=5; while true ; do echo -n "Response time for http://$host:" ;curl -s -w %{time_total}\\n -o /dev/null http://$host ;sleep $delay; done

Results:
Response time for http://www.google.de:0,025
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,026
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,024
Response time for http://www.google.de:0,024
.......

ADVANCED:
Here is a more advance version which performs more timing tests:

host="www.google.de"; delay=5; while true ; do echo "------"; curl -s -w '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' -o /dev/null http://$host; sleep $delay; done

Results:
Lookup time: 0,002
Connect time: 0,011
PreXfer time: 0,011
StartXfer time: 0,022
.
Total time: 0,023
------
.
Lookup time: 0,001
Connect time: 0,012
PreXfer time: 0,013
StartXfer time: 0,023
.
Total time: 0,023
------
.......

Meanings:
Lookup time: The time, in seconds, it took from the start until the name resolving was completed.
Connect time: The time, in seconds, it took from the start until the TCP connect to the remote host was completed.
PreXfer time: The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all ‘pre-transfer’ commands and negotiations that are specific to the particular protocol(s) involved.
StartXfer time: The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes ‘time_pretransfer’ and also the time the server needed to calculate the result.

13 Sep 14 Installing Shinken in Debian Wheezy

Debian Wheezy does offer the installation of the full (a bit old)shinken, BUT it doesn’t offer the Installation of the WebUI.
Here is a better way to install everything including pnp4nagios and check_mk in one go:

STEPS:
Install Shinken
wget http://www.shinken-monitoring.org/install -O /tmp/install_shinken.sh
cd /tmp && sh install_shinken.sh

Configure shinken
vim /usr/local/shinken/etc/shinken-specific.cfg
Change the http://YOURSERVERNAME/ in:
define module {
module_name GRAPHITE_UI
module_type graphite_webui
uri http://YOURSERVERNAME/
templates_path /usr/local/shinken/share/templates/graphite/
}

Because of a bug in the stop arbiter script we need to make some tiny changes:
Edit /usr/local/shinken/bin/stop_arbiter.sh
change the line
kill $(cat "$DIR"/../var/arbiterd.pid)
to
kill $(cat "$DIR"/../var/arbiter.pid)

Unfortunately the process skonf doesn’t want to stop when I run the script: /usr/local/shinken/bin/stop_all.sh
So we need to doctor the script:
vim /usr/local/shinken/bin/stop_skonf.sh
Change the following:
FROM:
kill $(cat "$DIR"/../var/skonfd.pid)
TO:
kill -9 $(ps aux | grep skonf.cfg | grep -v 'grep' | awk '{print $2}')

Start the whole shinken
cd /usr/local/shinken/bin
./launch_all.sh

WebUI


Change the password of the admin user:
cd /usr/local/shinken/etc/
htpasswd htpasswd.users admin


Now connect to the WebUI

In browser:
http://yourserver.name:7727/
To Stop all the services
/usr/local/shinken/bin/stop_all.sh
To de-install shinken
cd /usr/local/shinken/
./install -u

22 Aug 14 Installing Icinga2, pnp4nagios, NRPE Agent and Graphite in Debian Wheezy

Since Icinga2 is not part of the stable version of Debian Wheezy we need to install it from the wheezy-backports. Therefore the following steps are suggested:

Add the backports to the Debian repositories list:
mcedit /etc/apt/sources.list
Add the following lines:
deb http://ftp.debian.org/debian/ wheezy-backports main contrib non-free
deb-src http://ftp.debian.org/debian/ wheezy-backports main contrib non-free

Update the repo db.
apt-get update
Install icinga2 and Graphite from backports
apt-get install -t wheezy-backports icinga2 icinga2-classicui libapache2-mod-wsgi
During the installation the password of Icinga Admin user icingaadmin will be asked to be created.
Give this password 2 times.

PNP4NAGIOS


(Ref: Based on: https://wiki.icinga.org/display/howtos/Setting+up+PNP+with+Icinga2+on+Debian)
apt-get update && apt-get install --no-install-recommends pnp4nagios rrdcached
update-rc.d rrdcached defaults

We will install PNP with “Bulk mode with NPCD”
icinga2-enable-feature perfdata
Edit the file /etc/pnp4nagios/apache.conf and change the line:
AuthUserFile /etc/nagios3/htpasswd.users
TO:
AuthuserFile /etc/icinga2/classicui/htpasswd.users
Edit the file /etc/pnp4nagios/npcd.cfg and make the changes as follows:
#perfdata_spool_dir = /var/spool/pnp4nagios/npcd/
perfdata_spool_dir = /var/spool/icinga2/perfdata

Enable the npcd daemon
Edit /etc/default/npcd and change the Run line to:
Run="yes"
update-rc.d npcd defaults
mkdir -p /var/cache/rrdcached
chown nagios: /var/cache/rrdcached

Edit /etc/default/rrdcached and add the following line:
OPTS="-w 1800 -z 1800 -j /var/cache/rrdcached -s nagios -m 0660 -l unix:/var/run/rrdcached.sock"
Put the Apache user www-data into the nagios group
usermod -G nagios www-data
Edit the file /etc/pnp4nagios/config.php and modify these 2 variables:
#$conf['nagios_base'] = "/cgi-bin/nagios3";
$conf['nagios_base'] = "/cgi-bin/icinga";
...
#$conf['RRD_DAEMON_OPTS'] = '';
$conf['RRD_DAEMON_OPTS'] = 'unix:/var/run/rrdcached.sock';

Edit the file /etc/pnp4nagios/process_perfdata.cfg and change the RRD_DAEMON_OPTS to:
RRD_DAEMON_OPTS = unix:/var/run/rrdcached.sock

Edit the file /etc/icinga2/conf.d/templates.conf and
– Add the following templates at the very beginning of the file:
template Host "pnp-hst" {
action_url = "/pnp4nagios/graph?host=$HOSTNAME$' class='tips' rel='/pnp4nagios/popup?host=$HOSTNAME$&srv=_HOST_"
}
template Service "pnp-svc" {
action_url = "/pnp4nagios/graph?host=$HOSTNAME$&srv=$SERVICEDESC$' class='tips' rel='/pnp4nagios/popup?host=$HOSTNAME$&srv=$SERVICEDESC$"
}

– And add the import lines:
In the Host template:
template Host "generic-host" {
max_check_attempts = 5
check_interval = 1m
retry_interval = 30s
check_command = "hostalive"
import "pnp-hst"
}

In the Services template
template Service "generic-service" {
max_check_attempts = 3
check_interval = 1m
retry_interval = 30s
import "pnp-svc"
}

Restart All the Daemons
service rrdcached restart
service npcd restart
service icinga2 restart

GRAPHITE


Install Graphite-Carbon
apt-get install -t wheezy-backports graphite-carbon
Enable the Graphite Module
icinga2-enable-feature graphite
Enable the auto-start of Icinga2
update-rc.d icinga2 defaults
Enable the auto-start of Graphite-Carbon
Edit /etc/default/graphite-carbon
Change the line: CARBON_CACHE_ENABLED=false
TO CARBON_CACHE_ENABLED=true
Then run: update-rc.d carbon-cache defaults

Graphite WEB Interface


Although Graphite-carbon is part of Wheezy, Graphite-Web is not.
So we need to install it using pip
(Instructions based on: https://gist.github.com/tristanbes/4046457)
Install Graphite-Web
apt-get install -y python2.6 python-pip python-cairo python-django python-django-tagging
apt-get install -y libapache2-mod-wsgi python-twisted python-memcache python-pysqlite2 python-simplejson build-essential python-dev
pip install graphite-web

Add the Apache config symlink:
cp /opt/graphite/examples/example-graphite-vhost.conf /opt/graphite/examples/graphite.conf
ln -s /opt/graphite/examples/graphite.conf /etc/apache2/sites-enabled/

Adapt the following Apache configuration to your environment
/opt/graphite/examples/graphite.conf
For Debian you also need to change the following Entry:
FROM: WSGISocketPrefix run/wsgi
TO: WSGISocketPrefix /var/run/apache2/wsgi
Configure Graphite
cd /opt/graphite/conf/
cp dashboard.conf.example dashboard.conf
cp graphTemplates.conf.example graphTemplates.conf
cp graphite.wsgi.example graphite.wsgi

Symlink the Debian Graphite-Carbon data directory to this Graphite-Web installation:
rmdir /opt/graphite/storage/whisper
ln -s /var/lib/graphite/whisper /opt/graphite/storage/whisper

Configure the local settings of Graphite:
cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
Edit the file /opt/graphite/webapp/graphite/local_settings.py
and change the following settings:
#SECRET_KEY = 'UNSAFE_DEFAULT'
TO: SECRET_KEY = 'YOUR_OWN_KEY_WITHOUT_SPACES'
Change the following lines FROM:
#LOG_RENDERING_PERFORMANCE = True
#LOG_CACHE_PERFORMANCE = True
#LOG_METRIC_ACCESS = True

TO:
LOG_RENDERING_PERFORMANCE = False
LOG_CACHE_PERFORMANCE = False
LOG_METRIC_ACCESS = False

Note:Activate and adapt other values as you see appropriate. Like:
TIME_ZONE = 'Europe/Berlin'

Finish the Graphite-Web installation process
cd /opt/graphite/webapp/graphite
python manage.py syncdb

You’ll be presented with the following question:
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no):

Answer no.
Now you might get some error message but for now they should be ignored.
The administrator account will be created later.

Make the data area writable by Apache user (www-data)
chown -R www-data:www-data /opt/graphite/storage/

Configure Graphite-Carbon to receive Icinga data properly
Edit the file: /etc/carbon/storage-schemas.conf
Add the following lines:
[icinga_internals]
pattern = ^icinga\..*\.(max_check_attempts|reachable|current_attempt|execution_time|latency|state|state_type)
retentions = 5m:7d
#
[icinga_default]
# intervals like PNP4Nagios uses them per default
pattern = ^icinga\.
retentions = 1m:2d,5m:10d,30m:90d,360m:4y

Make a fix of graphite carbon:
Otherwise the start of carbon gives the following error:
‘ImportError: cannot import name daemonize’
pip install daemonize
vim /usr/lib/python2.7/dist-packages/carbon/util.py

And change the line:
from twisted.scripts._twistd_unix import daemonize
to
import daemonize

Start Graphite-Carbon daemon
service carbon-cache start
Reload Apache2 configurations
service apache2 reload

Access the Icinga admin web interface via:
http://my.monitoring.com/icinga2-classicui/
Login:
Name: icingaadmin
Password: (admin password you created during installation)

Access the Graphite graphs via:
http://graphite.my.monitoring.com/
graphite.yourserver.com Is the Name of ‘ServerName’ config in /opt/graphite/examples/graphite.conf

Configure a SuperAdmin for enabling saving Graphite graphs.
cd /opt/graphite/webapp/graphite
export LC_ALL=en_GB.UTF-8
export LANG=en_GB.UTF-8
python manage.py createsuperuser

And give the new administrator credentials.

Saving graphs in graphite:
In order to be able to save graphs in Graphite-Web you need to:
– login as administrator on the graphite-web interface with the URL:
http://your.graphite.com/admin/
– Create new user(s) (give access rights in thier profile)
– Login in the graphite-web as one of the users created.
– A new Disk button will appear on top left of the Graph composer window.

Using the Nagios/Icinga agent NRPE


Note: Icinga is happy to use the regular method of calling nagios scripts, in the remote monitored server, by using the SSH command. BUT in case you blocked this port, in whatever manner, I suggest using the NRPE agent which listens on port 5666 and is dedicated for monitoring only.
INSTALLATION:
This installation is to be repeated in each server you wish to monitor.
apt-get install nagios-nrpe-server nagios-plugins sudo
The following package is recommended to make tests locally of the configured NRPE daemon:
apt-get install nagios-nrpe-plugin
To allow some commands that require being root, you allow the user nagios running the scripts contained in in the nagios /usr/lib/nagios/plugins/
WARNING: In some systems this might pose a security problem. So be careful when using this method.
nagios ALL=(ALL) NOPASSWD: /usr/lib/nagios/plugins/
Edit the NRPE configuration file /etc/nagios/nrpe_local.cfg and add the following entry:
command_prefix=/usr/bin/sudo
Note: If you have a firewall make sure the port 5666 is open, otherwise you can change the port in this same file as desired.
server_port=5666
If your monitored server has more than one network interface, it might be wise to limit the connection to only one interface through which the connection from Icinga2 server is made,. Otherwise do not use this entry and nrpe daemon will bind to all interfaces.
Example:
server_address=12.23.34.56
This is the address of the interface NRPE should bind to.
Tip: For testing locally the nrpe commands you can:
– set the above server_address=127.0.0.1
– restart the nrpe daemon(service nagios-nrpe-server restart)
– and use the following command format to test any nrpe command:
/usr/lib/nagios/plugins/check_nrpe -H 127.0.0.1 ........
And the most important setting is to limit the NRPE connections to the Icinga server by configuring this entry.
If you have 2 or more servers that should be allowed to talk to NRPE, delimit them with a comma.
Example:
allowed_hosts=34.45.23.67,34.45.23.68,127.0.0.1

Calling NRPE


It is recommended to always add the localhost(127.0.0.1) in order to be able to test the NRPE commands locally for debug purposes.
Each command you want to call from Icinga2 has to be entered here. The command arguments can be fixed like the following:
command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /dev/hda1
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 150 -c 200

If you want to control the WARNING, CRITICAL levels and other arguments from Icinga when calling the nrpe command, then use the following method:
In this case remember to add the following entry in the configuration file:
dont_blame_nrpe=1
Examples:
command[check_users]=/usr/lib/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
command[check_load]=/usr/lib/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
command[check_disk]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
command[check_procs]=/usr/lib/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$

Note: Take a look at the commands provided by the installed packages in the directory /usr/lib/nagios/plugins/ and enter the ones you will need for monitoring this server.
So resume, a sample of my NRPE server configuration(/etc/nagios/nrpe_local.cfg):
######################################
# Do any local nrpe configuration here
######################################
server_port=5666
server_address=192.168.100.5
allowed_hosts=192.168.100.3,127.0.0.1
command_prefix=/usr/bin/sudo
dont_blame_nrpe=1
command[check_users]=/usr/lib/nagios/plugins/check_users -w $ARG1$ -c $ARG2$
command[check_load]=/usr/lib/nagios/plugins/check_load -w $ARG1$ -c $ARG2$
command[check_disk]=/usr/lib/nagios/plugins/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
command[check_procs]=/usr/lib/nagios/plugins/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$
........ (a long list of commands are listed here)

In the above configuration I’m using the internal LAN to connect from the Icinga2 server (192.168.100.5) to all of my servers to be monitored. To make temporary tests locally, the server_address needs to be changed to 127.0.0.1, restart the nrpe daemon(service nagios-nrpe-server restart) and use the /usr/lib/nagios/plugins/check_nrpe command with arguments as above to test the local nrpe daemon. When finished, revert to the proper binding address of server_address.

When finished restart NRPE:
service nagios-nrpe-server restart

Compiling NRPE from sources

If you find the distribution package of NRPE too old and want to compile your own do the following:
– Install the Debian nagios-nrpe-server and nagios-nrpe-plugin as shown above
– Compile and overwrite the Debian nrpe binaries with the following commands.
mkdir -p /usr/src/NRPE
cd /usr/src/NRPE
wget http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.15/nrpe-2.15.tar.gz
tar fvxz nrpe-2.15.tar.gz
cd nrpe-2.15
./configure --with-ssl-lib=$(dpkg -L libssl1.0.0 | grep libcrypto | cut -d/ -f1,2,3,4)
make

Installing the daemon and the check plugin:
cp src/nrpe /usr/sbin/
cp src/check_nrpe /usr/lib/nagios/plugins/

Testing the local NRPE daemon locally with an example command:
/usr/lib/nagios/plugins/check_nrpe -H 127.0.0.1 -c check_disk -a 15 17 /
Example of proper Result:
DISK OK - free space: / 17576 MB (91% inode=94%);| /=1557MB;20143;20141;0;20158

Icinga server configuration for issuing remote NRPE commands


To be continued ……

Note: In order to make sure you are installing only Icinga2 etc. from backports and leave all other packages to stable version in future installations, after installation of Icinga2 just comment out the above added backport lines in /etc/apt/sources.list and update the repo. db again (apt-get update).

Happy Monitoring!!