2014-02-28 18:40:43

Qemu systemd service

Stopping a Qemu vm on shutdown is a tricky thing with systemd. The solution mentioned in the arch wiki isnt working for me. It always kills the vm instantly.

After some trying I found a solution that works for me. Socat is required for this and the vm should listen to acpi events like power button pressed.

The systemd service file (/etc/systemd/system/qemu@.service) is shown below:

[Unit]
Description=QEMU virtual machine (%i)
After=network.target netctl@br0.service

[Service]
Type=forking
PIDFile=/run/qemu_%i.pid

EnvironmentFile=/etc/conf.d/qemu.d/%i

ExecStart=/usr/bin/qemu-system-x86_64 -name %i -daemonize -pidfile /run/qemu_%i.pid -monitor unix:/tmp/%i.sock,server,nowait $args

ExecStop=/bin/sh -c '/usr/bin/echo system_powerdown | /usr/bin/socat - UNIX-CONNECT:/tmp/%i.sock; while ps ax | grep "/usr/bin/qemu-system-x86_64 -name %i" | grep -vq grep; do sleep 1; done'
TimeoutStopSec=30

[Install]
WantedBy=multi-user.target

As you see, the configuration for each vm is located in /etc/conf.d/qemu/$name. A sample config could look like this:

args=-cpu kvm64 -display none -m 7G -smp 2 -enable-kvm -net nic,macaddr=01:23:45:67:89:AB -net bridge,br=br0 /path/to/vm.img

A comment to the kvm64 processor type: I used to use the host cpu type but with the Intel(R) Core(TM) i7-4770 CPU of the server the vms were throwing NMI errors and one oopsed because of this some days ago.


Posted by toerb | Permanent link | File under: arch

2014-02-23 21:06:16

Gmane mailing list statistics with graph

Gmane is a large mailing list archive. There you can get statistics for most common lists. You get the data in the form of "date #mails #spam". Example given: http://gmane.org/output-rate.php?group=gmane.psychology.depression:

date posting-rate spam-rate
20081113 1 0

If you want the data on a monthly base you can use this script. It sums all mails for a month and print it out. Spam is ignored but you can easily add it. Same example as above:

$ mail_statistics.sh gmane.psychology.depression
200811 1

I use this data to generate plots with gnuplot from it. To get the x-axis right you need to set the timefmt to "%Y%m" for understanding values like "201402". The complete gnuplot config I use for this purpose:

set autoscale
set grid
set xdata time
set timefmt "%Y%m"
set xtics format "%Y"
set xrange ["200201":"201404"] noreverse
set yrange [0:4100]
set output "images/postfix.png"
set terminal png size 900,600
set xlabel "Jahr"
set ylabel "Mails"
set title "postfix-users@postfix.org"
plot "gnuplot/postfix.dat" using 1:2 title 'mails' with boxes

As you can see in the gnuplot file I use my script to generate statistics for "postfix-users@postfix.org", the postfix users mailing list. To generate monthly statistics with my script you need to provide the gmane group name. It always starts with "gmane" and consists out of tags divided by dots. Examples: gmane.linux.arch.general, gmane.psychology.depression, gmane.mail.postfix.user, gmane.mail.exim.user.

Source of mail_statistics.sh:

#!/usr/bin/env bash

gmane=$1
TEMP=$(mktemp)
wget -q "http://gmane.org/output-rate.php?group=${gmane}" -O "${TEMP}"
grep -q "date posting-rate spam-rate" "${TEMP}" || { echo wrong parameter; rm ${TEMP}; exit; }
counter_messages=0
#counter_spam=0
previous=""
(grep -v date "${TEMP}"; echo end 0 0) | while read day; 
do
        read date messages spam <<< "${day}"
        timestamp=${date:0:6}
        if [ "${timestamp}" != "${previous}" ] && [ -n "${previous}" ];
        then
                echo "${previous:0:4}${previous: -2} ${counter_messages}"
                counter_messages=0
#               counter_spam=0
        fi
        counter_messages=$((${counter_messages}+${messages}))
#       counter_spam=$((${counter_spam}+${spam}))
        previous=${timestamp}
done
rm "${TEMP}"

To generate postfix-users mailing list statistics the commando

$ mail_statistics.sh gmane.mail.postfix.user > gnuplot/postfix.dat; gnuplot gnuplot/postfix.plt

gives you the following postfix.png in images/


Posted by toerb | Permanent link | File under: bash

2014-02-20 17:38:35

Arch website down after update (--force?)

Florian Pritz (sys admin for aldeeran) posted as the arch website/wiki/forums went down for two hours on monday the 17th february:

Simple update gone bad.

--force? ;)


Posted by toerb | Permanent link | File under: arch

2014-02-10 23:09:34

Huge Packages (HP)

*facepalm*


Posted by toerb | Permanent link

2014-02-03 17:59:08

Hetzner backup server readonly

The problems with Hetzners backup server got even more curious. Waiting for an answer from the support I started a manual backup for testing. To my surprise duplicity threw other errors than the common timeouts:

sftp put of /tmp/duplicity-dlrNxy-tempdir/mktemp-ll8AE0-3 (as duplicity-inc.20140201T020105Z.to.20140203T163417Z.vol1.difftar.gpg) failed: Failure (Try 2 of 5) Will retry in 10 seconds.

Wondering what that could be I tested the response time and got values of about 0.2 seconds. Wow this sounds like its not the timeout problem.

So, now I connected on the server per sftp. Browsing my files works but the write test fails:

sftp> mkdir test
Couldn't create directory: Failure

Free space isn‘t the problem. So the only thing I can do is writing a ticket again.. BTW Hetzners status website doesn‘t list this faulty service yet.


Posted by toerb | Permanent link | File under: hetzner, backup

2014-02-02 20:08:19

High latency of Hetzner backup servers

My server is located in a Hetzner computer center. The backups are done with duplicity and are running every night. The data is put on backupservers from Hetzner.

Sometimes a backup fails with the following (shortened) error:

BackendException: ssh connection to $backup_server failed: Error reading SSH protocol banner

Duplicity uses paramiko as a ssh backend. The timeout for the ssh banner is set in the file transport.py on line 363:

        self.banner_timeout = 15        # how long (seconds) to wait for the SSH banner

Setting the value to something higher (e.g. 120) can help. For an estimation try this:

time echo quit | sftp $backup_server

On this sunday evening I get an answer time from about 2 minutes. I adjusted my transport.py to this value and keep an eye on the logs the next days.


Posted by toerb | Permanent link | File under: hetzner, backup

2014-01-31 11:27:56

Getting Nginx in a chroot jail

Running $software in a Jail is always a nice thing. Looking for a good howto I found this article in the Arch Linux wiki.

If you want the steps all in one shell script, try this one from adityamukhos gist page.


Posted by toerb | Permanent link | File under: arch

2014-01-30 19:05:05

TP-Link WDR4300 VLan bug

For my new home network I wanted a VLan for guest WLan and untagged traffic for the regular users. As I tested the configuration some weird problems occured: Packets where being sent untagged from the WDR4300 to my router. The router sent untagged packets back but they didn‘t reach the client.

Searching for this problem on the web I found this bug. A Solution was added at the bottom of the page but it has only been approved by one person. I built another workaround: The switch connecting my router and the WDR4300 is a Cisco SG200-08, a small managed switch. I configured it to receive VLan 1 from WDR as tagged. So the switch can send the previous untagged packages as tagged with VID 1 and the Cisco takes care the rest.


Posted by toerb | Permanent link | File under: openwrt