ccomb

Comparatif Dedibox V1 / V2 / V3

Lecture

En lecture, la dedibox v3 est la plus rapide des trois, il n'y a aucun doute.

Écriture

En écriture, c'est un peu moins clair : la dedibox v3 est plus rapide dans 4 tests sur 6.

Comparatif Ubuntu / Debian

Voyons maintenant la différence entre Debian et Ubuntu. Ici les machines sont censées être les mêmes, sont toutes les deux vides et non utilisées, et sont installées avec les dernières versions de Debian 6.0 et Ubuntu 10.04. Les noyaux sont tous les deux des 2.6.32.

On note une très forte différence entre les deux : la machine sous Debian est beaucoup plus rapide.

Lecture

Écriture

Bizzare, mais même constat, la dedibox sous Debian est en moyenne une fois et demi plus rapide que celle avec Ubuntu.

Je me garderai de faire des commentaires, vu les conditions dans lesquelles ces tests ont été réalisés, mais ça mériterait de creuser un peu...

Script d'extraction

Si vous voulez reproduire les grahiques, voici comment faire.

Il faut récupérer la sortie de « iozone -a » en ne gardant que le tableau, puis changer l'en-tête du tableau pour qu'il tienne sur une seule ligne, et sauver ça dans un fichier texte. Ensuite il faut installer Matplotlib 1.0. La version 0.99 fournie avec Ubuntu 10.04 ne convient pas, il faut compiler la version 1.0. Je vous laisse le doux plaisir de découvrir les dépendances à installer

$ virtualenv sandbox
$ sandbox/bin/pip install  install http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.0/matplotlib-1.0.0.tar.gz/download

Ensuite voici le script qui a permis d'extraire et tracer les données, que j'appelle bench.py :

#!/usr/bin/env python
# coding: utf-8
import sys
import pylab, numpy as np
from numpy import log2
from mpl_toolkits.mplot3d import axes3d

tests = {}

files = sys.argv[1:]
pylab_colors = ('r', 'g', 'b', 'y', 'm', 'y', 'k')
assert len(files) <= len(pylab_colors)
colors = dict([(f, pylab_colors[i]) for i, f in enumerate(files)])

for k, filename in enumerate(files):
    with open(filename) as f:
        tests[filename] = {}
        tests[filename]['titles'] = f.readline().split()[2:]
    tests[filename]['raw'] = np.loadtxt(filename, dtype=int, skiprows=1)
    for i, testname in enumerate(tests[filename]['titles']):
        tests[filename][testname] = {}
        data = tests[filename][testname]['data'] = np.zeros((20, 15))
        data[:] = np.nan
        raw = tests[filename]['raw']
        data[log2(raw[:,0]).astype(int), log2(raw[:,1]).astype(int)] = raw[:, i+2]
        data /= 1024.0
        tests[filename][testname]['mean'] = data[-np.isnan(data)].mean()
        tests[filename][testname]['max'] = data[-np.isnan(data)].max()


axes = {}
X, Y = np.mgrid[0:20,0:15]
for testname in tests.values()[0]['titles']:
    ax = axes3d.Axes3D(pylab.figure())
    ax.w_xaxis.set_ticks(range(3,21,2))
    ax.w_yaxis.set_ticks(range(2,15,2))
    ax.w_xaxis.set_ticklabels(2**np.arange(3,21,2))
    ax.w_yaxis.set_ticklabels(2**np.arange(2,15,2))
    ax.set_xlabel('kB')
    ax.set_ylabel('reclen')
    ax.set_zlabel('MB/s')
    means = dict([(filename, tests[filename][testname]['mean']) for filename in files])
    maxs = max([tests[filename][testname]['max'] for filename in files])
    ax.text(X.min(), 15, maxs, 'test: ' + testname, color='k')
    for i, filename in enumerate(sorted(means, key=lambda x:-means[x])):
        i+=1
        Z = tests[filename][testname]['data']
        c = colors[filename]
        ax.plot_wireframe(X, Y, Z, color=c)
        ax.text(X.min(), 15, maxs*(1-0.07*i),
                str(int(means[filename])) + " MB/s : " + filename, color=c)
    #pylab.gcf().set_size_inches(6,4)
    pylab.savefig(testname + '.png')

pylab.show()

Pour le lancer avec la nouvelle version de matplotlib, il suffit d'utiliser le Python de la sandbox. Le script prend en paramètres les fichiers contenant les résultats iozone

$ ./sandbox/bin/python bench.py resultat1.txt resultat2.txt resultat3.txt

Il est possible d'écrire un script beaucoup plus propre, mais je voulais juste m'obliger à le faire avec Numpy et Matplotlib et sans y passer trop de temps non plus. Notez qu'une des lignes du script utilise une affectation grâce au Fancy Indexing de Numpy:

data[log2(raw[:,0]).astype(int), log2(raw[:,1]).astype(int)] = raw[:, i+2]

z3c.checkversions

As part of my work on the the Zope Toolkit 1.0 and BlueBream 1.0, I've released several versions of z3c.checkversions. This package is part of the z3c (Zope 3 Community) namespace, but it does not depend on any Zope package and has zero dependencies (except setuptools). It provides you with a single checkversions script which can be used with any project. This is particularly useful, for instance, on large Plone projects, with many packages. The more packages you use, the more bugs you find. You shouldn't loose time on bugs that are already fixed by newer versions of packages! In the Zope community, it's generally safe to upgrade to the latest minor versions of packages. If you're using package foo version 1.0.2, and discover that version 1.0.6 and 1.1.0 are available, you can upgrade immediately to 1.0.6 without risk.

As a summary, versions are commonly formed with 3 numbers : X.Y.Z:

• a new minor Z version only provides bugfixes. No new features are allowed.
• a new intermediate Y version should provide new (backward compatible) features
• a new major X version is expected to cause breakages and is probably backward-incompatible.

Discovering new versions on the system Python

Install z3c.checkversions as an administrator. This is safe because there is no dependencies, and you can uninstall it easily with pip afterwards:

$ sudo pip install z3c.checkversions  # or sudo easy_install z3c.checkversions
$ checkversions
# Checking your installed distributions
pida=0.6.2
python-dateutil=1.5
genshi=0.6
buildbot=0.8.1
pyparsing=1.5.3
(...)

If you add the -v option, all parsed versions will be displayed, as well as the current installed versions:

$ checkversions -v
# Checking your installed distributions
twisted-words=10.0.0
pida=0.6.2 # was: 0.5.0
foolscap=0.5.1
distribute=0.6.14
twisted-names=10.0.0
bicyclerepair=0.9
python-dateutil=1.5 # was: 1.4.1
genshi=0.6 # was: 0.5.1
buildbot=0.8.1 # was: 0.7.12
pyparsing=1.5.3 # was: 1.5.2
(...)

If you only want the new intermediate versions, add -l 1. If you only want the new minor (bugfix) versions, add -l 2 instead. If you ever have some packages with 4 version digits, you can even use -l 3:

$ checkversions -v -l 2
# Checking your installed distributions
twisted-words=10.0.0
pida=0.5.1 # was: 0.5.0
foolscap=0.5.1
distribute=0.6.14
twisted-names=10.0.0
bicyclerepair=0.9
python-dateutil=1.5
genshi=0.6
buildbot=0.8.1
pyparsing=1.5.3
(...)

Now you can uninstall it:

$ sudo pip uninstall z3c.checkversions

Working with a virtualenv

Do the same, but install it in a virtualenv:

$ virtualenv --no-site-packages --distribute sandbox
$ sandbox/bin/pip install z3c.checkversions

Then the script is available only in the virtualenv:

$ sandbox/bin/checkversions -v
# Checking your installed distributions
pip=0.8 # was: 0.7.1
python=2.6
distribute=0.6.14 # was: 0.6.10
z3c.checkversions=0.4
wsgiref=0.1.2
(...)

Working with a buildout

If the [versions] section of your buildout is huge, finding new versions one by one can be painful, particularly if you want to find only new minor versions. In that case just install z3c.checkversions in your buildout. Don't forget to enable the [buildout] extra:

[some_part]
recipe = zc.recipe.egg
eggs = z3c.checkversions [buildout]

Then run your buildout again, and run the checkversions script by passing the buildout.cfg (or versions.cfg) file as an argument:

$ bin/checkversions -v -l 2  versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.6 # was: 3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

Blacklist and incremental options

While setting up the AFPY buildbots for the Zope Toolkit, I've been experimenting with a special buildbot that would be able to create a new Zope Toolkit version list automatically, by testing arbitrary new versions. For this purpose I've added two options to z3c.checkversions:

• -1 or --incremental
• -b or --blacklist

The first option allows to suggest only one new version:

$ checkversions -v -l 2 --incremental versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

The second option allows to not suggest a version which is in a blacklist version file. Say we have the following blacklist.cfg file:

zope.app.container=3.5.6

Then the script will suggest all the highest new versions, except the one in the blacklist:

$ checkversions -v -l 2 -b blacklist.cfg versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.5 # was: 3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

Since zope.app.container=3.5.6 is part of the blacklist, the script suggests the version just below it: zope.app.container=3.5.5.

Automatic discovery of the Zope Toolkit version list

I'm using these two options in this special Zope Toolkit buildbot: http://buildbot.afpy.org:8018/waterfall

This buildbot automatically upgrades one package version (builder at the left), then run the tests. If tests fail, the package is added in the blacklist (builder at the right). In the next run, the buildbot will choose the version just below. If tests fail, it will downgrade again. If tests pass, it will remember the current versions and try to upgrade another package. When the buildbot finally reach a stable state, we automatically get all the latest working versions.

During the first cycles, the buildbot has given the following behaviour :

• it started with ZTK 1.0a2
• it upgraded (arbitrary) zope.dublincore 3.6.0 -> 3.6.3
• tests failed
• it added zope.dublincore 3.6.3 in the blacklist
• it upgraded zope.dublincore 3.6.0 -> 3.6.2
• tests failed
• it added zope.dublincore 3.6.2 in the blacklist
• it upgraded zope.dublincore 3.6.0 -> 3.6.1
• tests failed
• it added zope.dublincore 3.6.1 in the blacklist
• so it keeps zope.dublincore 3.6.0 (until 3.6.4 is released)
• it upgraded lxml 2.2.6 -> 2.2.7
• tests passed
• lxml 2.2.7 is kept for future builds
• etc....

The cycle is not optimal, because zope.dublincore 3.6.3 maybe works well when upgraded with another package. So maybe I should try the opposite: upgrading packages bottom up, one by one, until the highest one is reached.