3. autopkgtest: Automatische Tests für Pakete¶
The DEP 8 specification defines how automatic testing can very easily be integrated into packages. To integrate a test into a package, all you need to do is:
fügen Sie folgendes zum Abschnitt “Source” in debian/control hinzu:
XS-Testsuite: autopkgtest
erstellen Sie eine Datei namens debian/tests/control, die die Anforderungen für die Testumgebung festlegt,
fügen Sie die Tests in debian/tests/ ein.
3.1. Anforderungen für die Testumgebung¶
In debian/tests/control you specify what to expect from the testbed. So for example you list all the required packages for the tests, if the testbed gets broken during the build or if root permissions are required. The DEP 8 specification lists all available options.
Im Folgenden schauen uns das Quellpaket glib2.0 an. Im einfachsten Fall sieht die Datei so aus:
Tests: build
Depends: libglib2.0-dev, build-essential
Das stellt für den Test debian/tests/build sicher, dass die Pakete libglib2.0-dev und build-essential installiert sind.
Bemerkung
Sie können in der Depends-Zeile @ benutzen, um festzulegen, dass Sie alle Pakete installiert haben wollen, die aus dem jeweiligen Quellpaket erzeugt werden.
3.2. Die eigentlichen Tests¶
Der passende Test für das obige Beispiel könnte folgender sein:
#!/bin/sh
# autopkgtest check: Build and run a program against glib, to verify that the
# headers and pkg-config file are installed correctly
# (C) 2012 Canonical Ltd.
# Author: Martin Pitt <martin.pitt@ubuntu.com>
set -e
WORKDIR=$(mktemp -d)
trap "rm -rf $WORKDIR" 0 INT QUIT ABRT PIPE TERM
cd $WORKDIR
cat <<EOF > glibtest.c
#include <glib.h>
int main()
{
g_assert_cmpint (g_strcmp0 (NULL, "hello"), ==, -1);
g_assert_cmpstr (g_find_program_in_path ("bash"), ==, "/bin/bash");
return 0;
}
EOF
gcc -o glibtest glibtest.c `pkg-config --cflags --libs glib-2.0`
echo "build: OK"
[ -x glibtest ]
./glibtest
echo "run: OK"
An dieser Stelle wird ein sehr einfaches Stück C-Code in ein temporäres Verzeichnis geschrieben. Dies wird mit den Systembibliotheken übersetzt (wobei die Flags und Bibliothekpfade benutzt werden, wie sie uns von pkg-config geliefert werden). Dann wird der resultierende Binär-Code ausgeführt, der grundlegende Funtionen in glib testet.
While this test is very small and simple, it covers quite a lot: that your -dev package has all necessary dependencies, that your package installs working pkg-config files, headers and libraries are put into the right place, or that the compiler and linker work. This helps to uncover critical issues early on.
3.3. Den Test ausführen¶
While the test script can be easily executed on its own, it is strongly recommended to actually use adt-run from the autopkgtest package for verifying that your test works; otherwise, if it fails in the Ubuntu Continuous Integration (CI) system, it will not land in Ubuntu. This also avoids cluttering your workstation with test packages or test configuration if the test does something more intrusive than the simple example above.
The README.running-tests (online version) documentation explains all available testbeds (schroot, LXC, QEMU, etc.) and the most common scenarios how to run your tests with adt-run, e. g. with locally built binaries, locally modified tests, etc.
The Ubuntu CI system uses the QEMU runner and runs the tests from the packages in the archive, with -proposed enabled. To reproduce the exact same environment, first install the necessary packages:
sudo apt-get install autopkgtest qemu-system qemu-utils
Now build a testbed with:
adt-buildvm-ubuntu-cloud -v
(Please see its manpage and --help output for selecting different releases, architectures, output directory, or using proxies). This will build e. g. adt-trusty-amd64-cloud.img.
Then run the tests of a source package like libpng in that QEMU image:
adt-run libpng --- qemu adt-trusty-amd64-cloud.img
The Ubuntu CI system runs packages with -proposed enabled; to enable that, run:
adt-run libpng -U --apt-pocket=proposed --- qemu adt-trusty-amd64-cloud.img
The adt-run manpage has a lot more valuable information on other testing options.
3.4. Weitere Beispiele¶
Diese Liste ist nicht vollständig, aber wird dir sicher einen guten Einblick geben wie automatisierte Testdurchläufe in Ubuntu implementiert und benutzt werden.
- The libxml2 tests are very similar. They also run a test-build of a simple piece of C code and execute it.
- The gtk+3.0 tests also do a compile/link/run check in the “build” test. There is an additional “python3-gi” test which verifies that the GTK library can also be used through introspection.
- In the ubiquity tests the upstream test-suite is executed.
- The gvfs tests have comprehensive testing of their functionality and are very interesting because they emulate usage of CDs, Samba, DAV and other bits.
3.5. Ubuntu Infrastruktur¶
Packages which have autopkgtest enabled will have their tests run whenever they get uploaded or any of their dependencies change. The output of automatically run autopkgtest tests can be viewed on the web and is regularly updated.
Debian also uses adt-run to run package tests, although currently only in schroots, so results may vary a bit. Results and logs can be seen on http://ci.debian.net. So please submit any test fixes or new tests to Debian as well.
3.6. Die Tests in Ubuntu bekommen¶
Der Prozess, um einen autopkgtest in Ubuntu einzubringen ist größtenteils identisch mit fixing a bug in Ubuntu. Im Prinzip muss man einfach:
Führe bzr branch ubuntu:<Paketname> aus,
bearbeiten Sie debian/control um die Tests zu aktivitieren,
fügen Sie ein debian/tests-Verzeichnis hinzu,
- write the debian/tests/control based on the DEP 8 Specification,
fügen Sie Ihre(n) Test(s) zu debian/tests hinzu,
die Änderungen committen, sie nach Launchpad hochladen, und einen Merge vorschlagen, sie dann überprüfen lassen, genau wie jede andere Änderung an einem Quellpaket auch.
3.7. Was du machen kannst¶
The Ubuntu Engineering team put together a list of required test-cases, where packages which need tests are put into different categories. Here you can find examples of these tests and easily assign them to yourself.
If you should run into any problems, you can join the #ubuntu-quality IRC channel to get in touch with developers who can help you.