i3 testsuite
============
-Michael Stapelberg <michael+i3@stapelberg.de>
-September 2011
+Michael Stapelberg <michael@i3wm.org>
+September 2012
This document explains how the i3 testsuite works, how to use it and extend it.
It is targeted at developers who not necessarily have been doing testing before
test manually if everything works. Having a testcase not only helps you with
that, but it will also be useful for every future change.
+== Relevant documentation
+
+Apart from this document, you should also have a look at:
+
+1. The "Modern Perl" book, which can be found at
+ http://onyxneon.com/books/modern_perl/modern_perl_a4.pdf
+2. The latest Perl documentation of the "i3test" (general testcase setup) and
+ "i3test::Test" (additional test instructions) modules:
+ http://build.i3wm.org/docs/lib-i3test.html respectively
+ http://build.i3wm.org/docs/lib-i3test-test.html
+3. The latest documentation on i3’s IPC interface:
+ http://build.i3wm.org/docs/ipc.html
+
== Implementation
For several reasons, the i3 testsuite has been implemented in Perl:
2. Perl is widely available and has a well-working package infrastructure.
3. The author is familiar with Perl :).
+4. It is a good idea to use a different language for the tests than the
+ implementation itself.
Please do not start programming language flamewars at this point.
+=== Installing the dependencies
+
+As usual with Perl programs, the testsuite ships with a +Makefile.PL+.
+This file specifies which Perl modules the testsuite depends on and can be used
+to install all of them.
+
+Perl modules are distributed via CPAN, and there is the official, standard CPAN
+client, simply called +cpan+. It comes with every Perl installation and can be
+used to install the testsuite. Many users prefer to use the more modern
++cpanminus+ instead, though (because it asks no questions and just works):
+
+The tests additionally require +Xephyr(1)+ to run a nested X server. Install
++xserver-xephyr+ on Debian or +xorg-xserver-xephyr+ on Arch Linux.
+
+.Installing testsuite dependencies using cpanminus (preferred)
+--------------------------------------------------------------------------------
+$ cd ~/i3/testcases
+$ sudo apt-get install cpanminus
+$ sudo cpanm .
+--------------------------------------------------------------------------------
+
+If you don’t want to use cpanminus for some reason, the same works with cpan:
+
+.Installing testsuite dependencies using cpan
+--------------------------------------------------------------------------------
+$ cd ~/i3/testcases
+$ sudo cpan .
+--------------------------------------------------------------------------------
+
+In case you don’t have root permissions, you can also install into your home
+directory, see http://michael.stapelberg.de/cpan/
+
=== Mechanisms
==== Script: complete-run
with an appropriate configuration file and creates a folder for each run
containing the appropriate i3 logfile for each testcase. The latest folder can
always be found under the symlink +latest/+. Unless told differently, it will
-run the tests on a separate X server instance (using the Xdummy script).
+run the tests on a separate X server instance (using Xephyr).
+
+Xephyr will open a window where you can inspect the running test. You can run
+the tests without an X session with Xvfb, such as with +xvfb-run
+./complete-run+. This will also speed up the tests signficantly especially on
+machines without a powerful video card.
.Example invocation of complete-run.pl+
---------------------------------------
$ less latest/i3-log-for-04-floating.t
----------------------------------------
+If your attempt to run the tests with a bare call to ./complete-run.pl fails, try this:
+
+---------------------------------------------------
+$ ./complete-run.pl --parallel=1 --keep-xserver-output
+---------------------------------------------------
+
+This will show the output of Xephyr, which is the X server implementation we
+use for testing.
+
==== IPC interface
The testsuite makes extensive use of the IPC (Inter-Process Communication)
=== Filesystem structure
In the git root of i3, the testcases live in the folder +testcases+. This
-folder contains the +complete-run.pl+ and +Xdummy+ scripts and a base
-configuration file which will be used for the tests. The different testcases
-(their file extension is .t, not .pl) themselves can be found in the
-conventionally named subfolder +t+:
+folder contains the +complete-run.pl+ and a base configuration file which will
+be used for the tests. The different testcases (their file extension is .t, not
+.pl) themselves can be found in the conventionally named subfolder +t+:
.Filesystem structure
--------------------------------------------
│ │ ├── omitted for brevity
│ │ ├── ...
│ │ └── 74-regress-focus-toggle.t
-│ └── Xdummy
--------------------------------------------
== Anatomy of a testcase
projects / i3 / i3.github.io / blobdiff