i3 testsuite
============
-Michael Stapelberg <michael+i3@stapelberg.de>
-October 2011
+Michael Stapelberg <michael@i3wm.org>
+September 2012
This document explains how the i3 testsuite works, how to use it and extend it.
It is targeted at developers who not necessarily have been doing testing before
test manually if everything works. Having a testcase not only helps you with
that, but it will also be useful for every future change.
+== Relevant documentation
+
+Apart from this document, you should also have a look at:
+
+1. The "Modern Perl" book, which can be found at
+ http://onyxneon.com/books/modern_perl/modern_perl_a4.pdf
+2. The latest Perl documentation of the "i3test" (general testcase setup) and
+ "i3test::Test" (additional test instructions) modules:
+ http://build.i3wm.org/docs/lib-i3test.html respectively
+ http://build.i3wm.org/docs/lib-i3test-test.html
+3. The latest documentation on i3’s IPC interface:
+ http://build.i3wm.org/docs/ipc.html
+
== Implementation
For several reasons, the i3 testsuite has been implemented in Perl:
2. Perl is widely available and has a well-working package infrastructure.
3. The author is familiar with Perl :).
+4. It is a good idea to use a different language for the tests than the
+ implementation itself.
Please do not start programming language flamewars at this point.
+=== Installing the dependencies
+
+As usual with Perl programs, the testsuite ships with a +Makefile.PL+.
+This file specifies which Perl modules the testsuite depends on and can be used
+to install all of them.
+
+Perl modules are distributed via CPAN, and there is the official, standard CPAN
+client, simply called +cpan+. It comes with every Perl installation and can be
+used to install the testsuite. Many users prefer to use the more modern
++cpanminus+ instead, though (because it asks no questions and just works):
+
+.Installing testsuite dependencies using cpanminus (preferred)
+--------------------------------------------------------------------------------
+$ cd ~/i3/testcases
+$ sudo apt-get install cpanminus
+$ sudo cpanm .
+--------------------------------------------------------------------------------
+
+If you don’t want to use cpanminus for some reason, the same works with cpan:
+
+.Installing testsuite dependencies using cpan
+--------------------------------------------------------------------------------
+$ cd ~/i3/testcases
+$ sudo cpan .
+--------------------------------------------------------------------------------
+
+In case you don’t have root permissions, you can also install into your home
+directory, see http://michael.stapelberg.de/cpan/
+
=== Mechanisms
==== Script: complete-run
testcases by default, but you can be more specific and let it only run one or
more testcases. Also, it takes care of starting up a separate instance of i3
with an appropriate configuration file and creates a folder for each run
-containing the appropriate i3 logfile for each testcase. The latest folder can
-always be found under the symlink +latest/+. It is recommended that you run the
-tests on one or more separate X server instances (you can only start one window
-manager per X session), for example using the provided Xdummy script.
-+complete-run.pl+ takes one or more X11 display specifications and parallelizes
-the testcases appropriately:
+containing the appropriate i3 logfile for each testcase. The latest folder can
+always be found under the symlink +latest/+. Unless told differently, it will
+run the tests on a separate X server instance (using the Xdummy script).
.Example invocation of complete-run.pl+
---------------------------------------
$ cd ~/i3/testcases
-# start two dummy X11 instances in the background
-$ ./Xdummy :1 &
-$ ./Xdummy :2 &
-
-$ ./complete-run.pl -d :1,:2
+$ ./complete-run.pl
# output omitted because it is very long
All tests successful.
Files=78, Tests=734, 27 wallclock secs ( 0.38 usr 0.48 sys + 17.65 cusr 3.21 csys = 21.72 CPU)
Result: PASS
-$ ./complete-run.pl -d :1 t/04-floating.t
+$ ./complete-run.pl t/04-floating.t
[:3] i3 startup: took 0.07s, status = 1
[:3] Running t/04-floating.t with logfile testsuite-2011-09-24-16-06-04-4.0.2-226-g1eb011a/i3-log-for-04-floating.t
[:3] t/04-floating.t finished
$ less latest/i3-log-for-04-floating.t
----------------------------------------
+If your attempt to run the tests with a bare call to ./complete-run.pl fails, try this:
+
+---------------------------------------------------
+$ ./complete-run.pl --parallel=1 --keep-xdummy-output
+---------------------------------------------------
+
+One common cause of failures is not having the X dummy server module
+installed. Under Debian and Ubuntu this is the package
++xserver-xorg-video-dummy+.
+
==== IPC interface
The testsuite makes extensive use of the IPC (Inter-Process Communication)
├── testcases
│ ├── complete-run.pl
│ ├── i3-test.config
+│ ├── lib
+│ │ ├── i3test.pm
+│ │ ├── SocketActivation.pm
+│ │ └── StartXDummy.pm
│ ├── t
│ │ ├── 00-load.t
│ │ ├── 01-tile.t
│ │ ├── ...
│ │ ├── omitted for brevity
│ │ ├── ...
-│ │ ├── 74-regress-focus-toggle.t
-│ │ └── lib
-│ │ └── i3test.pm
+│ │ └── 74-regress-focus-toggle.t
│ └── Xdummy
--------------------------------------------
An alternative approach to using socket activation is polling for the existance
of the IPC socket and connecting to it. While this might be slightly easier to
-implement, it wastes CPU time and is considerably more ugly than this solution
+implement, it wastes CPU time and is considerably uglier than this solution
:). After all, +lib/SocketActivation.pm+ contains only 54 SLOC.
projects / i3 / i3 / blobdiff