1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
\r
2 "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
\r
3 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
\r
5 <link rel="icon" type="image/x-icon" href="/favicon.ico">
\r
6 <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=UTF-8" />
\r
7 <meta name="generator" content="AsciiDoc 8.6.9" />
\r
8 <title>i3: i3 testsuite</title>
\r
9 <link rel="stylesheet" href="/css/style.css" type="text/css" />
\r
10 <link rel="stylesheet" href="/css/xhtml11.css" type="text/css" />
\r
11 <script type="text/javascript">
\r
13 document.addEventListener("DOMContentLoaded", function(){asciidoc.footnotes(); asciidoc.toc(2);}, false);
\r
16 <script type="text/javascript" src="/js/asciidoc-xhtml11.js"></script>
\r
18 <body class="article">
\r
21 <a href="/"><h1 id="title">i3 - improved tiling WM</h1></a>
\r
23 <li><a style="border-bottom: 2px solid #fff" href="/docs">Docs</a></li>
\r
24 <li><a href="/screenshots">Screens</a></li>
\r
25 <li><a href="https://www.reddit.com/r/i3wm/">FAQ</a></li>
\r
26 <li><a href="/contact">Contact</a></li>
\r
27 <li><a href="https://github.com/i3/i3/issues">Bugs</a></li>
\r
29 <br style="clear: both">
\r
32 <h1>i3 testsuite</h1>
\r
33 <span id="author">Michael Stapelberg</span><br />
\r
34 <span id="email"><tt><<a href="mailto:michael@i3wm.org">michael@i3wm.org</a>></tt></span><br />
\r
35 <span id="revdate">September 2012</span>
\r
37 <div id="toctitle">Table of Contents</div>
38 <noscript><p><b>JavaScript must be enabled in your browser to display the table of contents.</b></p></noscript>
42 <div class="sectionbody">
\r
43 <div class="paragraph"><p>This document explains how the i3 testsuite works, how to use it and extend it.
\r
44 It is targeted at developers who not necessarily have been doing testing before
\r
45 or have not been testing in Perl before. In general, the testsuite is not of
\r
46 interest for end users.</p></div>
\r
50 <h2 id="_introduction">1. Introduction</h2>
\r
51 <div class="sectionbody">
\r
52 <div class="paragraph"><p>The i3 testsuite is a collection of files which contain testcases for various
\r
53 i3 features. Some of them test if a certain workflow works correctly (moving
\r
54 windows, focus behaviour, …). Others are regression tests and contain code
\r
55 which previously made i3 crash or lead to unexpected behaviour. They then check
\r
56 if i3 still runs (meaning it did not crash) and if it handled everything
\r
57 correctly.</p></div>
\r
58 <div class="paragraph"><p>The goal of having these tests is to automatically find problems and to
\r
59 automatically get a feel for whether a change in the source code breaks any
\r
60 existing feature. After every modification of the i3 sourcecode, the developer
\r
61 should run the full testsuite. If one of the tests fails, the corresponding
\r
62 problem should be fixed (or, in some cases, the testcase has to be modified).
\r
63 For every bugreport, a testcase should be written to test the correct
\r
64 behaviour. Initially, it will fail, but after fixing the bug, it will pass.
\r
65 This ensures (or increases the chance) that bugs which have been fixed once
\r
66 will never be found again.</p></div>
\r
67 <div class="paragraph"><p>Also, when implementing a new feature, a testcase might be a good way to be
\r
68 able to easily test if the feature is working correctly. Many developers will
\r
69 test manually if everything works. Having a testcase not only helps you with
\r
70 that, but it will also be useful for every future change.</p></div>
\r
74 <h2 id="_relevant_documentation">2. Relevant documentation</h2>
\r
75 <div class="sectionbody">
\r
76 <div class="paragraph"><p>Apart from this document, you should also have a look at:</p></div>
\r
77 <div class="olist arabic"><ol class="arabic">
\r
80 The "Modern Perl" book, which can be found at
\r
81 <a href="http://onyxneon.com/books/modern_perl/modern_perl_a4.pdf">http://onyxneon.com/books/modern_perl/modern_perl_a4.pdf</a>
\r
86 The latest Perl documentation of the "i3test" (general testcase setup) and
\r
87 "i3test::Test" (additional test instructions) modules:
\r
88 <a href="http://build.i3wm.org/docs/lib-i3test.html">http://build.i3wm.org/docs/lib-i3test.html</a> respectively
\r
89 <a href="http://build.i3wm.org/docs/lib-i3test-test.html">http://build.i3wm.org/docs/lib-i3test-test.html</a>
\r
94 The latest documentation on i3’s IPC interface:
\r
95 <a href="http://build.i3wm.org/docs/ipc.html">http://build.i3wm.org/docs/ipc.html</a>
\r
101 <div class="sect1">
\r
102 <h2 id="_implementation">3. Implementation</h2>
\r
103 <div class="sectionbody">
\r
104 <div class="paragraph"><p>For several reasons, the i3 testsuite has been implemented in Perl:</p></div>
\r
105 <div class="olist arabic"><ol class="arabic">
\r
108 Perl has a long tradition of testing. Every popular/bigger Perl module which
\r
109 you can find on CPAN will not only come with documentation, but also with
\r
110 tests. Therefore, the available infrastructure for tests is comprehensive.
\r
111 See for example the excellent <a href="http://search.cpan.org/perldoc?Test::More">http://search.cpan.org/perldoc?Test::More</a>
\r
112 and the referenced <a href="http://search.cpan.org/perldoc?Test::Tutorial">http://search.cpan.org/perldoc?Test::Tutorial</a>.
\r
117 Perl is widely available and has a well-working package infrastructure.
\r
122 The author is familiar with Perl :).
\r
127 It is a good idea to use a different language for the tests than the
\r
128 implementation itself.
\r
132 <div class="paragraph"><p>Please do not start programming language flamewars at this point.</p></div>
\r
133 <div class="sect2">
\r
134 <h3 id="_installing_the_dependencies">3.1. Installing the dependencies</h3>
\r
135 <div class="paragraph"><p>As usual with Perl programs, the testsuite ships with a <tt>Makefile.PL</tt>.
\r
136 This file specifies which Perl modules the testsuite depends on and can be used
\r
137 to install all of them.</p></div>
\r
138 <div class="paragraph"><p>Perl modules are distributed via CPAN, and there is the official, standard CPAN
\r
139 client, simply called <tt>cpan</tt>. It comes with every Perl installation and can be
\r
140 used to install the testsuite. Many users prefer to use the more modern
\r
141 <tt>cpanminus</tt> instead, though (because it asks no questions and just works):</p></div>
\r
142 <div class="paragraph"><p>The tests additionally require <tt>Xephyr(1)</tt> to run a nested X server. Install
\r
143 <tt>xserver-xephyr</tt> on Debian or <tt>xorg-xserver-xephyr</tt> on Arch Linux.</p></div>
\r
144 <div class="listingblock">
\r
145 <div class="title">Installing testsuite dependencies using cpanminus (preferred)</div>
\r
146 <div class="content">
\r
147 <pre><tt>$ cd ~/i3/testcases
\r
148 $ sudo apt-get install cpanminus
\r
150 $ cd ~/i3/AnyEvent-I3
\r
151 $ sudo cpanm .</tt></pre>
\r
153 <div class="paragraph"><p>If you don’t want to use cpanminus for some reason, the same works with cpan:</p></div>
\r
154 <div class="listingblock">
\r
155 <div class="title">Installing testsuite dependencies using cpan</div>
\r
156 <div class="content">
\r
157 <pre><tt>$ cd ~/i3/testcases
\r
159 $ cd ~/i3/AnyEvent-I3
\r
160 $ sudo cpan .</tt></pre>
\r
162 <div class="paragraph"><p>In case you don’t have root permissions, you can also install into your home
\r
163 directory, see <a href="http://michael.stapelberg.de/cpan/">http://michael.stapelberg.de/cpan/</a></p></div>
\r
165 <div class="sect2">
\r
166 <h3 id="_mechanisms">3.2. Mechanisms</h3>
\r
167 <div class="sect3">
\r
168 <h4 id="_script_complete_run">3.2.1. Script: complete-run</h4>
\r
169 <div class="paragraph"><p>The testcases are run by a script called <tt>complete-run.pl</tt>. It runs all
\r
170 testcases by default, but you can be more specific and let it only run one or
\r
171 more testcases. Also, it takes care of starting up a separate instance of i3
\r
172 with an appropriate configuration file and creates a folder for each run
\r
173 containing the appropriate i3 logfile for each testcase. The latest folder can
\r
174 always be found under the symlink <tt>latest/</tt>. Unless told differently, it will
\r
175 run the tests on a separate X server instance (using Xephyr).</p></div>
\r
176 <div class="paragraph"><p>Xephyr will open a window where you can inspect the running test. You can run
\r
177 the tests without an X session with Xvfb, such as with <tt>xvfb-run
\r
178 ./complete-run</tt>. This will also speed up the tests significantly especially on
\r
179 machines without a powerful video card.</p></div>
\r
180 <div class="listingblock">
\r
181 <div class="title">Example invocation of <tt>complete-run.pl</tt></div>
\r
182 <div class="content">
\r
187 $ mkdir -p build && cd build
\r
192 # output omitted because it is very long
\r
196 $ ./complete-run.pl
\r
197 # output omitted because it is very long
\r
198 All tests successful.
\r
199 Files=78, Tests=734, 27 wallclock secs ( 0.38 usr 0.48 sys + 17.65 cusr 3.21 csys = 21.72 CPU)
\r
202 $ ./complete-run.pl t/04-floating.t
\r
203 [:3] i3 startup: took 0.07s, status = 1
\r
204 [:3] Running t/04-floating.t with logfile testsuite-2011-09-24-16-06-04-4.0.2-226-g1eb011a/i3-log-for-04-floating.t
\r
205 [:3] t/04-floating.t finished
\r
207 output for t/04-floating.t:
\r
208 ok 1 - use X11::XCB::Window;
\r
209 ok 2 - The object isa X11::XCB::Window
\r
210 ok 3 - Window is mapped
\r
211 ok 4 - i3 raised the width to 75
\r
212 ok 5 - i3 raised the height to 50
\r
213 ok 6 - i3 did not map it to (0x0)
\r
214 ok 7 - The object isa X11::XCB::Window
\r
215 ok 8 - i3 let the width at 80
\r
216 ok 9 - i3 let the height at 90
\r
217 ok 10 - i3 mapped it to x=1
\r
218 ok 11 - i3 mapped it to y=18
\r
219 ok 12 - The object isa X11::XCB::Window
\r
220 ok 13 - i3 let the width at 80
\r
221 ok 14 - i3 let the height at 90
\r
224 All tests successful.
\r
225 Files=1, Tests=14, 0 wallclock secs ( 0.01 usr 0.00 sys + 0.19 cusr 0.03 csys = 0.23 CPU)
\r
228 $ less latest/i3-log-for-04-floating.t</tt></pre>
\r
230 <div class="paragraph"><p>If your attempt to run the tests with a bare call to ./complete-run.pl fails, try this:</p></div>
\r
231 <div class="listingblock">
\r
232 <div class="content">
\r
233 <pre><tt>$ ./complete-run.pl --parallel=1 --keep-xserver-output</tt></pre>
\r
235 <div class="paragraph"><p>This will show the output of Xephyr, which is the X server implementation we
\r
236 use for testing.</p></div>
\r
237 <div class="sect4">
\r
238 <h5 id="_make_command_tt_make_check_tt">make command: <tt>make check</tt></h5>
\r
239 <div class="paragraph"><p>Make check runs the i3 testsuite.
\r
240 You can still use ./testcases/complete-run.pl to get the interactive progress output.</p></div>
\r
241 <div class="listingblock">
\r
242 <div class="title">Example invocation of <tt>make check</tt></div>
\r
243 <div class="content">
\r
248 $ mkdir -p build && cd build
\r
253 # output omitted because it is very long
\r
256 # output omitted because it is very long
\r
257 PASS: testcases/complete-run.pl
\r
258 ============================================================================
\r
259 Testsuite summary for i3 4.13
\r
260 ============================================================================
\r
268 ============================================================================
\r
270 $ less test-suite.log</tt></pre>
\r
274 <div class="sect3">
\r
275 <h4 id="_coverage_testing">3.2.2. Coverage testing</h4>
\r
276 <div class="paragraph"><p>Coverage testing is possible with <tt>lcov</tt>, the front-end for GCC’s coverage
\r
277 testing tool <tt>gcov</tt>. The testcases can generate a nice html report that tells
\r
278 you which functions and lines were covered during a run of the tests. You can
\r
279 use this tool to judge how effective your tests are.</p></div>
\r
280 <div class="paragraph"><p>To use test coverage tools, first compile with coverage enabled.</p></div>
\r
281 <div class="listingblock">
\r
282 <div class="content">
\r
283 <pre><tt>COVERAGE=1 make</tt></pre>
\r
285 <div class="paragraph"><p>Then run the tests with the <tt>--coverage-testing</tt> flag.</p></div>
\r
286 <div class="listingblock">
\r
287 <div class="content">
\r
288 <pre><tt>./complete-run.pl --coverage-testing</tt></pre>
\r
290 <div class="paragraph"><p>Then open <tt>latest/i3-coverage/index.html</tt> in your web browser.</p></div>
\r
292 <div class="sect3">
\r
293 <h4 id="_ipc_interface">3.2.3. IPC interface</h4>
\r
294 <div class="paragraph"><p>The testsuite makes extensive use of the IPC (Inter-Process Communication)
\r
295 interface which i3 provides. It is used for the startup process of i3, for
\r
296 terminating it cleanly and (most importantly) for modifying and getting the
\r
297 current state (layout tree).</p></div>
\r
298 <div class="paragraph"><p>See [<a href="http://i3wm.org/docs/ipc.html">http://i3wm.org/docs/ipc.html</a>] for documentation on the IPC interface.</p></div>
\r
300 <div class="sect3">
\r
301 <h4 id="_x11_xcb">3.2.4. X11::XCB</h4>
\r
302 <div class="paragraph"><p>In order to open new windows, change attributes, get events, etc., the
\r
303 testsuite uses X11::XCB, a new (and quite specific to i3 at the moment) Perl
\r
304 module which uses the XCB protocol description to generate Perl bindings to
\r
305 X11. They work in a very similar way to libxcb (which i3 uses) and provide
\r
306 relatively high-level interfaces (objects such as <tt>X11::XCB::Window</tt>) as well as
\r
307 access to the low-level interface, which is very useful when testing a window
\r
311 <div class="sect2">
\r
312 <h3 id="_filesystem_structure">3.3. Filesystem structure</h3>
\r
313 <div class="paragraph"><p>In the git root of i3, the testcases live in the folder <tt>testcases</tt>. This
\r
314 folder contains the <tt>complete-run.pl</tt> and a base configuration file which will
\r
315 be used for the tests. The different testcases (their file extension is .t, not
\r
316 .pl) themselves can be found in the conventionally named subfolder <tt>t</tt>:</p></div>
\r
317 <div class="listingblock">
\r
318 <div class="title">Filesystem structure</div>
\r
319 <div class="content">
\r
320 <pre><tt>├── testcases
\r
321 │ ├── complete-run.pl
\r
322 │ ├── i3-test.config
\r
325 │ │ ├── SocketActivation.pm
\r
326 │ │ └── StartXDummy.pm
\r
330 │ │ ├── 02-fullscreen.t
\r
332 │ │ ├── omitted for brevity
\r
334 │ │ └── 74-regress-focus-toggle.t</tt></pre>
\r
339 <div class="sect1">
\r
340 <h2 id="_anatomy_of_a_testcase">4. Anatomy of a testcase</h2>
\r
341 <div class="sectionbody">
\r
342 <div class="paragraph"><p>Learning by example is definitely a good strategy when you are wondering how to
\r
343 write a testcase. Let’s take <tt>t/11-goto.t</tt> as an easy example and go through it
\r
344 step by step:</p></div>
\r
345 <div class="listingblock">
\r
346 <div class="title">t/11-goto.t: Boilerplate</div>
\r
347 <div class="content">
\r
349 # vim:ts=4:sw=4:expandtab
\r
354 my $x = X11::XCB::Connection->new;</tt></pre>
\r
356 <div class="paragraph"><p>This is what we call boilerplate. It exists at the top of every test file (to
\r
357 some extent). The first line is the shebang, which specifies that this file is
\r
358 a Perl script. The second line contains VIM specific settings on how to
\r
359 edit/format this file (use spaces instead of tabs, indent using 4 spaces).
\r
360 Afterwards, the <tt>i3test</tt> module is used. This module contains i3 testsuite
\r
361 specific functions which you are strongly encouraged to use. They make writing
\r
362 testcases a lot easier and will make it easier for other people to read your
\r
364 <div class="paragraph"><p>The next line uses the <tt>File::Temp</tt> module. This is specific to this testcase,
\r
365 because it needs to generate a temporary name during the test. Many testcases
\r
366 use only the <tt>i3test</tt> module.</p></div>
\r
367 <div class="paragraph"><p>The last line opens a connection to X11. You might or might not need this in
\r
368 your testcase, depending on whether you are going to open windows (etc.) or
\r
369 only use i3 commands.</p></div>
\r
370 <div class="listingblock">
\r
371 <div class="title">t/11-goto.t: Setup</div>
\r
372 <div class="content">
\r
373 <pre><tt>my $tmp = fresh_workspace;
\r
375 cmd 'split h';</tt></pre>
\r
377 <div class="paragraph"><p>The first line calls i3test’s <tt>fresh_workspace</tt> function which looks for a
\r
378 currently unused workspace, switches to it, and returns its name. The variable
\r
379 <tt>$tmp</tt> will end up having a value such as <tt>"/tmp/87kBVcHbA9"</tt>. Note that this
\r
380 is not (necessarily) a valid path, it’s just a random workspace name.</p></div>
\r
381 <div class="paragraph"><p>So, now that we are on a new workspace, we ensure that the workspace uses
\r
382 horizontal orientation by issuing the <tt>split h</tt> command (see the i3 User’s
\r
383 Guide for a list of commands). This is not strictly necessary, but good style.
\r
384 In general, the <tt>cmd</tt> function executes the specified i3 command by using the
\r
385 IPC interface and returns once i3 acknowledged the command.</p></div>
\r
386 <div class="listingblock">
\r
387 <div class="title">t/11-goto.t: Setup</div>
\r
388 <div class="content">
\r
389 <pre><tt>#####################################################################
\r
390 # Create two windows and make sure focus switching works
\r
391 #####################################################################
\r
393 my $top = open_window($x);
\r
394 my $mid = open_window($x);
\r
395 my $bottom = open_window($x);</tt></pre>
\r
397 <div class="paragraph"><p>In every major section of a testcase, you should put a comment like the one
\r
398 above. This makes it immediately clear how the file is structured.</p></div>
\r
399 <div class="paragraph"><p>The <tt>open_window</tt> function opens a standard window, which will then be put into
\r
400 tiling mode by i3. If you want a floating window, use the
\r
401 <tt>open_floating_window</tt> function. These functions accept the same parameters as
\r
402 <tt>X11::XCB::Window→new</tt>, see the i3test documentation at TODO.</p></div>
\r
403 <div class="listingblock">
\r
404 <div class="title">t/11-goto.t: Helper function</div>
\r
405 <div class="content">
\r
407 # Returns the input focus after sending the given command to i3 via IPC
\r
408 # and syncing with i3
\r
415 return $x->input_focus;
\r
418 <div class="paragraph"><p>This section defines a helper function which will be used over and over in this
\r
419 testcase. If you have code which gets executed more than once or twice
\r
420 (depending on the length of your test, use your best judgement), please put it
\r
421 in a function. Tests should be short, concise and clear.</p></div>
\r
422 <div class="paragraph"><p>The <tt>focus_after</tt> function executes a command and returns the X11 focus after
\r
423 the command was executed. The <tt>sync_with_i3</tt> command makes sure that i3 could
\r
424 push its state to X11. See <a href="#i3_sync">[i3_sync]</a> to learn how this works exactly.</p></div>
\r
425 <div class="listingblock">
\r
426 <div class="title">t/11-goto.t: Test assumptions</div>
\r
427 <div class="content">
\r
428 <pre><tt>$focus = $x->input_focus;
\r
429 is($focus, $bottom->id, "Latest window focused");
\r
431 $focus = focus_after('focus left');
\r
432 is($focus, $mid->id, "Middle window focused");</tt></pre>
\r
434 <div class="paragraph"><p>Now, we run the first two real tests. They use <tt>Test::More</tt>'s <tt>is</tt> function,
\r
435 which compares two values and prints the differences if they are not the same.
\r
436 After the arguments, we supply a short comment to indicate what we are testing
\r
437 here. This makes it vastly more easy for the developer to spot which testcase
\r
438 is the problem in case one fails.</p></div>
\r
439 <div class="paragraph"><p>The first test checks that the most recently opened window is focused.
\r
440 Afterwards, the command <tt>focus left</tt> is issued and it is verified that the
\r
441 middle window now has focus.</p></div>
\r
442 <div class="paragraph"><p>Note that this is not a comprehensive test of the <tt>focus</tt> command — we would
\r
443 have to test wrapping, focus when using a more complex layout, focusing the
\r
444 parent/child containers, etc. But that is not the point of this testcase.
\r
445 Instead, we just want to know if <tt>$x→input_focus</tt> corresponds with what we are
\r
446 expecting. If not, something is completely wrong with the test environment and
\r
447 this trivial test will fail.</p></div>
\r
448 <div class="listingblock">
\r
449 <div class="title">t/11-goto.t: Test that the feature does not work (yet)</div>
\r
450 <div class="content">
\r
451 <pre><tt>#####################################################################
\r
452 # Now goto a mark which does not exist
\r
453 #####################################################################
\r
455 my $random_mark = mktemp('mark.XXXXXX');
\r
457 $focus = focus_after(qq|[con_mark="$random_mark"] focus|);
\r
458 is($focus, $mid->id, "focus unchanged");</tt></pre>
\r
460 <div class="paragraph"><p>Syntax hint: The qq keyword is the interpolating quote operator. It lets you
\r
461 chose a quote character (in this case the <tt>|</tt> character, a pipe). This makes
\r
462 having double quotes in our string easy.</p></div>
\r
463 <div class="paragraph"><p>In this new major section, a random mark (mark is an identifier for a window,
\r
464 see "VIM-like marks" in the i3 User’s Guide) will be generated. Afterwards, we
\r
465 test that trying to focus that mark will not do anything. This is important: Do
\r
466 not only test that using a feature has the expected outcome, but also test that
\r
467 using it without properly initializing it does no harm. This command could for
\r
468 example have changed focus anyways (a bug) or crash i3 (obviously a bug).</p></div>
\r
469 <div class="listingblock">
\r
470 <div class="title">t/11-goto.t: Test that the feature does work</div>
\r
471 <div class="content">
\r
472 <pre><tt>cmd "mark $random_mark";
\r
474 $focus = focus_after('focus left');
\r
475 is($focus, $top->id, "Top window focused");
\r
477 $focus = focus_after(qq|[con_mark="$random_mark"] focus|);
\r
478 is($focus, $mid->id, "goto worked");</tt></pre>
\r
480 <div class="paragraph"><p>Remember: Focus was on the middle window (we verified that earlier in "Test
\r
481 assumptions"). We now mark the middle window with our randomly generated mark.
\r
482 Afterwards, we switch focus away from the middle window to be able to tell if
\r
483 focusing it via its mark will work. If the test works, the goto command seems
\r
484 to be working.</p></div>
\r
485 <div class="listingblock">
\r
486 <div class="title">t/11-goto.t: Test corner case</div>
\r
487 <div class="content">
\r
488 <pre><tt># check that we can specify multiple criteria
\r
490 $focus = focus_after('focus left');
\r
491 is($focus, $top->id, "Top window focused");
\r
493 $focus = focus_after(qq|[con_mark="$random_mark" con_mark="$random_mark"] focus|);
\r
494 is($focus, $mid->id, "goto worked");</tt></pre>
\r
496 <div class="paragraph"><p>Now we test the same feature, but specifying the mark twice in the command.
\r
497 This should have no effect, but let’s be sure: test it and see if things go
\r
499 <div class="listingblock">
\r
500 <div class="title">t/11-goto.t: Test second code path</div>
\r
501 <div class="content">
\r
502 <pre><tt>#####################################################################
\r
503 # Check whether the focus command will switch to a different
\r
504 # workspace if necessary
\r
505 #####################################################################
\r
507 my $tmp2 = fresh_workspace;
\r
509 is(focused_ws(), $tmp2, 'tmp2 now focused');
\r
511 cmd qq|[con_mark="$random_mark"] focus|;
\r
513 is(focused_ws(), $tmp, 'tmp now focused');</tt></pre>
\r
515 <div class="paragraph"><p>This part of the test checks that focusing windows by mark works across
\r
516 workspaces. It uses i3test’s <tt>focused_ws</tt> function to get the current
\r
517 workspace.</p></div>
\r
518 <div class="listingblock">
\r
519 <div class="title">t/11-goto.t: Test second code path</div>
\r
520 <div class="content">
\r
521 <pre><tt>done_testing;</tt></pre>
\r
523 <div class="paragraph"><p>The end of every testcase has to contain the <tt>done_testing</tt> line. This tells
\r
524 <tt>complete-run.pl</tt> that the test was finished successfully. If it does not
\r
525 occur, the test might have crashed during execution — some of the reasons why
\r
526 that could happen are bugs in the used modules, bugs in the testcase itself or
\r
527 an i3 crash resulting in the testcase being unable to communicate with i3 via
\r
528 IPC anymore.</p></div>
\r
531 <div class="sect1">
\r
532 <h2 id="i3_sync">5. Appendix A: The i3 sync protocol</h2>
\r
533 <div class="sectionbody">
\r
534 <div class="paragraph"><p>Consider the following situation: You open two windows in your testcase, then
\r
535 you use <tt>focus left</tt> and want to verify that the X11 focus has been updated
\r
536 properly. Sounds simple, right? Let’s assume you use this straight-forward
\r
537 implementation:</p></div>
\r
538 <div class="listingblock">
\r
539 <div class="title">Racey focus testcase</div>
\r
540 <div class="content">
\r
541 <pre><tt>my $left = open_window($x);
\r
542 my $right = open_window($x);
\r
544 is($x->input_focus, $left->id, 'left window focused');</tt></pre>
\r
546 <div class="paragraph"><p>However, the test fails. Sometimes. Apparently, there is a race condition in
\r
547 your test. If you think about it, this is because you are using two different
\r
548 pieces of software: You tell i3 to update focus, i3 confirms that, and then you
\r
549 ask X11 to give you the current focus. There is a certain time i3 needs to
\r
550 update the X11 state. If the testcase gets CPU time before X11 processed i3’s
\r
551 requests, the test will fail.</p></div>
\r
552 <div class="imageblock">
\r
553 <div class="content">
\r
554 <img src="i3-sync.png" alt="Diagram of the race condition" />
\r
556 <div class="title">Figure 1. Diagram of the race condition</div>
\r
558 <div class="paragraph"><p>One way to "solve" this would be to add <tt>sleep 0.5;</tt> after the <tt>cmd</tt> call.
\r
559 After 0.5 seconds it should be safe to assume that focus has been updated,
\r
561 <div class="paragraph"><p>In practice, this usually works. However, it has several problems:</p></div>
\r
562 <div class="olist arabic"><ol class="arabic">
\r
565 This is obviously not a clean solution, but a workaround. Ugly.
\r
570 On very slow machines, this might not work. Unlikely, but in different
\r
571 situations (a delay to wait for i3 to startup) the necessary time is much
\r
572 harder to guess, even for fast machines.
\r
577 This <strong>wastes a lot of time</strong>. Usually, your computer is much faster than 0.5s
\r
578 to update the status. However, sometimes, it might take 0.4s, so we can’t
\r
579 make it <tt>sleep 0.1</tt>.
\r
583 <div class="paragraph"><p>To illustrate how grave the problem with wasting time actually is: Before
\r
584 removing all sleeps from the testsuite, a typical run using 4 separate X
\r
585 servers took around 50 seconds on my machine. After removing all the sleeps,
\r
586 we achieved times of about 25 seconds. This is very significant and influences
\r
587 the way you think about tests — the faster they are, the more likely you are
\r
588 to check whether everything still works quite often (which you should).</p></div>
\r
589 <div class="paragraph"><p>What I am trying to say is: Delays adds up quickly and make the test suite
\r
590 less robust.</p></div>
\r
591 <div class="paragraph"><p>The real solution for this problem is a mechanism which I call "the i3 sync
\r
592 protocol". The idea is to send a request (which does not modify state) via X11
\r
593 to i3 which will then be answered. Due to the request’s position in the event
\r
594 queue (<strong>after</strong> all previous events), you can be sure that by the time you
\r
595 receive the reply, all other events have been dealt with by i3 (and, more
\r
596 importantly, X11).</p></div>
\r
597 <div class="imageblock">
\r
598 <div class="content">
\r
599 <img src="i3-sync-working.png" alt="Diagram of the i3 sync solution" />
\r
601 <div class="title">Figure 2. Diagram of the i3 sync solution</div>
\r
603 <div class="sect2">
\r
604 <h3 id="_implementation_details">5.1. Implementation details</h3>
\r
605 <div class="paragraph"><p>The client which wants to sync with i3 initiates the protocol by sending a
\r
606 ClientMessage to the X11 root window:</p></div>
\r
607 <div class="listingblock">
\r
608 <div class="title">Send ClientMessage</div>
\r
609 <div class="content">
\r
610 <pre><tt># Generate a ClientMessage, see xcb_client_message_t
\r
611 my $msg = pack "CCSLLLLLLL",
\r
612 CLIENT_MESSAGE, # response_type
\r
615 $root, # destination window
\r
616 $x->atom(name => 'I3_SYNC')->id,
\r
618 $_sync_window->id, # data[0]: our own window id
\r
619 $myrnd, # data[1]: a random value to identify the request
\r
624 # Send it to the root window -- since i3 uses the SubstructureRedirect
\r
625 # event mask, it will get the ClientMessage.
\r
626 $x->send_event(0, $root, EVENT_MASK_SUBSTRUCTURE_REDIRECT, $msg);</tt></pre>
\r
628 <div class="paragraph"><p>i3 will then reply with the same ClientMessage, sent to the window specified in
\r
629 <tt>data[0]</tt>. In the reply, <tt>data[0]</tt> and <tt>data[1]</tt> are exactly the same as in the
\r
630 request. You should use a random value in <tt>data[1]</tt> and check that you received
\r
631 the same one when getting the reply.</p></div>
\r
635 <div class="sect1">
\r
636 <h2 id="_appendix_b_socket_activation">6. Appendix B: Socket activation</h2>
\r
637 <div class="sectionbody">
\r
638 <div class="paragraph"><p>Socket activation is a mechanism which was made popular by systemd, an init
\r
639 replacement. It basically describes creating a listening socket before starting
\r
640 a program. systemd will invoke the program only when an actual connection to
\r
641 the socket is made, hence the term socket activation.</p></div>
\r
642 <div class="paragraph"><p>The interesting part of this (in the i3 context) is that you can very precisely
\r
643 detect when the program is ready (finished its initialization).</p></div>
\r
644 <div class="sect2">
\r
645 <h3 id="_preparing_the_listening_socket">6.1. Preparing the listening socket</h3>
\r
646 <div class="paragraph"><p><tt>complete-run.pl</tt> will create a listening UNIX socket which it will then pass
\r
647 to i3. This socket will be used by i3 as an additional IPC socket, just like
\r
648 the one it will create on its own. Passing the socket happens implicitly
\r
649 because children will inherit the parent’s sockets when fork()ing and sockets
\r
650 will continue to exist after an exec() call (unless CLOEXEC is set of course).</p></div>
\r
651 <div class="paragraph"><p>The only explicit things <tt>complete-run.pl</tt> has to do is setting the <tt>LISTEN_FDS</tt>
\r
652 environment variable to the number of sockets which exist (1 in our case) and
\r
653 setting the <tt>LISTEN_PID</tt> environment variable to the current process ID. Both
\r
654 variables are necessary so that the program (i3) knows how many sockets it
\r
655 should use and if the environment variable is actually intended for it. i3 will
\r
656 then start looking for sockets at file descriptor 3 (since 0, 1 and 2 are used
\r
657 for stdin, stdout and stderr, respectively).</p></div>
\r
658 <div class="paragraph"><p>The actual Perl code which sets up the socket, fork()s, makes sure the socket
\r
659 has file descriptor 3 and sets up the environment variables follows (shortened
\r
661 <div class="listingblock">
\r
662 <div class="title">Setup socket and environment</div>
\r
663 <div class="content">
\r
664 <pre><tt>my $socket = IO::Socket::UNIX->new(
\r
666 Local => $args{unix_socket_path},
\r
671 $ENV{LISTEN_PID} = $$;
\r
672 $ENV{LISTEN_FDS} = 1;
\r
674 # Only pass file descriptors 0 (stdin), 1 (stdout),
\r
675 # 2 (stderr) and 3 (socket) to the child.
\r
678 # If the socket does not use file descriptor 3 by chance
\r
679 # already, we close fd 3 and dup2() the socket to 3.
\r
680 if (fileno($socket) != 3) {
\r
682 POSIX::dup2(fileno($socket), 3);
\r
685 exec "/usr/bin/i3";
\r
689 <div class="sect2">
\r
690 <h3 id="_waiting_for_a_reply">6.2. Waiting for a reply</h3>
\r
691 <div class="paragraph"><p>In the parent process, we want to know when i3 is ready to answer our IPC
\r
692 requests and handle our windows. Therefore, after forking, we immediately close
\r
693 the listening socket (i3 will handle this side of the socket) and connect to it
\r
694 (remember, we are talking about a named UNIX socket) as a client. This connect
\r
695 call will immediately succeed because the kernel buffers it. Then, we send a
\r
696 request (of type GET_TREE, but that is not really relevant). Writing data to
\r
697 the socket will also succeed immediately because, again, the kernel buffers it
\r
698 (only up to a certain amount of data of course).</p></div>
\r
699 <div class="paragraph"><p>Afterwards, we just blockingly wait until we get an answer. In the child
\r
700 process, i3 will setup the listening socket in its event loop. Immediately
\r
701 after actually starting the event loop, it will notice a new client connecting
\r
702 (the parent process) and handle its request. Since all initialization has been
\r
703 completed successfully by the time the event loop is entered, we can now assume
\r
704 that i3 is ready.</p></div>
\r
706 <div class="sect2">
\r
707 <h3 id="_timing_and_conclusion">6.3. Timing and conclusion</h3>
\r
708 <div class="paragraph"><p>A beautiful feature of this mechanism is that it does not depend on timing. It
\r
709 does not matter when the child process gets CPU time or when the parent process
\r
710 gets CPU time. On heavily loaded machines (or machines with multiple CPUs,
\r
711 cores or unreliable schedulers), this makes waiting for i3 much more robust.</p></div>
\r
712 <div class="paragraph"><p>Before using socket activation, we typically used a <tt>sleep(1)</tt> and hoped that
\r
713 i3 was initialized by that time. Of course, this breaks on some (slow)
\r
714 computers and wastes a lot of time on faster computers. By using socket
\r
715 activation, we decreased the total amount of time necessary to run all tests
\r
716 (72 files at the time of writing) from > 100 seconds to 16 seconds. This makes
\r
717 it significantly more attractive to run the test suite more often (or at all)
\r
718 during development.</p></div>
\r
719 <div class="paragraph"><p>An alternative approach to using socket activation is polling for the existence
\r
720 of the IPC socket and connecting to it. While this might be slightly easier to
\r
721 implement, it wastes CPU time and is considerably uglier than this solution
\r
722 :). After all, <tt>lib/SocketActivation.pm</tt> contains only 54 SLOC.</p></div>
\r
727 <div id="footnotes"><hr /></div>
\r
728 <div id="footer" lang="de">
\r
729 © 2009-2011 Michael Stapelberg, <a href="/impress.html">Impressum</a>
\r
projects / i3 / i3.github.io / blob