diff options
Diffstat (limited to 'nixos/doc/manual/development/writing-nixos-tests.xml')
| -rw-r--r-- | nixos/doc/manual/development/writing-nixos-tests.xml | 251 |
1 files changed, 251 insertions, 0 deletions
diff --git a/nixos/doc/manual/development/writing-nixos-tests.xml b/nixos/doc/manual/development/writing-nixos-tests.xml new file mode 100644 index 000000000000..bbb655eed2a6 --- /dev/null +++ b/nixos/doc/manual/development/writing-nixos-tests.xml @@ -0,0 +1,251 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-writing-nixos-tests"> + +<title>Writing Tests</title> + +<para>A NixOS test is a Nix expression that has the following structure: + +<programlisting> +import ./make-test.nix { + + # Either the configuration of a single machine: + machine = + { config, pkgs, ... }: + { <replaceable>configuration…</replaceable> + }; + + # Or a set of machines: + nodes = + { <replaceable>machine1</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + <replaceable>machine2</replaceable> = + { config, pkgs, ... }: { <replaceable>…</replaceable> }; + … + }; + + testScript = + '' + <replaceable>Perl code…</replaceable> + ''; +} +</programlisting> + +The attribute <literal>testScript</literal> is a bit of Perl code that +executes the test (described below). During the test, it will start +one or more virtual machines, the configuration of which is described +by the attribute <literal>machine</literal> (if you need only one +machine in your test) or by the attribute <literal>nodes</literal> (if +you need multiple machines). For instance, <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> +only needs a single machine to test whether users can log in on the +virtual console, whether device ownership is correctly maintained when +switching between consoles, and so on. On the other hand, <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>, +which tests NFS client and server functionality in the Linux kernel +(including whether locks are maintained across server crashes), +requires three machines: a server and two clients.</para> + +<para>There are a few special NixOS configuration options for test +VMs: + +<!-- FIXME: would be nice to generate this automatically. --> + +<variablelist> + + <varlistentry> + <term><option>virtualisation.memorySize</option></term> + <listitem><para>The memory of the VM in + megabytes.</para></listitem> + </varlistentry> + + <varlistentry> + <term><option>virtualisation.vlans</option></term> + <listitem><para>The virtual networks to which the VM is + connected. See <filename + xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> + for an example.</para></listitem> + </varlistentry> + + <varlistentry> + <term><option>virtualisation.writableStore</option></term> + <listitem><para>By default, the Nix store in the VM is not + writable. If you enable this option, a writable union file system + is mounted on top of the Nix store to make it appear + writable. This is necessary for tests that run Nix operations that + modify the store.</para></listitem> + </varlistentry> + +</variablelist> + +For more options, see the module <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>.</para> + +<para>The test script is a sequence of Perl statements that perform +various actions, such as starting VMs, executing commands in the VMs, +and so on. Each virtual machine is represented as an object stored in +the variable <literal>$<replaceable>name</replaceable></literal>, +where <replaceable>name</replaceable> is the identifier of the machine +(which is just <literal>machine</literal> if you didn’t specify +multiple machines using the <literal>nodes</literal> attribute). For +instance, the following starts the machine, waits until it has +finished booting, then executes a command and checks that the output +is more-or-less correct: + +<programlisting> +$machine->start; +$machine->waitForUnit("default.target"); +$machine->succeed("uname") =~ /Linux/; +</programlisting> + +The first line is actually unnecessary; machines are implicitly +started when you first execute an action on them (such as +<literal>waitForUnit</literal> or <literal>succeed</literal>). If you +have multiple machines, you can speed up the test by starting them in +parallel: + +<programlisting> +startAll; +</programlisting> + +</para> + +<para>The following methods are available on machine objects: + +<variablelist> + + <varlistentry> + <term><methodname>start</methodname></term> + <listitem><para>Start the virtual machine. This method is + asynchronous — it does not wait for the machine to finish + booting.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>shutdown</methodname></term> + <listitem><para>Shut down the machine, waiting for the VM to + exit.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>crash</methodname></term> + <listitem><para>Simulate a sudden power failure, by telling the VM + to exit immediately.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>block</methodname></term> + <listitem><para>Simulate unplugging the Ethernet cable that + connects the machine to the other machines.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>unblock</methodname></term> + <listitem><para>Undo the effect of + <methodname>block</methodname>.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>screenshot</methodname></term> + <listitem><para>Take a picture of the display of the virtual + machine, in PNG format. The screenshot is linked from the HTML + log.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>sendMonitorCommand</methodname></term> + <listitem><para>Send a command to the QEMU monitor. This is rarely + used, but allows doing stuff such as attaching virtual USB disks + to a running machine.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>sendKeys</methodname></term> + <listitem><para>Simulate pressing keys on the virtual keyboard, + e.g., <literal>sendKeys("ctrl-alt-delete")</literal>.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>sendChars</methodname></term> + <listitem><para>Simulate typing a sequence of characters on the + virtual keyboard, e.g., <literal>sendKeys("foobar\n")</literal> + will type the string <literal>foobar</literal> followed by the + Enter key.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>execute</methodname></term> + <listitem><para>Execute a shell command, returning a list + <literal>(<replaceable>status</replaceable>, + <replaceable>stdout</replaceable>)</literal>.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>succeed</methodname></term> + <listitem><para>Execute a shell command, raising an exception if + the exit status is not zero, otherwise returning the standard + output.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>fail</methodname></term> + <listitem><para>Like <methodname>succeed</methodname>, but raising + an exception if the command returns a zero status.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitUntilSucceeds</methodname></term> + <listitem><para>Repeat a shell command with 1-second intervals + until it succeeds.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitUntilFails</methodname></term> + <listitem><para>Repeat a shell command with 1-second intervals + until it fails.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForUnit</methodname></term> + <listitem><para>Wait until the specified systemd unit has reached + the “active” state.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForFile</methodname></term> + <listitem><para>Wait until the specified file + exists.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForOpenPort</methodname></term> + <listitem><para>Wait until a process is listening on the given TCP + port (on <literal>localhost</literal>, at least).</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForClosedPort</methodname></term> + <listitem><para>Wait until nobody is listening on the given TCP + port.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForX</methodname></term> + <listitem><para>Wait until the X11 server is accepting + connections.</para></listitem> + </varlistentry> + + <varlistentry> + <term><methodname>waitForWindow</methodname></term> + <listitem><para>Wait until an X11 window has appeared whose name + matches the given regular expression, e.g., + <literal>waitForWindow(qr/Terminal/)</literal>.</para></listitem> + </varlistentry> + +</variablelist> + +</para> + +</section> \ No newline at end of file |