diff options
author | Graham Christensen <graham@grahamc.com> | 2018-05-01 19:57:09 -0400 |
---|---|---|
committer | Graham Christensen <graham@grahamc.com> | 2018-05-01 19:57:09 -0400 |
commit | eca5c99bf8a115ffd9513f91decc064a5bb3ff6d (patch) | |
tree | 7b49bc123be12ca5344428c6975e4487e69d55e3 /nixos/doc/manual/development/writing-nixos-tests.xml | |
parent | 77161de4546697f9bf2da6d081eeba4c399b3313 (diff) | |
download | nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.gz nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.bz2 nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.lz nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.xz nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.zst nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.zip |
nixos docs: format =)
Diffstat (limited to 'nixos/doc/manual/development/writing-nixos-tests.xml')
-rw-r--r-- | nixos/doc/manual/development/writing-nixos-tests.xml | 590 |
1 files changed, 338 insertions, 252 deletions
diff --git a/nixos/doc/manual/development/writing-nixos-tests.xml b/nixos/doc/manual/development/writing-nixos-tests.xml index a8f6aa00858e..89a6a4423627 100644 --- a/nixos/doc/manual/development/writing-nixos-tests.xml +++ b/nixos/doc/manual/development/writing-nixos-tests.xml @@ -3,11 +3,10 @@ xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-writing-nixos-tests"> + <title>Writing Tests</title> -<title>Writing Tests</title> - -<para>A NixOS test is a Nix expression that has the following structure: - + <para> + A NixOS test is a Nix expression that has the following structure: <programlisting> import ./make-test.nix { @@ -32,277 +31,364 @@ import ./make-test.nix { ''; } </programlisting> - -The attribute <literal>testScript</literal> is a bit of Perl code that -executes the test (described below). During the test, it will start -one or more virtual machines, the configuration of which is described -by the attribute <literal>machine</literal> (if you need only one -machine in your test) or by the attribute <literal>nodes</literal> (if -you need multiple machines). For instance, <filename + The attribute <literal>testScript</literal> is a bit of Perl code that + executes the test (described below). During the test, it will start one or + more virtual machines, the configuration of which is described by the + attribute <literal>machine</literal> (if you need only one machine in your + test) or by the attribute <literal>nodes</literal> (if you need multiple + machines). For instance, + <filename xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename> -only needs a single machine to test whether users can log in on the -virtual console, whether device ownership is correctly maintained when -switching between consoles, and so on. On the other hand, <filename + only needs a single machine to test whether users can log in on the virtual + console, whether device ownership is correctly maintained when switching + between consoles, and so on. On the other hand, + <filename xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>, -which tests NFS client and server functionality in the Linux kernel -(including whether locks are maintained across server crashes), -requires three machines: a server and two clients.</para> - -<para>There are a few special NixOS configuration options for test -VMs: + which tests NFS client and server functionality in the Linux kernel + (including whether locks are maintained across server crashes), requires + three machines: a server and two clients. + </para> + <para> + There are a few special NixOS configuration options for test VMs: <!-- FIXME: would be nice to generate this automatically. --> - -<variablelist> - - <varlistentry> - <term><option>virtualisation.memorySize</option></term> - <listitem><para>The memory of the VM in - megabytes.</para></listitem> - </varlistentry> - - <varlistentry> - <term><option>virtualisation.vlans</option></term> - <listitem><para>The virtual networks to which the VM is - connected. See <filename + <variablelist> + <varlistentry> + <term><option>virtualisation.memorySize</option> + </term> + <listitem> + <para> + The memory of the VM in megabytes. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><option>virtualisation.vlans</option> + </term> + <listitem> + <para> + The virtual networks to which the VM is connected. See + <filename xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename> - for an example.</para></listitem> - </varlistentry> - - <varlistentry> - <term><option>virtualisation.writableStore</option></term> - <listitem><para>By default, the Nix store in the VM is not - writable. If you enable this option, a writable union file system - is mounted on top of the Nix store to make it appear - writable. This is necessary for tests that run Nix operations that - modify the store.</para></listitem> - </varlistentry> - -</variablelist> - -For more options, see the module <filename -xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>.</para> - -<para>The test script is a sequence of Perl statements that perform -various actions, such as starting VMs, executing commands in the VMs, -and so on. Each virtual machine is represented as an object stored in -the variable <literal>$<replaceable>name</replaceable></literal>, -where <replaceable>name</replaceable> is the identifier of the machine -(which is just <literal>machine</literal> if you didn’t specify -multiple machines using the <literal>nodes</literal> attribute). For -instance, the following starts the machine, waits until it has -finished booting, then executes a command and checks that the output -is more-or-less correct: - + for an example. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><option>virtualisation.writableStore</option> + </term> + <listitem> + <para> + By default, the Nix store in the VM is not writable. If you enable this + option, a writable union file system is mounted on top of the Nix store + to make it appear writable. This is necessary for tests that run Nix + operations that modify the store. + </para> + </listitem> + </varlistentry> + </variablelist> + For more options, see the module + <filename +xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>. + </para> + + <para> + The test script is a sequence of Perl statements that perform various + actions, such as starting VMs, executing commands in the VMs, and so on. Each + virtual machine is represented as an object stored in the variable + <literal>$<replaceable>name</replaceable></literal>, where + <replaceable>name</replaceable> is the identifier of the machine (which is + just <literal>machine</literal> if you didn’t specify multiple machines + using the <literal>nodes</literal> attribute). For instance, the following + starts the machine, waits until it has finished booting, then executes a + command and checks that the output is more-or-less correct: <programlisting> $machine->start; $machine->waitForUnit("default.target"); $machine->succeed("uname") =~ /Linux/; </programlisting> - -The first line is actually unnecessary; machines are implicitly -started when you first execute an action on them (such as -<literal>waitForUnit</literal> or <literal>succeed</literal>). If you -have multiple machines, you can speed up the test by starting them in -parallel: - + The first line is actually unnecessary; machines are implicitly started when + you first execute an action on them (such as <literal>waitForUnit</literal> + or <literal>succeed</literal>). If you have multiple machines, you can speed + up the test by starting them in parallel: <programlisting> startAll; </programlisting> - -</para> - -<para>The following methods are available on machine objects: - -<variablelist> - - <varlistentry> - <term><methodname>start</methodname></term> - <listitem><para>Start the virtual machine. This method is - asynchronous — it does not wait for the machine to finish - booting.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>shutdown</methodname></term> - <listitem><para>Shut down the machine, waiting for the VM to - exit.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>crash</methodname></term> - <listitem><para>Simulate a sudden power failure, by telling the VM - to exit immediately.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>block</methodname></term> - <listitem><para>Simulate unplugging the Ethernet cable that - connects the machine to the other machines.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>unblock</methodname></term> - <listitem><para>Undo the effect of - <methodname>block</methodname>.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>screenshot</methodname></term> - <listitem><para>Take a picture of the display of the virtual - machine, in PNG format. The screenshot is linked from the HTML - log.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>getScreenText</methodname></term> - <listitem><para>Return a textual representation of what is currently - visible on the machine's screen using optical character - recognition.</para> - <note><para>This requires passing <option>enableOCR</option> to the test - attribute set.</para></note></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>sendMonitorCommand</methodname></term> - <listitem><para>Send a command to the QEMU monitor. This is rarely - used, but allows doing stuff such as attaching virtual USB disks - to a running machine.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>sendKeys</methodname></term> - <listitem><para>Simulate pressing keys on the virtual keyboard, - e.g., <literal>sendKeys("ctrl-alt-delete")</literal>.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>sendChars</methodname></term> - <listitem><para>Simulate typing a sequence of characters on the - virtual keyboard, e.g., <literal>sendKeys("foobar\n")</literal> - will type the string <literal>foobar</literal> followed by the - Enter key.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>execute</methodname></term> - <listitem><para>Execute a shell command, returning a list - <literal>(<replaceable>status</replaceable>, - <replaceable>stdout</replaceable>)</literal>.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>succeed</methodname></term> - <listitem><para>Execute a shell command, raising an exception if - the exit status is not zero, otherwise returning the standard - output.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>fail</methodname></term> - <listitem><para>Like <methodname>succeed</methodname>, but raising - an exception if the command returns a zero status.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitUntilSucceeds</methodname></term> - <listitem><para>Repeat a shell command with 1-second intervals - until it succeeds.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitUntilFails</methodname></term> - <listitem><para>Repeat a shell command with 1-second intervals - until it fails.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForUnit</methodname></term> - <listitem><para>Wait until the specified systemd unit has reached - the “active” state.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForFile</methodname></term> - <listitem><para>Wait until the specified file - exists.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForOpenPort</methodname></term> - <listitem><para>Wait until a process is listening on the given TCP - port (on <literal>localhost</literal>, at least).</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForClosedPort</methodname></term> - <listitem><para>Wait until nobody is listening on the given TCP - port.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForX</methodname></term> - <listitem><para>Wait until the X11 server is accepting - connections.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForText</methodname></term> - <listitem><para>Wait until the supplied regular expressions matches - the textual contents of the screen by using optical character recognition - (see <methodname>getScreenText</methodname>).</para> - <note><para>This requires passing <option>enableOCR</option> to the test - attribute set.</para></note></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>waitForWindow</methodname></term> - <listitem><para>Wait until an X11 window has appeared whose name - matches the given regular expression, e.g., - <literal>waitForWindow(qr/Terminal/)</literal>.</para></listitem> - </varlistentry> - - <varlistentry> - <term><methodname>copyFileFromHost</methodname></term> - <listitem><para>Copies a file from host to machine, e.g., - <literal>copyFileFromHost("myfile", "/etc/my/important/file")</literal>.</para> - <para>The first argument is the file on the host. The file needs to be - accessible while building the nix derivation. The second argument is - the location of the file on the machine.</para> + </para> + + <para> + The following methods are available on machine objects: + <variablelist> + <varlistentry> + <term><methodname>start</methodname> + </term> + <listitem> + <para> + Start the virtual machine. This method is asynchronous — it does not + wait for the machine to finish booting. + </para> </listitem> - </varlistentry> - - <varlistentry> - <term><methodname>systemctl</methodname></term> + </varlistentry> + <varlistentry> + <term><methodname>shutdown</methodname> + </term> + <listitem> + <para> + Shut down the machine, waiting for the VM to exit. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>crash</methodname> + </term> <listitem> - <para>Runs <literal>systemctl</literal> commands with optional support for - <literal>systemctl --user</literal></para> - <para> - <programlisting> + <para> + Simulate a sudden power failure, by telling the VM to exit immediately. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>block</methodname> + </term> + <listitem> + <para> + Simulate unplugging the Ethernet cable that connects the machine to the + other machines. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>unblock</methodname> + </term> + <listitem> + <para> + Undo the effect of <methodname>block</methodname>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>screenshot</methodname> + </term> + <listitem> + <para> + Take a picture of the display of the virtual machine, in PNG format. The + screenshot is linked from the HTML log. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>getScreenText</methodname> + </term> + <listitem> + <para> + Return a textual representation of what is currently visible on the + machine's screen using optical character recognition. + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>sendMonitorCommand</methodname> + </term> + <listitem> + <para> + Send a command to the QEMU monitor. This is rarely used, but allows doing + stuff such as attaching virtual USB disks to a running machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>sendKeys</methodname> + </term> + <listitem> + <para> + Simulate pressing keys on the virtual keyboard, e.g., + <literal>sendKeys("ctrl-alt-delete")</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>sendChars</methodname> + </term> + <listitem> + <para> + Simulate typing a sequence of characters on the virtual keyboard, e.g., + <literal>sendKeys("foobar\n")</literal> will type the string + <literal>foobar</literal> followed by the Enter key. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>execute</methodname> + </term> + <listitem> + <para> + Execute a shell command, returning a list + <literal>(<replaceable>status</replaceable>, + <replaceable>stdout</replaceable>)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>succeed</methodname> + </term> + <listitem> + <para> + Execute a shell command, raising an exception if the exit status is not + zero, otherwise returning the standard output. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>fail</methodname> + </term> + <listitem> + <para> + Like <methodname>succeed</methodname>, but raising an exception if the + command returns a zero status. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitUntilSucceeds</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it succeeds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitUntilFails</methodname> + </term> + <listitem> + <para> + Repeat a shell command with 1-second intervals until it fails. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForUnit</methodname> + </term> + <listitem> + <para> + Wait until the specified systemd unit has reached the “active” state. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForFile</methodname> + </term> + <listitem> + <para> + Wait until the specified file exists. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForOpenPort</methodname> + </term> + <listitem> + <para> + Wait until a process is listening on the given TCP port (on + <literal>localhost</literal>, at least). + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForClosedPort</methodname> + </term> + <listitem> + <para> + Wait until nobody is listening on the given TCP port. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForX</methodname> + </term> + <listitem> + <para> + Wait until the X11 server is accepting connections. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForText</methodname> + </term> + <listitem> + <para> + Wait until the supplied regular expressions matches the textual contents + of the screen by using optical character recognition (see + <methodname>getScreenText</methodname>). + </para> + <note> + <para> + This requires passing <option>enableOCR</option> to the test attribute + set. + </para> + </note> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>waitForWindow</methodname> + </term> + <listitem> + <para> + Wait until an X11 window has appeared whose name matches the given + regular expression, e.g., <literal>waitForWindow(qr/Terminal/)</literal>. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>copyFileFromHost</methodname> + </term> + <listitem> + <para> + Copies a file from host to machine, e.g., + <literal>copyFileFromHost("myfile", "/etc/my/important/file")</literal>. + </para> + <para> + The first argument is the file on the host. The file needs to be + accessible while building the nix derivation. The second argument is the + location of the file on the machine. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term><methodname>systemctl</methodname> + </term> + <listitem> + <para> + Runs <literal>systemctl</literal> commands with optional support for + <literal>systemctl --user</literal> + </para> + <para> +<programlisting> $machine->systemctl("list-jobs --no-pager"); // runs `systemctl list-jobs --no-pager` $machine->systemctl("list-jobs --no-pager", "any-user"); // spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager` </programlisting> - </para> + </para> </listitem> - </varlistentry> + </varlistentry> + </variablelist> + </para> -</variablelist> - -</para> - -<para> - To test user units declared by <literal>systemd.user.services</literal> the optional <literal>$user</literal> - argument can be used: - - <programlisting> + <para> + To test user units declared by <literal>systemd.user.services</literal> the + optional <literal>$user</literal> argument can be used: +<programlisting> $machine->start; $machine->waitForX; $machine->waitForUnit("xautolock.service", "x-session-user"); </programlisting> - This applies to <literal>systemctl</literal>, <literal>getUnitInfo</literal>, - <literal>waitForUnit</literal>, <literal>startJob</literal> - and <literal>stopJob</literal>. -</para> - + <literal>waitForUnit</literal>, <literal>startJob</literal> and + <literal>stopJob</literal>. + </para> </section> |