about summary refs log tree commit diff
path: root/nixos/doc/manual/development/writing-nixos-tests.xml
diff options
context:
space:
mode:
authorGraham Christensen <graham@grahamc.com>2018-05-01 19:57:09 -0400
committerGraham Christensen <graham@grahamc.com>2018-05-01 19:57:09 -0400
commiteca5c99bf8a115ffd9513f91decc064a5bb3ff6d (patch)
tree7b49bc123be12ca5344428c6975e4487e69d55e3 /nixos/doc/manual/development/writing-nixos-tests.xml
parent77161de4546697f9bf2da6d081eeba4c399b3313 (diff)
downloadnixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.gz
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.bz2
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.lz
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.xz
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.tar.zst
nixlib-eca5c99bf8a115ffd9513f91decc064a5bb3ff6d.zip
nixos docs: format =)
Diffstat (limited to 'nixos/doc/manual/development/writing-nixos-tests.xml')
-rw-r--r--nixos/doc/manual/development/writing-nixos-tests.xml590
1 files changed, 338 insertions, 252 deletions
diff --git a/nixos/doc/manual/development/writing-nixos-tests.xml b/nixos/doc/manual/development/writing-nixos-tests.xml
index a8f6aa00858e..89a6a4423627 100644
--- a/nixos/doc/manual/development/writing-nixos-tests.xml
+++ b/nixos/doc/manual/development/writing-nixos-tests.xml
@@ -3,11 +3,10 @@
         xmlns:xi="http://www.w3.org/2001/XInclude"
         version="5.0"
         xml:id="sec-writing-nixos-tests">
+ <title>Writing Tests</title>
 
-<title>Writing Tests</title>
-
-<para>A NixOS test is a Nix expression that has the following structure:
-
+ <para>
+  A NixOS test is a Nix expression that has the following structure:
 <programlisting>
 import ./make-test.nix {
 
@@ -32,277 +31,364 @@ import ./make-test.nix {
     '';
 }
 </programlisting>
-
-The attribute <literal>testScript</literal> is a bit of Perl code that
-executes the test (described below). During the test, it will start
-one or more virtual machines, the configuration of which is described
-by the attribute <literal>machine</literal> (if you need only one
-machine in your test) or by the attribute <literal>nodes</literal> (if
-you need multiple machines). For instance, <filename
+  The attribute <literal>testScript</literal> is a bit of Perl code that
+  executes the test (described below). During the test, it will start one or
+  more virtual machines, the configuration of which is described by the
+  attribute <literal>machine</literal> (if you need only one machine in your
+  test) or by the attribute <literal>nodes</literal> (if you need multiple
+  machines). For instance,
+  <filename
 xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/login.nix">login.nix</filename>
-only needs a single machine to test whether users can log in on the
-virtual console, whether device ownership is correctly maintained when
-switching between consoles, and so on. On the other hand, <filename
+  only needs a single machine to test whether users can log in on the virtual
+  console, whether device ownership is correctly maintained when switching
+  between consoles, and so on. On the other hand,
+  <filename
 xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nfs.nix">nfs.nix</filename>,
-which tests NFS client and server functionality in the Linux kernel
-(including whether locks are maintained across server crashes),
-requires three machines: a server and two clients.</para>
-
-<para>There are a few special NixOS configuration options for test
-VMs:
+  which tests NFS client and server functionality in the Linux kernel
+  (including whether locks are maintained across server crashes), requires
+  three machines: a server and two clients.
+ </para>
 
+ <para>
+  There are a few special NixOS configuration options for test VMs:
 <!-- FIXME: would be nice to generate this automatically. -->
-
-<variablelist>
-
-  <varlistentry>
-    <term><option>virtualisation.memorySize</option></term>
-    <listitem><para>The memory of the VM in
-    megabytes.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><option>virtualisation.vlans</option></term>
-    <listitem><para>The virtual networks to which the VM is
-    connected. See <filename
+  <variablelist>
+   <varlistentry>
+    <term><option>virtualisation.memorySize</option>
+    </term>
+    <listitem>
+     <para>
+      The memory of the VM in megabytes.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><option>virtualisation.vlans</option>
+    </term>
+    <listitem>
+     <para>
+      The virtual networks to which the VM is connected. See
+      <filename
     xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/nat.nix">nat.nix</filename>
-    for an example.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><option>virtualisation.writableStore</option></term>
-    <listitem><para>By default, the Nix store in the VM is not
-    writable. If you enable this option, a writable union file system
-    is mounted on top of the Nix store to make it appear
-    writable. This is necessary for tests that run Nix operations that
-    modify the store.</para></listitem>
-  </varlistentry>
-
-</variablelist>
-
-For more options, see the module <filename
-xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>.</para>
-
-<para>The test script is a sequence of Perl statements that perform
-various actions, such as starting VMs, executing commands in the VMs,
-and so on. Each virtual machine is represented as an object stored in
-the variable <literal>$<replaceable>name</replaceable></literal>,
-where <replaceable>name</replaceable> is the identifier of the machine
-(which is just <literal>machine</literal> if you didn’t specify
-multiple machines using the <literal>nodes</literal> attribute). For
-instance, the following starts the machine, waits until it has
-finished booting, then executes a command and checks that the output
-is more-or-less correct:
-
+      for an example.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><option>virtualisation.writableStore</option>
+    </term>
+    <listitem>
+     <para>
+      By default, the Nix store in the VM is not writable. If you enable this
+      option, a writable union file system is mounted on top of the Nix store
+      to make it appear writable. This is necessary for tests that run Nix
+      operations that modify the store.
+     </para>
+    </listitem>
+   </varlistentry>
+  </variablelist>
+  For more options, see the module
+  <filename
+xlink:href="https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/virtualisation/qemu-vm.nix">qemu-vm.nix</filename>.
+ </para>
+
+ <para>
+  The test script is a sequence of Perl statements that perform various
+  actions, such as starting VMs, executing commands in the VMs, and so on. Each
+  virtual machine is represented as an object stored in the variable
+  <literal>$<replaceable>name</replaceable></literal>, where
+  <replaceable>name</replaceable> is the identifier of the machine (which is
+  just <literal>machine</literal> if you didn’t specify multiple machines
+  using the <literal>nodes</literal> attribute). For instance, the following
+  starts the machine, waits until it has finished booting, then executes a
+  command and checks that the output is more-or-less correct:
 <programlisting>
 $machine->start;
 $machine->waitForUnit("default.target");
 $machine->succeed("uname") =~ /Linux/;
 </programlisting>
-
-The first line is actually unnecessary; machines are implicitly
-started when you first execute an action on them (such as
-<literal>waitForUnit</literal> or <literal>succeed</literal>). If you
-have multiple machines, you can speed up the test by starting them in
-parallel:
-
+  The first line is actually unnecessary; machines are implicitly started when
+  you first execute an action on them (such as <literal>waitForUnit</literal>
+  or <literal>succeed</literal>). If you have multiple machines, you can speed
+  up the test by starting them in parallel:
 <programlisting>
 startAll;
 </programlisting>
-
-</para>
-
-<para>The following methods are available on machine objects:
-
-<variablelist>
-
-  <varlistentry>
-    <term><methodname>start</methodname></term>
-    <listitem><para>Start the virtual machine. This method is
-    asynchronous — it does not wait for the machine to finish
-    booting.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>shutdown</methodname></term>
-    <listitem><para>Shut down the machine, waiting for the VM to
-    exit.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>crash</methodname></term>
-    <listitem><para>Simulate a sudden power failure, by telling the VM
-    to exit immediately.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>block</methodname></term>
-    <listitem><para>Simulate unplugging the Ethernet cable that
-    connects the machine to the other machines.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>unblock</methodname></term>
-    <listitem><para>Undo the effect of
-    <methodname>block</methodname>.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>screenshot</methodname></term>
-    <listitem><para>Take a picture of the display of the virtual
-    machine, in PNG format. The screenshot is linked from the HTML
-    log.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>getScreenText</methodname></term>
-    <listitem><para>Return a textual representation of what is currently
-    visible on the machine's screen using optical character
-    recognition.</para>
-    <note><para>This requires passing <option>enableOCR</option> to the test
-    attribute set.</para></note></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>sendMonitorCommand</methodname></term>
-    <listitem><para>Send a command to the QEMU monitor. This is rarely
-    used, but allows doing stuff such as attaching virtual USB disks
-    to a running machine.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>sendKeys</methodname></term>
-    <listitem><para>Simulate pressing keys on the virtual keyboard,
-    e.g., <literal>sendKeys("ctrl-alt-delete")</literal>.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>sendChars</methodname></term>
-    <listitem><para>Simulate typing a sequence of characters on the
-    virtual keyboard, e.g., <literal>sendKeys("foobar\n")</literal>
-    will type the string <literal>foobar</literal> followed by the
-    Enter key.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>execute</methodname></term>
-    <listitem><para>Execute a shell command, returning a list
-    <literal>(<replaceable>status</replaceable>,
-    <replaceable>stdout</replaceable>)</literal>.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>succeed</methodname></term>
-    <listitem><para>Execute a shell command, raising an exception if
-    the exit status is not zero, otherwise returning the standard
-    output.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>fail</methodname></term>
-    <listitem><para>Like <methodname>succeed</methodname>, but raising
-    an exception if the command returns a zero status.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitUntilSucceeds</methodname></term>
-    <listitem><para>Repeat a shell command with 1-second intervals
-    until it succeeds.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitUntilFails</methodname></term>
-    <listitem><para>Repeat a shell command with 1-second intervals
-    until it fails.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForUnit</methodname></term>
-    <listitem><para>Wait until the specified systemd unit has reached
-    the “active” state.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForFile</methodname></term>
-    <listitem><para>Wait until the specified file
-    exists.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForOpenPort</methodname></term>
-    <listitem><para>Wait until a process is listening on the given TCP
-    port (on <literal>localhost</literal>, at least).</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForClosedPort</methodname></term>
-    <listitem><para>Wait until nobody is listening on the given TCP
-    port.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForX</methodname></term>
-    <listitem><para>Wait until the X11 server is accepting
-    connections.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForText</methodname></term>
-    <listitem><para>Wait until the supplied regular expressions matches
-    the textual contents of the screen by using optical character recognition
-    (see <methodname>getScreenText</methodname>).</para>
-    <note><para>This requires passing <option>enableOCR</option> to the test
-    attribute set.</para></note></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>waitForWindow</methodname></term>
-    <listitem><para>Wait until an X11 window has appeared whose name
-    matches the given regular expression, e.g.,
-    <literal>waitForWindow(qr/Terminal/)</literal>.</para></listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>copyFileFromHost</methodname></term>
-    <listitem><para>Copies a file from host to machine, e.g.,
-    <literal>copyFileFromHost("myfile", "/etc/my/important/file")</literal>.</para>
-    <para>The first argument is the file on the host. The file needs to be
-    accessible while building the nix derivation. The second argument is
-    the location of the file on the machine.</para>
+ </para>
+
+ <para>
+  The following methods are available on machine objects:
+  <variablelist>
+   <varlistentry>
+    <term><methodname>start</methodname>
+    </term>
+    <listitem>
+     <para>
+      Start the virtual machine. This method is asynchronous — it does not
+      wait for the machine to finish booting.
+     </para>
     </listitem>
-  </varlistentry>
-
-  <varlistentry>
-    <term><methodname>systemctl</methodname></term>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>shutdown</methodname>
+    </term>
+    <listitem>
+     <para>
+      Shut down the machine, waiting for the VM to exit.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>crash</methodname>
+    </term>
     <listitem>
-      <para>Runs <literal>systemctl</literal> commands with optional support for
-      <literal>systemctl --user</literal></para>
-    <para>
-      <programlisting>
+     <para>
+      Simulate a sudden power failure, by telling the VM to exit immediately.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>block</methodname>
+    </term>
+    <listitem>
+     <para>
+      Simulate unplugging the Ethernet cable that connects the machine to the
+      other machines.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>unblock</methodname>
+    </term>
+    <listitem>
+     <para>
+      Undo the effect of <methodname>block</methodname>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>screenshot</methodname>
+    </term>
+    <listitem>
+     <para>
+      Take a picture of the display of the virtual machine, in PNG format. The
+      screenshot is linked from the HTML log.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>getScreenText</methodname>
+    </term>
+    <listitem>
+     <para>
+      Return a textual representation of what is currently visible on the
+      machine's screen using optical character recognition.
+     </para>
+     <note>
+      <para>
+       This requires passing <option>enableOCR</option> to the test attribute
+       set.
+      </para>
+     </note>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>sendMonitorCommand</methodname>
+    </term>
+    <listitem>
+     <para>
+      Send a command to the QEMU monitor. This is rarely used, but allows doing
+      stuff such as attaching virtual USB disks to a running machine.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>sendKeys</methodname>
+    </term>
+    <listitem>
+     <para>
+      Simulate pressing keys on the virtual keyboard, e.g.,
+      <literal>sendKeys("ctrl-alt-delete")</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>sendChars</methodname>
+    </term>
+    <listitem>
+     <para>
+      Simulate typing a sequence of characters on the virtual keyboard, e.g.,
+      <literal>sendKeys("foobar\n")</literal> will type the string
+      <literal>foobar</literal> followed by the Enter key.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>execute</methodname>
+    </term>
+    <listitem>
+     <para>
+      Execute a shell command, returning a list
+      <literal>(<replaceable>status</replaceable>,
+      <replaceable>stdout</replaceable>)</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>succeed</methodname>
+    </term>
+    <listitem>
+     <para>
+      Execute a shell command, raising an exception if the exit status is not
+      zero, otherwise returning the standard output.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>fail</methodname>
+    </term>
+    <listitem>
+     <para>
+      Like <methodname>succeed</methodname>, but raising an exception if the
+      command returns a zero status.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitUntilSucceeds</methodname>
+    </term>
+    <listitem>
+     <para>
+      Repeat a shell command with 1-second intervals until it succeeds.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitUntilFails</methodname>
+    </term>
+    <listitem>
+     <para>
+      Repeat a shell command with 1-second intervals until it fails.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForUnit</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until the specified systemd unit has reached the “active” state.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForFile</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until the specified file exists.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForOpenPort</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until a process is listening on the given TCP port (on
+      <literal>localhost</literal>, at least).
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForClosedPort</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until nobody is listening on the given TCP port.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForX</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until the X11 server is accepting connections.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForText</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until the supplied regular expressions matches the textual contents
+      of the screen by using optical character recognition (see
+      <methodname>getScreenText</methodname>).
+     </para>
+     <note>
+      <para>
+       This requires passing <option>enableOCR</option> to the test attribute
+       set.
+      </para>
+     </note>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>waitForWindow</methodname>
+    </term>
+    <listitem>
+     <para>
+      Wait until an X11 window has appeared whose name matches the given
+      regular expression, e.g., <literal>waitForWindow(qr/Terminal/)</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>copyFileFromHost</methodname>
+    </term>
+    <listitem>
+     <para>
+      Copies a file from host to machine, e.g.,
+      <literal>copyFileFromHost("myfile", "/etc/my/important/file")</literal>.
+     </para>
+     <para>
+      The first argument is the file on the host. The file needs to be
+      accessible while building the nix derivation. The second argument is the
+      location of the file on the machine.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term><methodname>systemctl</methodname>
+    </term>
+    <listitem>
+     <para>
+      Runs <literal>systemctl</literal> commands with optional support for
+      <literal>systemctl --user</literal>
+     </para>
+     <para>
+<programlisting>
         $machine->systemctl("list-jobs --no-pager"); // runs `systemctl list-jobs --no-pager`
         $machine->systemctl("list-jobs --no-pager", "any-user"); // spawns a shell for `any-user` and runs `systemctl --user list-jobs --no-pager`
       </programlisting>
-    </para>
+     </para>
     </listitem>
-  </varlistentry>
+   </varlistentry>
+  </variablelist>
+ </para>
 
-</variablelist>
-
-</para>
-
-<para>
-  To test user units declared by <literal>systemd.user.services</literal> the optional <literal>$user</literal>
-  argument can be used:
-
-  <programlisting>
+ <para>
+  To test user units declared by <literal>systemd.user.services</literal> the
+  optional <literal>$user</literal> argument can be used:
+<programlisting>
     $machine->start;
     $machine->waitForX;
     $machine->waitForUnit("xautolock.service", "x-session-user");
   </programlisting>
-
   This applies to <literal>systemctl</literal>, <literal>getUnitInfo</literal>,
-  <literal>waitForUnit</literal>, <literal>startJob</literal>
-  and <literal>stopJob</literal>.
-</para>
-
+  <literal>waitForUnit</literal>, <literal>startJob</literal> and
+  <literal>stopJob</literal>.
+ </para>
 </section>