summary refs log tree commit diff
path: root/doc
diff options
context:
space:
mode:
authorLéo Gaspard <leo@gaspard.io>2018-08-04 12:18:28 +0900
committerLéo Gaspard <leo@gaspard.io>2018-10-30 21:31:39 +0900
commit02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165 (patch)
tree36033a707a534eaffc6992a11dbb61ec779fbf07 /doc
parentde8f3b422aa390bb15900e482cef1a3ee227f3e5 (diff)
downloadnixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar.gz
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar.bz2
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar.lz
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar.xz
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.tar.zst
nixlib-02e1f00ffdd0c1a74fbc0be5f314b8921b0c9165.zip
dovecot, opensmtpd: add link to test in `meta.tests`
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
Diffstat (limited to 'doc')
-rw-r--r--doc/meta.xml55
1 files changed, 55 insertions, 0 deletions
diff --git a/doc/meta.xml b/doc/meta.xml
index 496b32916552..51c7b2dfc88f 100644
--- a/doc/meta.xml
+++ b/doc/meta.xml
@@ -252,6 +252,61 @@ meta.platforms = stdenv.lib.platforms.linux;
    </varlistentry>
    <varlistentry>
     <term>
+     <varname>tests</varname>
+    </term>
+    <listitem>
+     <para>
+      An attribute set with as values tests. A test is a derivation, which
+      builds successfully when the test passes, and fails to build otherwise. A
+      derivation that is a test requires some <literal>meta</literal> elements
+      to be defined: <literal>needsVMSupport</literal> (automatically filled-in
+      for NixOS tests) and <literal>timeout</literal>.
+     </para>
+     <para>
+      The NixOS tests are available as <literal>nixosTests</literal> in
+      parameters of derivations. For instance, the OpenSMTPD derivation
+      includes lines similar to:
+<programlisting>
+{ /* ... */, nixosTests }:
+{
+  # ...
+  meta.tests = {
+    basic-functionality-and-dovecot-integration = nixosTests.opensmtpd;
+  };
+}
+</programlisting>
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term>
+     <varname>timeout</varname>
+    </term>
+    <listitem>
+     <para>
+      A timeout (in seconds) for building the derivation. If the derivation
+      takes longer than this time to build, it can fail due to breaking the
+      timeout. However, all computers do not have the same computing power,
+      hence some builders may decide to apply a multiplicative factor to this
+      value. When filling this value in, try to keep it approximately
+      consistent with other values already present in
+      <literal>nixpkgs</literal>.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term>
+     <varname>needsVMSupport</varname>
+    </term>
+    <listitem>
+     <para>
+      A boolan that states whether the derivation requires build-time support
+      for Virtual Machine to build successfully.
+     </para>
+    </listitem>
+   </varlistentry>
+   <varlistentry>
+    <term>
      <varname>hydraPlatforms</varname>
     </term>
     <listitem>