Merge commit 'a323570a264da96a0b0bcc1c9aa017794acdc752'

10 files changed, 161 insertions, 18 deletions

diff --git a/nixpkgs/doc/builders/fetchers.chapter.md b/nixpkgs/doc/builders/fetchers.chapter.md
index b1b00106b6c9..e36724f295f9 100644
--- a/nixpkgs/doc/builders/fetchers.chapter.md
+++ b/nixpkgs/doc/builders/fetchers.chapter.md
@@ -6,7 +6,7 @@ When using Nix, you will frequently need to download source code and other files
 
 Because fixed output derivations are _identified_ by their hash, a common mistake is to update a fetcher's URL or a version parameter, without updating the hash. **This will cause the old contents to be used.** So remember to always invalidate the hash argument.
 
-For those who develop and maintain fetcheres, a similar problem arises with changes to the implementation of a fetcher. These may cause a fixed output derivation to fail, but won't normally be caught by tests because the supposed output is already in the store or cache. For the purpose of testing, you can use a trick that is embodied by the [`invalidateFetcherByDrvHash`](#sec-pkgs-invalidateFetcherByDrvHash) function. It uses the derivation `name` to create a unique output path per fetcher implementation, defeating the caching precisely where it would be harmful.
+For those who develop and maintain fetchers, a similar problem arises with changes to the implementation of a fetcher. These may cause a fixed output derivation to fail, but won't normally be caught by tests because the supposed output is already in the store or cache. For the purpose of testing, you can use a trick that is embodied by the [`invalidateFetcherByDrvHash`](#sec-pkgs-invalidateFetcherByDrvHash) function. It uses the derivation `name` to create a unique output path per fetcher implementation, defeating the caching precisely where it would be harmful.
 
 ## `fetchurl` and `fetchzip` {#fetchurl}
 
diff --git a/nixpkgs/doc/builders/packages/etc-files.section.md b/nixpkgs/doc/builders/packages/etc-files.section.md
new file mode 100644
index 000000000000..2405a54634d8
--- /dev/null
+++ b/nixpkgs/doc/builders/packages/etc-files.section.md
@@ -0,0 +1,18 @@
+# /etc files {#etc}
+
+Certain calls in glibc require access to runtime files found in /etc such as `/etc/protocols` or `/etc/services` -- [getprotobyname](https://linux.die.net/man/3/getprotobyname) is one such function.
+
+On non-NixOS distributions these files are typically provided by packages (i.e. [netbase](https://packages.debian.org/sid/netbase)) if not already pre-installed in your distribution. This can cause non-reproducibility for code if they rely on these files being present.
+
+If [iana-etc](https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.iana-etc.x86_64-linux) is part of your _buildInputs_ then it will set the environment varaibles `NIX_ETC_PROTOCOLS` and `NIX_ETC_SERVICES` to the corresponding files in the package through a _setup-hook_.
+
+
+```bash
+> nix-shell -p iana-etc
+
+[nix-shell:~]$ env | grep NIX_ETC
+NIX_ETC_SERVICES=/nix/store/aj866hr8fad8flnggwdhrldm0g799ccz-iana-etc-20210225/etc/services
+NIX_ETC_PROTOCOLS=/nix/store/aj866hr8fad8flnggwdhrldm0g799ccz-iana-etc-20210225/etc/protocols
+```
+
+Nixpkg's version of [glibc](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/glibc/default.nix) has been patched to check for the existence of these environment variables. If the environment variable are *not set*, then it will attempt to find the files at the default location within _/etc_.
diff --git a/nixpkgs/doc/builders/packages/index.xml b/nixpkgs/doc/builders/packages/index.xml
index f5b05b0bbccf..206e1e49f1f8 100644
--- a/nixpkgs/doc/builders/packages/index.xml
+++ b/nixpkgs/doc/builders/packages/index.xml
@@ -17,6 +17,7 @@
  <xi:include href="kakoune.section.xml" />
  <xi:include href="linux.section.xml" />
  <xi:include href="locales.section.xml" />
+ <xi:include href="etc-files.section.xml" />
  <xi:include href="nginx.section.xml" />
  <xi:include href="opengl.section.xml" />
  <xi:include href="shell-helpers.section.xml" />
diff --git a/nixpkgs/doc/contributing/coding-conventions.chapter.md b/nixpkgs/doc/contributing/coding-conventions.chapter.md
index 85c8626bd99c..7a8e7741a330 100644
--- a/nixpkgs/doc/contributing/coding-conventions.chapter.md
+++ b/nixpkgs/doc/contributing/coding-conventions.chapter.md
@@ -181,6 +181,21 @@
   rev = "${version}";
   ```
 
+- Filling lists condionally _should_ be done with `lib.optional(s)` instead of using `if cond then [ ... ] else null` or `if cond then [ ... ] else [ ]`.
+
+  ```nix
+  buildInputs = lib.optional stdenv.isDarwin iconv;
+  ```
+
+  instead of
+
+  ```nix
+  buildInputs = if stdenv.isDarwin then [ iconv ] else null;
+  ```
+
+  As an exception, an explicit conditional expression with null can be used when fixing a important bug without triggering a mass rebuild.
+  If this is done a follow up pull request _should_ be created to change the code to `lib.optional(s)`.
+
 - Arguments should be listed in the order they are used, with the exception of `lib`, which always goes first.
 
 ## Package naming {#sec-package-naming}
diff --git a/nixpkgs/doc/languages-frameworks/agda.section.md b/nixpkgs/doc/languages-frameworks/agda.section.md
index 2b7c35f68d3b..775a7a1a6429 100644
--- a/nixpkgs/doc/languages-frameworks/agda.section.md
+++ b/nixpkgs/doc/languages-frameworks/agda.section.md
@@ -158,7 +158,23 @@ This can be overridden.
 
 By default, Agda sources are files ending on `.agda`, or literate Agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised Agda source extensions can be extended by setting the `extraExtensions` config variable.
 
-## Adding Agda packages to Nixpkgs {#adding-agda-packages-to-nixpkgs}
+## Maintaining the Agda package set on Nixpkgs {#maintaining-the-agda-package-set-on-nixpkgs}
+
+We are aiming at providing all common Agda libraries as packages on `nixpkgs`,
+and keeping them up to date.
+Contributions and maintenance help is always appreciated,
+but the maintenance effort is typically low since the Agda ecosystem is quite small.
+
+The `nixpkgs` Agda package set tries to take up a role similar to that of [Stackage](https://www.stackage.org/) in the Haskell world.
+It is a curated set of libraries that:
+
+1. Always work together.
+2. Are as up-to-date as possible.
+
+While the Haskell ecosystem is huge, and Stackage is highly automatised,
+the Agda package set is small and can (still) be maintained by hand.
+
+### Adding Agda packages to Nixpkgs {#adding-agda-packages-to-nixpkgs}
 
 To add an Agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other Agda libraries, so the top line of the `default.nix` can look like:
 
@@ -192,3 +208,49 @@ mkDerivation {
 This library has a file called `.agda-lib`, and so we give an empty string to `libraryFile` as nothing precedes `.agda-lib` in the filename. This file contains `name: IAL-1.3`, and so we let `libraryName =  "IAL-1.3"`. This library does not use an `Everything.agda` file and instead has a Makefile, so there is no need to set `everythingFile` and we set a custom `buildPhase`.
 
 When writing an Agda package it is essential to make sure that no `.agda-lib` file gets added to the store as a single file (for example by using `writeText`). This causes Agda to think that the nix store is a Agda library and it will attempt to write to it whenever it typechecks something. See [https://github.com/agda/agda/issues/4613](https://github.com/agda/agda/issues/4613).
+
+In the pull request adding this library,
+you can test whether it builds correctly by writing in a comment:
+
+```
+@ofborg build agdaPackages.iowa-stdlib
+```
+
+### Maintaining Agda packages
+
+As mentioned before, the aim is to have a compatible, and up-to-date package set.
+These two conditions sometimes exclude each other:
+For example, if we update `agdaPackages.standard-library` because there was an upstream release,
+this will typically break many reverse dependencies,
+i.e. downstream Agda libraries that depend on the standard library.
+In `nixpkgs` we are typically among the first to notice this,
+since we have build tests in place to check this.
+
+In a pull request updating e.g. the standard library, you should write the following comment:
+
+```
+@ofborg build agdaPackages.standard-library.passthru.tests
+```
+
+This will build all reverse dependencies of the standard library,
+for example `agdaPackages.agda-categories`, or `agdaPackages.generic`.
+
+In some cases it is useful to build _all_ Agda packages.
+This can be done with the following Github comment:
+
+```
+@ofborg build agda.passthru.tests.allPackages
+```
+
+Sometimes, the builds of the reverse dependencies fail because they have not yet been updated and released.
+You should drop the maintainers a quick issue notifying them of the breakage,
+citing the build error (which you can get from the ofborg logs).
+If you are motivated, you might even send a pull request that fixes it.
+Usually, the maintainers will answer within a week or two with a new release.
+Bumping the version of that reverse dependency should be a further commit on your PR.
+
+In the rare case that a new release is not to be expected within an acceptable time,
+simply mark the broken package as broken by setting `meta.broken = true;`.
+This will exclude it from the build test.
+It can be added later when it is fixed,
+and does not hinder the advancement of the whole package set in the meantime.
diff --git a/nixpkgs/doc/languages-frameworks/dotnet.section.md b/nixpkgs/doc/languages-frameworks/dotnet.section.md
index 1bcb6e45210e..9bf96f3198a1 100644
--- a/nixpkgs/doc/languages-frameworks/dotnet.section.md
+++ b/nixpkgs/doc/languages-frameworks/dotnet.section.md
@@ -28,8 +28,7 @@ mkShell {
   packages = [
     (with dotnetCorePackages; combinePackages [
       sdk_3_1
-      sdk_3_0
-      sdk_2_1
+      sdk_5_0
     ])
   ];
 }
@@ -64,12 +63,46 @@ $ dotnet --info
 
 The `dotnetCorePackages.sdk_X_Y` is preferred over the old dotnet-sdk as both major and minor version are very important for a dotnet environment. If a given minor version isn't present (or was changed), then this will likely break your ability to build a project.
 
-## dotnetCorePackages.sdk vs dotnetCorePackages.net vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore {#dotnetcorepackages.sdk-vs-dotnetcorepackages.net-vs-dotnetcorepackages.netcore-vs-dotnetcorepackages.aspnetcore}
+## dotnetCorePackages.sdk vs dotnetCorePackages.runtime vs dotnetCorePackages.aspnetcore {#dotnetcorepackages.sdk-vs-dotnetcorepackages.runtime-vs-dotnetcorepackages.aspnetcore}
 
-The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `net`, `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. For runtime versions >= .NET 5 `net` is used while `netcore` is used for older .NET Core runtime version.
+The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `runtime` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications.
 
 ## Packaging a Dotnet Application {#packaging-a-dotnet-application}
 
-Ideally, we would like to build against the sdk, then only have the dotnet runtime available in the runtime closure.
+To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions:
+
+* `projectFile` has to be used for specifying the dotnet project file relative to the source root. These usually have `.sln` or `.csproj` file extensions.
+* `nugetDeps` has to be used to specify the NuGet dependency file. Unfortunately, these cannot be deterministically fetched without a lockfile. This file should be generated using `nuget-to-nix` tool, which is available in nixpkgs.
+* `executables` is used to specify which executables get wrapped to `$out/bin`, relative to `$out/lib/$pname`. If this is unset, all executables generated will get installed. If you do not want to install any, set this to `[]`.
+* `runtimeDeps` is used to wrap libraries into `LD_LIBRARY_PATH`. This is how dotnet usually handles runtime dependencies.
+* `buildType` is used to change the type of build. Possible values are `Release`, `Debug`, etc. By default, this is set to `Release`.
+* `dotnet-sdk` is useful in cases where you need to change what dotnet SDK is being used.
+* `dotnet-runtime` is useful in cases where you need to change what dotnet runtime is being used.
+* `dotnetRestoreFlags` can be used to pass flags to `dotnet restore`.
+* `dotnetBuildFlags` can be used to pass flags to `dotnet build`.
+* `dotnetInstallFlags` can be used to pass flags to `dotnet install`.
+* `dotnetFlags` can be used to pass flags to all of the above phases.
+
+Here is an example `default.nix`, using some of the previously discussed arguments:
+```nix
+{ lib, buildDotnetModule, dotnetCorePackages, ffmpeg }:
+
+buildDotnetModule rec {
+  pname = "someDotnetApplication";
+  version = "0.1";
+
+  src = ./.;
+
+  projectFile = "src/project.sln";
+  nugetDeps = ./deps.nix; # File generated with `nuget-to-nix path/to/src > deps.nix`.
 
-TODO: Create closure-friendly way to package dotnet applications
+  dotnet-sdk = dotnetCorePackages.sdk_3_1;
+  dotnet-runtime = dotnetCorePackages.net_5_0;
+  dotnetFlags = [ "--runtime linux-x64" ];
+
+  executables = [ "foo" ]; # This wraps "$out/lib/$pname/foo" to `$out/bin/foo`.
+  executables = []; # Don't install any executables.
+
+  runtimeDeps = [ ffmpeg ]; # This will wrap ffmpeg's library path into `LD_LIBRARY_PATH`.
+}
+```
diff --git a/nixpkgs/doc/languages-frameworks/index.xml b/nixpkgs/doc/languages-frameworks/index.xml
index b010f27cac02..f221693e764c 100644
--- a/nixpkgs/doc/languages-frameworks/index.xml
+++ b/nixpkgs/doc/languages-frameworks/index.xml
@@ -12,6 +12,7 @@
  <xi:include href="coq.section.xml" />
  <xi:include href="crystal.section.xml" />
  <xi:include href="dhall.section.xml" />
+ <xi:include href="dotnet.section.xml" />
  <xi:include href="emscripten.section.xml" />
  <xi:include href="gnome.section.xml" />
  <xi:include href="go.section.xml" />
diff --git a/nixpkgs/doc/languages-frameworks/java.section.md b/nixpkgs/doc/languages-frameworks/java.section.md
index 77919d43f748..371bdf6323fb 100644
--- a/nixpkgs/doc/languages-frameworks/java.section.md
+++ b/nixpkgs/doc/languages-frameworks/java.section.md
@@ -72,6 +72,15 @@ in
   ...
 ```
 
+You can also specify what JDK your JRE should be based on, for example
+selecting a 'headless' build to avoid including a link to GTK+:
+
+```nix
+my_jre = pkgs.jre_minimal.override {
+  jdk = jdk11_headless;
+};
+```
+
 Note all JDKs passthru `home`, so if your application requires
 environment variables like `JAVA_HOME` being set, that can be done in a
 generic fashion with the `--set` argument of `makeWrapper`:
diff --git a/nixpkgs/doc/languages-frameworks/r.section.md b/nixpkgs/doc/languages-frameworks/r.section.md
index 56e3da64df2a..ad0fb10987c9 100644
--- a/nixpkgs/doc/languages-frameworks/r.section.md
+++ b/nixpkgs/doc/languages-frameworks/r.section.md
@@ -96,6 +96,11 @@ re-enter the shell.
 
 ## Updating the package set {#updating-the-package-set}
 
+There is a script and associated environment for regenerating the package
+sets and synchronising the rPackages tree to the current CRAN and matching
+BIOC release. These scripts are found in the `pkgs/development/r-modules`
+directory and executed as follows:
+
 ```bash
 nix-shell generate-shell.nix
 
@@ -112,12 +117,11 @@ Rscript generate-r-packages.R bioc-experiment > bioc-experiment-packages.nix.new
 mv bioc-experiment-packages.nix.new bioc-experiment-packages.nix
 ```
 
-`generate-r-packages.R <repo>` reads  `<repo>-packages.nix`, therefor the renaming.
-
-## Testing if the Nix-expression could be evaluated {#testing-if-the-nix-expression-could-be-evaluated}
-
-```bash
-nix-build test-evaluation.nix --dry-run
-```
+`generate-r-packages.R <repo>` reads  `<repo>-packages.nix`, therefore
+the renaming.
 
-If this exits fine, the expression is ok. If not, you have to edit `default.nix`
+Some packages require overrides to specify external dependencies or other
+patches and special requirements. These overrides are specified in the
+`pkgs/development/r-modules/default.nix` file. As the `*-packages.nix`
+contents are automatically generated it should not be edited and broken
+builds should be addressed using overrides.
diff --git a/nixpkgs/doc/stdenv/stdenv.chapter.md b/nixpkgs/doc/stdenv/stdenv.chapter.md
index 9befcaa51a9a..02042407f6ce 100644
--- a/nixpkgs/doc/stdenv/stdenv.chapter.md
+++ b/nixpkgs/doc/stdenv/stdenv.chapter.md
@@ -373,11 +373,11 @@ Additional file types can be supported by setting the `unpackCmd` variable (see
 
 ##### `srcs` / `src` {#var-stdenv-src}
 
-The list of source files or directories to be unpacked or copied. One of these must be set.
+The list of source files or directories to be unpacked or copied. One of these must be set. Note that if you use `srcs`, you should also set `sourceRoot` or `setSourceRoot`.
 
 ##### `sourceRoot` {#var-stdenv-sourceRoot}
 
-After running `unpackPhase`, the generic builder changes the current directory to the directory created by unpacking the sources. If there are multiple source directories, you should set `sourceRoot` to the name of the intended directory.
+After running `unpackPhase`, the generic builder changes the current directory to the directory created by unpacking the sources. If there are multiple source directories, you should set `sourceRoot` to the name of the intended directory. Set `sourceRoot = ".";` if you use `srcs` and control the unpack phase yourself.
 
 ##### `setSourceRoot` {#var-stdenv-setSourceRoot}