Merge remote-tracking branch 'origin/staging-next' into staging

Conflicts:
    pkgs/development/tools/language-servers/ansible-language-server/default.nix

9 files changed, 135 insertions, 6 deletions

diff --git a/doc/builders/images.xml b/doc/builders/images.xml
index f86ebd86bee4..7d06130e3eca 100644
--- a/doc/builders/images.xml
+++ b/doc/builders/images.xml
@@ -10,4 +10,5 @@
  <xi:include href="images/ocitools.section.xml" />
  <xi:include href="images/snaptools.section.xml" />
  <xi:include href="images/portableservice.section.xml" />
+ <xi:include href="images/makediskimage.section.xml" />
 </chapter>
diff --git a/doc/builders/images/makediskimage.section.md b/doc/builders/images/makediskimage.section.md
new file mode 100644
index 000000000000..9798a0be4d46
--- /dev/null
+++ b/doc/builders/images/makediskimage.section.md
@@ -0,0 +1,107 @@
+# `<nixpkgs/nixos/lib/make-disk-image.nix>` {#sec-make-disk-image}
+
+`<nixpkgs/nixos/lib/make-disk-image.nix>` is a function to create _disk images_ in multiple formats: raw, QCOW2 (QEMU), QCOW2-Compressed (compressed version), VDI (VirtualBox), VPC (VirtualPC).
+
+This function can create images in two ways:
+
+- using `cptofs` without any virtual machine to create a Nix store disk image,
+- using a virtual machine to create a full NixOS installation.
+
+When testing early-boot or lifecycle parts of NixOS such as a bootloader or multiple generations, it is necessary to opt for a full NixOS system installation.
+Whereas for many web servers, applications, it is possible to work with a Nix store only disk image and is faster to build.
+
+NixOS tests also use this function when preparing the VM. The `cptofs` method is used when `virtualisation.useBootLoader` is false (the default). Otherwise the second method is used.
+
+## Features
+
+For reference, read the function signature source code for documentation on arguments: <https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-disk-image.nix>.
+Features are separated in various sections depending on if you opt for a Nix-store only image or a full NixOS image.
+
+### Common
+
+- arbitrary NixOS configuration
+- automatic or bound disk size: `diskSize` parameter, `additionalSpace` can be set when `diskSize` is `auto` to add a constant of disk space 
+- multiple partition table layouts: EFI, legacy, legacy + GPT, hybrid, none through `partitionTableType` parameter
+- OVMF or EFI firmwares and variables templates can be customized
+- root filesystem `fsType` can be customized to whatever `mkfs.${fsType}` exist during operations
+- root filesystem label can be customized, defaults to `nix-store` if it's a Nix store image, otherwise `nixpkgs/nixos`
+- arbitrary code can be executed after disk image was produced with `postVM`
+- the current nixpkgs can be realized as a channel in the disk image, which will change the hash of the image when the sources are updated
+- additional store paths can be provided through `additionalPaths`
+
+### Full NixOS image
+
+- arbitrary contents with permissions can be placed in the target filesystem using `contents`
+- a `/etc/nixpkgs/nixos/configuration.nix` can be provided through `configFile`
+- bootloaders are supported
+- EFI variables can be mutated during image production and the result is exposed in `$out`
+- boot partition size when partition table is `efi` or `hybrid`
+
+### On bit-to-bit reproducibility
+
+Images are **NOT** deterministic, please do not hesitate to try to fix this, source of determinisms are (not exhaustive) :
+
+- bootloader installation have timestamps
+- SQLite Nix store database contain registration times
+- `/etc/shadow` is in a non-deterministic order
+
+A `deterministic` flag is available for best efforts determinism.
+
+## Usage
+
+To produce a Nix-store only image:
+```nix
+let
+  pkgs = import <nixpkgs> {};
+  lib = pkgs.lib;
+  make-disk-image = import <nixpkgs/nixos/lib/make-disk-image.nix>;
+in
+  make-disk-image {
+    inherit pkgs lib;
+    config = {};
+    additionalPaths = [ ];
+    format = "qcow2";
+    onlyNixStore = true;
+    partitionTableType = "none";
+    installBootLoader = false;
+    touchEFIVars = false;
+    diskSize = "auto";
+    additionalSpace = "0M"; # Defaults to 512M.
+    copyChannel = false;
+  }
+```
+
+Some arguments can be left out, they are shown explicitly for the sake of the example.
+
+Building this derivation will provide a QCOW2 disk image containing only the Nix store and its registration information.
+
+To produce a NixOS installation image disk with UEFI and bootloader installed:
+```nix
+let
+  pkgs = import <nixpkgs> {};
+  lib = pkgs.lib;
+  make-disk-image = import <nixpkgs/nixos/lib/make-disk-image.nix>;
+  evalConfig = import <nixpkgs/nixos/lib/eval-config.nix>;
+in
+  make-disk-image {
+    inherit pkgs lib;
+    config = evalConfig {
+      modules = [
+        {
+          fileSystems."/" = { device = "/dev/vda"; fsType = "ext4"; autoFormat = true; };
+          boot.grub.device = "/dev/vda";
+        }
+      ];
+    };
+    format = "qcow2";
+    onlyNixStore = false;
+    partitionTableType = "legacy+gpt";
+    installBootLoader = true;
+    touchEFIVars = true;
+    diskSize = "auto";
+    additionalSpace = "0M"; # Defaults to 512M.
+    copyChannel = false;
+  }
+```
+
+
diff --git a/doc/builders/images/ocitools.section.md b/doc/builders/images/ocitools.section.md
index d3ab8776786b..c35f65bce007 100644
--- a/doc/builders/images/ocitools.section.md
+++ b/doc/builders/images/ocitools.section.md
@@ -34,4 +34,4 @@ buildContainer {
 
 - `mounts` specifies additional mount points chosen by the user. By default only a minimal set of necessary filesystems are mounted into the container (e.g procfs, cgroupfs)
 
-- `readonly` makes the container\'s rootfs read-only if it is set to true. The default value is false `false`.
+- `readonly` makes the container's rootfs read-only if it is set to true. The default value is false `false`.
diff --git a/doc/builders/packages/dlib.section.md b/doc/builders/packages/dlib.section.md
index 8f0aa8610180..022195310a71 100644
--- a/doc/builders/packages/dlib.section.md
+++ b/doc/builders/packages/dlib.section.md
@@ -4,7 +4,7 @@
 
 ## Compiling without AVX support {#compiling-without-avx-support}
 
-Especially older CPUs don\'t support [AVX](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) (Advanced Vector Extensions) instructions that are used by DLib to optimize their algorithms.
+Especially older CPUs don't support [AVX](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) (Advanced Vector Extensions) instructions that are used by DLib to optimize their algorithms.
 
 On the affected hardware errors like `Illegal instruction` will occur. In those cases AVX support needs to be disabled:
 
diff --git a/doc/contributing/coding-conventions.chapter.md b/doc/contributing/coding-conventions.chapter.md
index 275a3c7af5d2..f6a0970165f5 100644
--- a/doc/contributing/coding-conventions.chapter.md
+++ b/doc/contributing/coding-conventions.chapter.md
@@ -260,6 +260,10 @@ When in doubt, consider refactoring the `pkgs/` tree, e.g. creating new categori
 
     - `development/tools/build-managers` (e.g. `gnumake`)
 
+  - **If it’s a _language server_:**
+
+    - `development/tools/language-servers` (e.g. `ccls` or `rnix-lsp`)
+
   - **Else:**
 
     - `development/tools/misc` (e.g. `binutils`)
diff --git a/doc/contributing/submitting-changes.chapter.md b/doc/contributing/submitting-changes.chapter.md
index e3b2990f84a5..96e2ecf970cb 100644
--- a/doc/contributing/submitting-changes.chapter.md
+++ b/doc/contributing/submitting-changes.chapter.md
@@ -199,7 +199,7 @@ It’s important to test any executables generated by a build when you change or
 
 ### Meets Nixpkgs contribution standards {#submitting-changes-contribution-standards}
 
-The last checkbox is fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md). The contributing document has detailed information on standards the Nix community has for commit messages, reviews, licensing of contributions you make to the project, etc\... Everyone should read and understand the standards the community has for contributing before submitting a pull request.
+The last checkbox is fits [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md). The contributing document has detailed information on standards the Nix community has for commit messages, reviews, licensing of contributions you make to the project, etc... Everyone should read and understand the standards the community has for contributing before submitting a pull request.
 
 ## Hotfixing pull requests {#submitting-changes-hotfixing-pull-requests}
 
diff --git a/doc/functions/nix-gitignore.section.md b/doc/functions/nix-gitignore.section.md
index 2fb833b23000..8eb4081d2878 100644
--- a/doc/functions/nix-gitignore.section.md
+++ b/doc/functions/nix-gitignore.section.md
@@ -4,7 +4,7 @@
 
 ## Usage {#sec-pkgs-nix-gitignore-usage}
 
-`pkgs.nix-gitignore` exports a number of functions, but you\'ll most likely need either `gitignoreSource` or `gitignoreSourcePure`. As their first argument, they both accept either 1. a file with gitignore lines or 2. a string with gitignore lines, or 3. a list of either of the two. They will be concatenated into a single big string.
+`pkgs.nix-gitignore` exports a number of functions, but you'll most likely need either `gitignoreSource` or `gitignoreSourcePure`. As their first argument, they both accept either 1. a file with gitignore lines or 2. a string with gitignore lines, or 3. a list of either of the two. They will be concatenated into a single big string.
 
 ```nix
 { pkgs ? import <nixpkgs> {} }:
@@ -30,7 +30,7 @@ gitignoreSourcePure = gitignoreFilterSourcePure (_: _: true);
 gitignoreSource = gitignoreFilterSource (_: _: true);
 ```
 
-Those filter functions accept the same arguments the `builtins.filterSource` function would pass to its filters, thus `fn: gitignoreFilterSourcePure fn ""` should be extensionally equivalent to `filterSource`. The file is blacklisted if it\'s blacklisted by either your filter or the gitignoreFilter.
+Those filter functions accept the same arguments the `builtins.filterSource` function would pass to its filters, thus `fn: gitignoreFilterSourcePure fn ""` should be extensionally equivalent to `filterSource`. The file is blacklisted if it's blacklisted by either your filter or the gitignoreFilter.
 
 If you want to make your own filter from scratch, you may use
 
diff --git a/doc/languages-frameworks/rust.section.md b/doc/languages-frameworks/rust.section.md
index a9d8e54cafd8..ec703105e15a 100644
--- a/doc/languages-frameworks/rust.section.md
+++ b/doc/languages-frameworks/rust.section.md
@@ -186,6 +186,23 @@ added. To find the correct hash, you can first use `lib.fakeSha256` or
 `lib.fakeHash` as a stub hash. Building the package (and thus the
 vendored dependencies) will then inform you of the correct hash.
 
+For usage outside nixpkgs, `allowBuiltinFetchGit` could be used to
+avoid having to specify `outputHashes`. For example:
+
+```nix
+rustPlatform.buildRustPackage rec {
+  pname = "myproject";
+  version = "1.0.0";
+
+  cargoLock = {
+    lockFile = ./Cargo.lock;
+    allowBuiltinFetchGit = true;
+  };
+
+  # ...
+}
+```
+
 ### Cargo features {#cargo-features}
 
 You can disable default features using `buildNoDefaultFeatures`, and
diff --git a/doc/using/configuration.chapter.md b/doc/using/configuration.chapter.md
index 0391af0f1760..e657cb21c295 100644
--- a/doc/using/configuration.chapter.md
+++ b/doc/using/configuration.chapter.md
@@ -73,7 +73,7 @@ There are also two ways to try compiling a package which has been marked as unsu
     }
     ```
 
-The difference between a package being unsupported on some system and being broken is admittedly a bit fuzzy. If a program *ought* to work on a certain platform, but doesn't, the platform should be included in `meta.platforms`, but marked as broken with e.g.  `meta.broken = !hostPlatform.isWindows`. Of course, this begs the question of what \"ought\" means exactly. That is left to the package maintainer.
+The difference between a package being unsupported on some system and being broken is admittedly a bit fuzzy. If a program *ought* to work on a certain platform, but doesn't, the platform should be included in `meta.platforms`, but marked as broken with e.g.  `meta.broken = !hostPlatform.isWindows`. Of course, this begs the question of what "ought" means exactly. That is left to the package maintainer.
 
 ## Installing unfree packages {#sec-allow-unfree}