Merge https://github.com/NixOS/nixpkgs

6 files changed, 229 insertions, 89 deletions

diff --git a/nixpkgs/doc/default.nix b/nixpkgs/doc/default.nix
index 61bbd2ba8dc8..8f5def70a6e7 100644
--- a/nixpkgs/doc/default.nix
+++ b/nixpkgs/doc/default.nix
@@ -24,6 +24,7 @@ let
       { name = "cli"; description = "command-line serialization functions"; }
       { name = "gvariant"; description = "GVariant formatted string serialization functions"; }
       { name = "customisation"; description = "Functions to customise (derivation-related) functions, derivatons, or attribute sets"; }
+      { name = "meta"; description = "functions for derivation metadata"; }
     ];
   };
 
diff --git a/nixpkgs/doc/languages-frameworks/nim.section.md b/nixpkgs/doc/languages-frameworks/nim.section.md
index 6b0fb3df0311..45cd07b3a3d8 100644
--- a/nixpkgs/doc/languages-frameworks/nim.section.md
+++ b/nixpkgs/doc/languages-frameworks/nim.section.md
@@ -1,74 +1,38 @@
 # Nim {#nim}
 
-## Overview {#nim-overview}
-
-The Nim compiler, a builder function, and some packaged libraries are available
-in Nixpkgs. Until now each compiler release has been effectively backwards
-compatible so only the latest version is available.
-
-## Nim program packages in Nixpkgs {#nim-program-packages-in-nixpkgs}
-
-Nim programs can be built using `nimPackages.buildNimPackage`. In the
-case of packages not containing exported library code the attribute
-`nimBinOnly` should be set to `true`.
+The Nim compiler and a builder function is available.
+Nim programs are built using `buildNimPackage` and a lockfile containing Nim dependencies.
 
 The following example shows a Nim program that depends only on Nim libraries:
-
 ```nix
-{ lib, nimPackages, fetchFromGitHub }:
+{ lib, buildNimPackage, fetchFromGitHub }:
 
-nimPackages.buildNimPackage (finalAttrs: {
+buildNimPackage { } (finalAttrs: {
   pname = "ttop";
-  version = "1.0.1";
-  nimBinOnly = true;
+  version = "1.2.7";
 
   src = fetchFromGitHub {
     owner = "inv2004";
     repo = "ttop";
     rev = "v${finalAttrs.version}";
-    hash = "sha256-x4Uczksh6p3XX/IMrOFtBxIleVHdAPX9e8n32VAUTC4=";
+    hash = "sha256-oPdaUqh6eN1X5kAYVvevOndkB/xnQng9QVLX9bu5P5E=";
   };
 
-  buildInputs = with nimPackages; [ asciigraph illwill parsetoml zippy ];
+  lockFile = ./lock.json;
 
-})
-```
-
-## Nim library packages in Nixpkgs {#nim-library-packages-in-nixpkgs}
-
-
-Nim libraries can also be built using `nimPackages.buildNimPackage`, but
-often the product of a fetcher is sufficient to satisfy a dependency.
-The `fetchgit`, `fetchFromGitHub`, and `fetchNimble` functions yield an
-output that can be discovered during the `configurePhase` of `buildNimPackage`.
-
-Nim library packages are listed in
-[pkgs/top-level/nim-packages.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/nim-packages.nix) and implemented at
-[pkgs/development/nim-packages](https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/nim-packages).
-
-The following example shows a Nim library that propagates a dependency on a
-non-Nim package:
-```nix
-{ lib, buildNimPackage, fetchNimble, SDL2 }:
-
-buildNimPackage (finalAttrs: {
-  pname = "sdl2";
-  version = "2.0.4";
-  src = fetchNimble {
-    inherit (finalAttrs) pname version;
-    hash = "sha256-Vtcj8goI4zZPQs2TbFoBFlcR5UqDtOldaXSH/+/xULk=";
-  };
-  propagatedBuildInputs = [ SDL2 ];
+  nimFlags = [
+    "-d:NimblePkgVersion=${finalAttrs.version}"
+  ];
 })
 ```
 
 ## `buildNimPackage` parameters {#buildnimpackage-parameters}
 
-All parameters from `stdenv.mkDerivation` function are still supported. The
-following are specific to `buildNimPackage`:
+The `buildNimPackage` function takes an attrset of parameters that are passed on to `stdenv.mkDerivation`.
 
-* `nimBinOnly ? false`: If `true` then build only the programs listed in
-  the Nimble file in the packages sources.
+The following parameters are specific to `buildNimPackage`:
+
+* `lockFile`: JSON formatted lockfile.
 * `nimbleFile`: Specify the Nimble file location of the package being built
   rather than discover the file at build-time.
 * `nimRelease ? true`: Build the package in *release* mode.
@@ -77,6 +41,72 @@ following are specific to `buildNimPackage`:
   Use this to specify defines with arguments in the form of `-d:${name}=${value}`.
 * `nimDoc` ? false`: Build and install HTML documentation.
 
-* `buildInputs` ? []: The packages listed here will be searched for `*.nimble`
-  files which are used to populate the Nim library path. Otherwise the standard
-  behavior is in effect.
+## Lockfiles {#nim-lockfiles}
+Nim lockfiles are created with the `nim_lk` utility.
+Run `nim_lk` with the source directory as an argument and it will print a lockfile to stdout.
+```sh
+$ cd nixpkgs
+$ nix build -f . ttop.src
+$ nix run -f . nim_lk ./result | jq --sort-keys > pkgs/by-name/tt/ttop/lock.json
+```
+
+## Lockfile dependency overrides {#nimoverrides}
+
+The `buildNimPackage` function matches the libraries specified by `lockFile` to attrset of override functions that are then applied to the package derivation.
+The default overrides are maintained as the top-level `nimOverrides` attrset at `pkgs/top-level/nim-overrides.nix`.
+
+For example, to propagate a dependency on SDL2 for lockfiles that select the Nim `sdl2` library, an overlay is added to the set in the `nim-overrides.nix` file:
+```nix
+{ lib
+/* … */
+, SDL2
+/* … */
+}:
+
+{
+  /* … */
+  sdl2 =
+    lockAttrs:
+    finalAttrs:
+    { buildInputs ? [ ], ... }:
+    {
+      buildInputs = buildInputs ++ [ SDL2 ];
+    };
+  /* … */
+}
+```
+
+The annotations in the `nim-overrides.nix` set are functions that take three arguments and return a new attrset to be overlayed on the package being built.
+- lockAttrs: the attrset for this library from within a lockfile. This can be used to implement library version constraints, such as marking libraries as broken or insecure.
+- finalAttrs: the final attrset passed by `buildNimPackage` to `stdenv.mkDerivation`.
+- prevAttrs: the attrset produced by initial arguments to `buildNimPackage` and any preceding lockfile overlays.
+
+### Overriding an Nim library override {#nimoverrides-overrides}
+
+The `nimOverrides` attrset makes it possible to modify overrides in a few different ways.
+
+Override a package internal to its definition:
+```nix
+{ lib, buildNimPackage, nimOverrides, libressl }:
+
+let
+  buildNimPackage' = buildNimPackage.override {
+    nimOverrides = nimOverrides.override { openssl = libressl; };
+  };
+in buildNimPackage' (finalAttrs: {
+  pname = "foo";
+  # …
+})
+
+```
+
+Override a package externally:
+```nix
+{ pkgs }: {
+  foo = pkgs.foo.override {
+    buildNimPackage = pkgs.buildNimPackage.override {
+      nimOverrides = pkgs.nimOverrides.override { openssl = libressl; };
+    };
+  };
+}
+```
diff --git a/nixpkgs/doc/languages-frameworks/texlive.section.md b/nixpkgs/doc/languages-frameworks/texlive.section.md
index 2ba846dc492d..8b1ed92a450c 100644
--- a/nixpkgs/doc/languages-frameworks/texlive.section.md
+++ b/nixpkgs/doc/languages-frameworks/texlive.section.md
@@ -98,24 +98,30 @@ Release 23.11 ships with a new interface that will eventually replace `texlive.c
 
 ## Custom packages {#sec-language-texlive-custom-packages}
 
-You may find that you need to use an external TeX package. A derivation for such package has to provide the contents of the "texmf" directory in its output and provide the appropriate `tlType` attribute (one of `"run"`, `"bin"`, `"doc"`, `"source"`). Dependencies on other TeX packages can be listed in the attribute `tlDeps`.
+You may find that you need to use an external TeX package. A derivation for such package has to provide the contents of the "texmf" directory in its `"tex"` output, according to the [TeX Directory Structure](https://tug.ctan.org/tds/tds.html). Dependencies on other TeX packages can be listed in the attribute `tlDeps`.
 
-Such derivation must then be listed in the attribute `pkgs` of an attribute set passed to `texlive.combine`, for instance by passing `extraPkgs = { pkgs = [ custom_package ]; };`. Within Nixpkgs, `pkgs` should be part of the derivation itself, allowing users to call `texlive.combine { inherit (texlive) scheme-small; inherit some_tex_package; }`.
+The functions `texlive.combine` and `texlive.withPackages` recognise the following outputs:
 
-Here is a (very verbose) example where the attribute `pkgs` is attached to the derivation itself, which requires creating a fixed point. See also the packages `auctex`, `eukleides`, `mftrace` for more examples.
+- `"out"`: contents are linked in the TeX Live environment, and binaries in the `$out/bin` folder are wrapped;
+- `"tex"`: linked in `$TEXMFDIST`; files should follow the TDS (for instance `$tex/tex/latex/foiltex/foiltex.cls`);
+- `"texdoc"`, `"texsource"`: ignored by default, treated as `"tex"`;
+- `"tlpkg"`: linked in `$TEXMFROOT/tlpkg`;
+- `"man"`, `"info"`, ...: the other outputs are combined into separate outputs.
+
+When using `pkgFilter`, `texlive.combine` will assign `tlType` respectively `"bin"`, `"run"`, `"doc"`, `"source"`, `"tlpkg"` to the above outputs.
+
+Here is a (very verbose) example. See also the packages `auctex`, `eukleides`, `mftrace` for more examples.
 
 ```nix
 with import <nixpkgs> {};
 
 let
-  foiltex = stdenvNoCC.mkDerivation (finalAttrs: {
+  foiltex = stdenvNoCC.mkDerivation {
     pname = "latex-foiltex";
     version = "2.1.4b";
-    passthru = {
-      pkgs = [ finalAttrs.finalPackage ];
-      tlDeps = with texlive; [ latex ];
-      tlType = "run";
-    };
+
+    outputs = [ "tex" "texdoc" ];
+    passthru.tlDeps = with texlive; [ latex ];
 
     srcs = [
       (fetchurl {
@@ -138,7 +144,13 @@ let
       runHook postUnpack
     '';
 
-    nativeBuildInputs = [ texlive.combined.scheme-small ];
+    nativeBuildInputs = [
+      (texliveSmall.withPackages (ps: with ps; [ cm-super hypdoc latexmk ]))
+      # multiple-outputs.sh fails if $out is not defined
+      (writeShellScript "force-tex-output.sh" ''
+        out="''${tex-}"
+      '')
+    ];
 
     dontConfigure = true;
 
@@ -148,15 +160,23 @@ let
       # Generate the style files
       latex foiltex.ins
 
+      # Generate the documentation
+      export HOME=.
+      latexmk -pdf foiltex.dtx
+
       runHook postBuild
     '';
 
     installPhase = ''
       runHook preInstall
 
-      path="$out/tex/latex/foiltex"
+      path="$tex/tex/latex/foiltex"
+      mkdir -p "$path"
+      cp *.{cls,def,clo,sty} "$path/"
+
+      path="$texdoc/doc/tex/latex/foiltex"
       mkdir -p "$path"
-      cp *.{cls,def,clo} "$path/"
+      cp *.pdf "$path/"
 
       runHook postInstall
     '';
@@ -167,12 +187,9 @@ let
       maintainers = with maintainers; [ veprbl ];
       platforms = platforms.all;
     };
-  });
-
-  latex_with_foiltex = texlive.combine {
-    inherit (texlive) scheme-small;
-    inherit foiltex;
   };
+
+  latex_with_foiltex = texliveSmall.withPackages (_: [ foiltex ]);
 in
   runCommand "test.pdf" {
     nativeBuildInputs = [ latex_with_foiltex ];
diff --git a/nixpkgs/doc/manpage-urls.json b/nixpkgs/doc/manpage-urls.json
index e83708dd64d6..a5064c8fcb5e 100644
--- a/nixpkgs/doc/manpage-urls.json
+++ b/nixpkgs/doc/manpage-urls.json
@@ -4,6 +4,8 @@
   "mpd.conf(5)": "https://mpd.readthedocs.io/en/latest/mpd.conf.5.html",
   "nix.conf(5)": "https://nixos.org/manual/nix/stable/command-ref/conf-file.html",
 
+  "portals.conf(5)": "https://github.com/flatpak/xdg-desktop-portal/blob/1.18.1/doc/portals.conf.rst.in",
+
   "journald.conf(5)": "https://www.freedesktop.org/software/systemd/man/journald.conf.html",
   "logind.conf(5)": "https://www.freedesktop.org/software/systemd/man/logind.conf.html",
   "networkd.conf(5)": "https://www.freedesktop.org/software/systemd/man/networkd.conf.html",
diff --git a/nixpkgs/doc/packages/darwin-builder.section.md b/nixpkgs/doc/packages/darwin-builder.section.md
index 89c2445667dc..0c5323705345 100644
--- a/nixpkgs/doc/packages/darwin-builder.section.md
+++ b/nixpkgs/doc/packages/darwin-builder.section.md
@@ -94,7 +94,11 @@ $ sudo launchctl kickstart -k system/org.nixos.nix-daemon
       system = linuxSystem;
       modules = [
         "${nixpkgs}/nixos/modules/profiles/macos-builder.nix"
-        { virtualisation.host.pkgs = pkgs; }
+        { virtualisation = {
+            host.pkgs = pkgs;
+            darwin-builder.workingDirectory = "/var/lib/darwin-builder";
+          };
+        };
       ];
     };
   in {
diff --git a/nixpkgs/doc/packages/linux.section.md b/nixpkgs/doc/packages/linux.section.md
index b64da85791a0..0edf40043361 100644
--- a/nixpkgs/doc/packages/linux.section.md
+++ b/nixpkgs/doc/packages/linux.section.md
@@ -2,9 +2,21 @@
 
 The Nix expressions to build the Linux kernel are in [`pkgs/os-specific/linux/kernel`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/kernel).
 
-The function that builds the kernel has an argument `kernelPatches` which should be a list of `{name, patch, extraConfig}` attribute sets, where `name` is the name of the patch (which is included in the kernel’s `meta.description` attribute), `patch` is the patch itself (possibly compressed), and `extraConfig` (optional) is a string specifying extra options to be concatenated to the kernel configuration file (`.config`).
+The function [`pkgs.buildLinux`](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/generic.nix) builds a kernel with [common configuration values](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/common-config.nix).
+This is the preferred option unless you have a very specific use case.
+Most kernels packaged in Nixpkgs are built that way, and it will also generate kernels suitable for NixOS.
+[`pkgs.linuxManualConfig`](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/manual-config.nix) requires a complete configuration to be passed.
+It has fewer additional features than `pkgs.buildLinux`, which provides common configuration values and exposes the `features` attribute, as explained below.
 
-The kernel derivation exports an attribute `features` specifying whether optional functionality is or isn’t enabled. This is used in NixOS to implement kernel-specific behaviour. For instance, if the kernel has the `iwlwifi` feature (i.e., has built-in support for Intel wireless chipsets), then NixOS doesn’t have to build the external `iwlwifi` package:
+Both functions have an argument `kernelPatches` which should be a list of `{name, patch, extraConfig}` attribute sets, where `name` is the name of the patch (which is included in the kernel’s `meta.description` attribute), `patch` is the patch itself (possibly compressed), and `extraConfig` (optional) is a string specifying extra options to be concatenated to the kernel configuration file (`.config`).
+
+The kernel derivation created with `pkgs.buildLinux` exports an attribute `features` specifying whether optional functionality is or isn’t enabled. This is used in NixOS to implement kernel-specific behaviour.
+
+:::{.example #ex-skip-package-from-kernel-feature}
+
+# Skipping an external package because of a kernel feature
+
+For instance, if the kernel has the `iwlwifi` feature (i.e., has built-in support for Intel wireless chipsets), then NixOS doesn’t have to build the external `iwlwifi` package:
 
 ```nix
 modulesTree = [kernel]
@@ -12,30 +24,104 @@ modulesTree = [kernel]
   ++ ...;
 ```
 
-How to add a new (major) version of the Linux kernel to Nixpkgs:
+:::
+
+If you are using a kernel packaged in Nixpkgs, you can customize it by overriding its arguments. For details on how each argument affects the generated kernel, refer to [the `pkgs.buildLinux` source code](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/generic.nix).
+
+:::{.example #ex-overriding-kernel-derivation}
+
+# Overriding the kernel derivation
+
+Assuming you are using the kernel from `pkgs.linux_latest`:
+
+```nix
+pkgs.linux_latest.override {
+  ignoreConfigErrors = true;
+  autoModules = false;
+  kernelPreferBuiltin = true;
+  extraStructuredConfig = with lib.kernel; {
+    DEBUG_KERNEL = yes;
+    FRAME_POINTER = yes;
+    KGDB = yes;
+    KGDB_SERIAL_CONSOLE = yes;
+    DEBUG_INFO = yes;
+  };
+}
+```
+
+:::
 
-1.  Copy the old Nix expression (e.g., `linux-2.6.21.nix`) to the new one (e.g., `linux-2.6.22.nix`) and update it.
+## Manual kernel configuration {#sec-manual-kernel-configuration}
 
-2.  Add the new kernel to the `kernels` attribute set in `linux-kernels.nix` (e.g., create an attribute `kernel_2_6_22`).
+Sometimes it may not be desirable to use kernels built with `pkgs.buildLinux`, especially if most of the common configuration has to be altered or disabled to achieve a kernel as expected by the target use case.
+An example of this is building a kernel for use in a VM or micro VM. You can use `pkgs.linuxManualConfig` in these cases. It requires the `src`, `version`, and `configfile` attributes to be specified.
 
-3.  Now we’re going to update the kernel configuration. First unpack the kernel. Then for each supported platform (`i686`, `x86_64`, `uml`) do the following:
+:::{.example #ex-using-linux-manual-config}
 
-    1.  Make a copy from the old config (e.g., `config-2.6.21-i686-smp`) to the new one (e.g., `config-2.6.22-i686-smp`).
+# Using `pkgs.linuxManualConfig` with a specific source, version, and config file
 
-    2.  Copy the config file for this platform (e.g., `config-2.6.22-i686-smp`) to `.config` in the kernel source tree.
+```nix
+{ pkgs, ... }: {
+  version = "6.1.55";
+  src = pkgs.fetchurl {
+    url = "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-${version}.tar.xz";
+    hash = "sha256:1h0mzx52q9pvdv7rhnvb8g68i7bnlc9rf8gy9qn4alsxq4g28zm8";
+  };
+  configfile = ./path_to_config_file;
+  linux = pkgs.linuxManualConfig {
+    inherit version src configfile;
+    allowImportFromDerivation = true;
+  };
+}
+```
+
+If necessary, the version string can be slightly modified to explicitly mark it as a custom version. If you do so, ensure the `modDirVersion` attribute matches the source's version, otherwise the build will fail.
+
+```nix
+{ pkgs, ... }: {
+  version = "6.1.55-custom";
+  modDirVersion = "6.1.55";
+  src = pkgs.fetchurl {
+    url = "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-${modDirVersion}.tar.xz";
+    hash = "sha256:1h0mzx52q9pvdv7rhnvb8g68i7bnlc9rf8gy9qn4alsxq4g28zm8";
+  };
+  configfile = ./path_to_config_file;
+  linux = pkgs.linuxManualConfig {
+    inherit version modDirVersion src configfile;
+    allowImportFromDerivation = true;
+  };
+}
+```
 
-    3.  Run `make oldconfig ARCH={i386,x86_64,um}` and answer all questions. (For the uml configuration, also add `SHELL=bash`.) Make sure to keep the configuration consistent between platforms (i.e., don’t enable some feature on `i686` and disable it on `x86_64`).
+:::
 
-    4.  If needed, you can also run `make menuconfig`:
+Additional attributes can be used with `linuxManualConfig` for further customisation. You're encouraged to read [the `pkgs.linuxManualConfig` source code](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/manual-config.nix) to understand how to use them.
 
-        ```ShellSession
-        $ nix-env -f "<nixpkgs>" -iA ncurses
-        $ export NIX_CFLAGS_LINK=-lncurses
-        $ make menuconfig ARCH=arch
-        ```
+To edit the `.config` file for Linux X.Y from within Nix, proceed as follows:
 
-    5.  Copy `.config` over the new config file (e.g., `config-2.6.22-i686-smp`).
+```ShellSession
+$ nix-shell '<nixpkgs>' -A linuxKernel.kernels.linux_X_Y.configEnv
+$ unpackPhase
+$ cd linux-*
+$ make nconfig
+```
+
+## Developing kernel modules {#sec-linux-kernel-developing-modules}
+
+When developing kernel modules it's often convenient to run the edit-compile-run loop as quickly as possible.
+See the snippet below as an example.
 
-4.  Test building the kernel: `nix-build -A linuxKernel.kernels.kernel_2_6_22`. If it compiles, ship it! For extra credit, try booting NixOS with it.
+:::{.example #ex-edit-compile-run-kernel-modules}
+
+# Edit-compile-run loop when developing `mellanox` drivers
+
+```ShellSession
+$ nix-build '<nixpkgs>' -A linuxPackages.kernel.dev
+$ nix-shell '<nixpkgs>' -A linuxPackages.kernel
+$ unpackPhase
+$ cd linux-*
+$ make -C $dev/lib/modules/*/build M=$(pwd)/drivers/net/ethernet/mellanox modules
+# insmod ./drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko
+```
 
-5.  It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the `linuxPackagesFor` function in `linux-kernels.nix` (such as the NVIDIA drivers, AUFS, etc.). If the updated packages aren’t backwards compatible with older kernels, you may need to keep the older versions around.
+:::