Merge branch 'nixos-unstable' of https://github.com/NixOS/nixpkgs

3 files changed, 211 insertions, 22 deletions

diff --git a/nixpkgs/doc/languages-frameworks/php.section.md b/nixpkgs/doc/languages-frameworks/php.section.md
index 6c4315f5c487..2ca55aef1eff 100644
--- a/nixpkgs/doc/languages-frameworks/php.section.md
+++ b/nixpkgs/doc/languages-frameworks/php.section.md
@@ -130,6 +130,7 @@ package: a project may depend on certain extensions and `composer`
 won't work with that project unless those extensions are loaded.
 
 Example of building `composer` with additional extensions:
+
 ```nix
 (php.withExtensions ({ all, enabled }:
   enabled ++ (with all; [ imagick redis ]))
@@ -138,7 +139,9 @@ Example of building `composer` with additional extensions:
 
 ### Overriding PHP packages {#ssec-php-user-guide-overriding-packages}
 
-`php-packages.nix` form a scope, allowing us to override the packages defined within. For example, to apply a patch to a `mysqlnd` extension, you can simply pass an overlay-style function to `php`’s `packageOverrides` argument:
+`php-packages.nix` form a scope, allowing us to override the packages defined
+within. For example, to apply a patch to a `mysqlnd` extension, you can simply
+pass an overlay-style function to `php`’s `packageOverrides` argument:
 
 ```nix
 php.override {
@@ -153,3 +156,138 @@ php.override {
   };
 }
 ```
+
+### Building PHP projects {#ssec-building-php-projects}
+
+With [Composer](https://getcomposer.org/), you can effectively build PHP
+projects by streamlining dependency management. As the de-facto standard
+dependency manager for PHP, Composer enables you to declare and manage the
+libraries your project relies on, ensuring a more organized and efficient
+development process.
+
+Composer is not a package manager in the same sense as `Yum` or `Apt` are. Yes,
+it deals with "packages" or libraries, but it manages them on a per-project
+basis, installing them in a directory (e.g. `vendor`) inside your project. By
+default, it does not install anything globally. This idea is not new and
+Composer is strongly inspired by node's `npm` and ruby's `bundler`.
+
+Currently, there is no other PHP tool that offers the same functionality as
+Composer. Consequently, incorporating a helper in Nix to facilitate building
+such applications is a logical choice.
+
+In a Composer project, dependencies are defined in a `composer.json` file,
+while their specific versions are locked in a `composer.lock` file. Some
+Composer-based projects opt to include this `composer.lock` file in their source
+code, while others choose not to.
+
+In Nix, there are multiple approaches to building a Composer-based project.
+
+One such method is the `php.buildComposerProject` helper function, which serves
+as a wrapper around `mkDerivation`.
+
+Using this function, you can build a PHP project that includes both a
+`composer.json` and `composer.lock` file. If the project specifies binaries
+using the `bin` attribute in `composer.json`, these binaries will be
+automatically linked and made accessible in the derivation. In this context,
+"binaries" refer to PHP scripts that are intended to be executable.
+
+To use the helper effectively, simply add the `vendorHash` attribute, which
+enables the wrapper to handle the heavy lifting.
+
+Internally, the helper operates in three stages:
+
+1. It constructs a `composerRepository` attribute derivation by creating a
+   composer repository on the filesystem containing dependencies specified in
+   `composer.json`. This process uses the function
+   `php.mkComposerRepository` which in turn uses the
+   `php.composerHooks.composerRepositoryHook` hook. Internaly this function uses
+   a custom
+   [Composer plugin](https://github.com/nix-community/composer-local-repo-plugin) to
+   generate the repository.
+2. The resulting `composerRepository` derivation is then used by the
+   `php.composerHooks.composerInstallHook` hook, which is responsible for
+   creating the final `vendor` directory.
+3. Any "binary" specified in the `composer.json` are linked and made accessible
+   in the derivation.
+
+As the autoloader optimization can be activated directly within the
+`composer.json` file, we do not enable any autoloader optimization flags.
+
+To customize the PHP version, you can specify the `php` attribute. Similarly, if
+you wish to modify the Composer version, use the `composer` attribute. It is
+important to note that both attributes should be of the `derivation` type.
+
+Here's an example of working code example using `php.buildComposerProject`:
+
+```nix
+{ php, fetchFromGitHub }:
+
+php.buildComposerProject (finalAttrs: {
+  pname = "php-app";
+  version = "1.0.0";
+
+  src = fetchFromGitHub {
+    owner = "git-owner";
+    repo = "git-repo";
+    rev = finalAttrs.version;
+    hash = "sha256-VcQRSss2dssfkJ+iUb5qT+FJ10GHiFDzySigcmuVI+8=";
+  };
+
+  # PHP version containing the `ast` extension enabled
+  php = php.buildEnv {
+    extensions = ({ enabled, all }: enabled ++ (with all; [
+      ast
+    ]));
+  };
+
+  # The composer vendor hash
+  vendorHash = "sha256-86s/F+/5cBAwBqZ2yaGRM5rTGLmou5//aLRK5SA0WiQ=";
+
+  # If the composer.lock file is missing from the repository, add it:
+  # composerLock = ./path/to/composer.lock;
+})
+```
+
+In case the file `composer.lock` is missing from the repository, it is possible
+to specify it using the `composerLock` attribute.
+
+The other method is to use all these methods and hooks individually. This has
+the advantage of building a PHP library within another derivation very easily
+when necessary.
+
+Here's a working code example to build a PHP library using `mkDerivation` and
+separate functions and hooks:
+
+```nix
+{ stdenvNoCC, fetchFromGitHub, php }:
+
+stdenvNoCC.mkDerivation (finalAttrs:
+let
+  src = fetchFromGitHub {
+    owner = "git-owner";
+    repo = "git-repo";
+    rev = finalAttrs.version;
+    hash = "sha256-VcQRSss2dssfkJ+iUb5qT+FJ10GHiFDzySigcmuVI+8=";
+  };
+in {
+  inherit src;
+  pname = "php-app";
+  version = "1.0.0";
+
+  buildInputs = [ php ];
+
+  nativeBuildInputs = [
+    php.packages.composer
+    # This hook will use the attribute `composerRepository`
+    php.composerHooks.composerInstallHook
+  ];
+
+  composerRepository = php.mkComposerRepository {
+    inherit (finalAttrs) src;
+    # Specifying a custom composer.lock since it is not present in the sources.
+    composerLock = ./composer.lock;
+    # The composer vendor hash
+    vendorHash = "sha256-86s/F+/5cBAwBqZ2yaGRM5rTGLmou5//aLRK5SA0WiQ=";
+  };
+})
+```
diff --git a/nixpkgs/doc/languages-frameworks/python.section.md b/nixpkgs/doc/languages-frameworks/python.section.md
index 2ced6d69edd5..ca0513fbde83 100644
--- a/nixpkgs/doc/languages-frameworks/python.section.md
+++ b/nixpkgs/doc/languages-frameworks/python.section.md
@@ -101,7 +101,7 @@ The following is an example:
 buildPythonPackage rec {
   pname = "pytest";
   version = "3.3.1";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
@@ -167,12 +167,15 @@ following are specific to `buildPythonPackage`:
 * `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
 * `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
   variable in wrapped programs.
-* `format ? "setuptools"`: Format of the source. Valid options are
-  `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`.
-  `"setuptools"` is for when the source has a `setup.py` and `setuptools` is
-  used to build a wheel, `flit`, in case `flit` should be used to build a wheel,
-  and `wheel` in case a wheel is provided. Use `other` when a custom
-  `buildPhase` and/or `installPhase` is needed.
+* `pyproject`: Whether the pyproject format should be used. When set to `true`,
+  `pypaBuildHook` will be used, and you can add the required build dependencies
+  from `build-system.requires` to `nativeBuildInputs`. Note that the pyproject
+  format falls back to using `setuptools`, so you can use `pyproject = true`
+  even if the package only has a `setup.py`. When set to `false`, you can
+  use the existing [hooks](#setup-hooks0 or provide your own logic to build the
+  package. This can be useful for packages that don't support the pyproject
+  format. When unset, the legacy `setuptools` hooks are used for backwards
+  compatibility.
 * `makeWrapperArgs ? []`: A list of strings. Arguments to be passed to
   `makeWrapper`, which wraps generated binaries. By default, the arguments to
   `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling
@@ -286,20 +289,25 @@ specifying an interpreter version), like this:
 python3.pkgs.buildPythonApplication rec {
   pname = "luigi";
   version = "2.7.9";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash  = "sha256-Pe229rT0aHwA98s+nTHQMEFKZPo/yw6sot8MivFDvAw=";
   };
 
+  nativeBuildInputs = [
+    python3.pkgs.setuptools
+    python3.pkgs.wheel
+  ];
+
   propagatedBuildInputs = with python3.pkgs; [
     tornado
     python-daemon
   ];
 
   meta = with lib; {
-    ...
+    # ...
   };
 }
 ```
@@ -716,8 +724,8 @@ We've now seen how to create an ad-hoc temporary shell session, and how to
 create a single script with Python dependencies, but in the course of normal
 development we're usually working in an entire package repository.
 
-As explained in the Nix manual, `nix-shell` can also load an expression from a
-`.nix` file. Say we want to have Python 3.11, `numpy` and `toolz`, like before,
+As explained [in the `nix-shell` section](https://nixos.org/manual/nix/stable/command-ref/nix-shell) of the Nix manual, `nix-shell` can also load an expression from a `.nix` file.
+Say we want to have Python 3.11, `numpy` and `toolz`, like before,
 in an environment. We can add a `shell.nix` file describing our dependencies:
 
 ```nix
@@ -858,18 +866,25 @@ building Python libraries is `buildPythonPackage`. Let's see how we can build th
 { lib
 , buildPythonPackage
 , fetchPypi
+, setuptools
+, wheel
 }:
 
 buildPythonPackage rec {
   pname = "toolz";
   version = "0.10.0";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
   };
 
+  nativeBuildInputs = [
+    setuptools
+    wheel
+  ];
+
   # has no tests
   doCheck = false;
 
@@ -918,13 +933,18 @@ with import <nixpkgs> {};
     my_toolz = python311.pkgs.buildPythonPackage rec {
       pname = "toolz";
       version = "0.10.0";
-      format = "setuptools";
+      pyproject = true;
 
       src = fetchPypi {
         inherit pname version;
         hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
       };
 
+      nativeBuildInputs = [
+        python311.pkgs.setuptools
+        python311.pkgs.wheel
+      ];
+
       # has no tests
       doCheck = false;
 
@@ -972,6 +992,9 @@ order to build [`datashape`](https://github.com/blaze/datashape).
 , buildPythonPackage
 , fetchPypi
 
+# build dependencies
+, setuptools, wheel
+
 # dependencies
 , numpy, multipledispatch, python-dateutil
 
@@ -982,13 +1005,18 @@ order to build [`datashape`](https://github.com/blaze/datashape).
 buildPythonPackage rec {
   pname = "datashape";
   version = "0.4.7";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash = "sha256-FLLvdm1MllKrgTGC6Gb0k0deZeVYvtCCLji/B7uhong=";
   };
 
+  nativeBuildInputs = [
+    setuptools
+    wheel
+  ];
+
   propagatedBuildInputs = [
     multipledispatch
     numpy
@@ -1023,6 +1051,8 @@ when building the bindings and are therefore added as `buildInputs`.
 { lib
 , buildPythonPackage
 , fetchPypi
+, setuptools
+, wheel
 , libxml2
 , libxslt
 }:
@@ -1030,13 +1060,18 @@ when building the bindings and are therefore added as `buildInputs`.
 buildPythonPackage rec {
   pname = "lxml";
   version = "3.4.4";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash = "sha256-s9NiusRxFydHzaNRMjjxFcvWxfi45jGb9ql6eJJyQJk=";
   };
 
+  nativeBuildInputs = [
+    setuptools
+    wheel
+  ];
+
   buildInputs = [
     libxml2
     libxslt
@@ -1067,6 +1102,10 @@ therefore we have to set `LDFLAGS` and `CFLAGS`.
 , buildPythonPackage
 , fetchPypi
 
+# build dependencies
+, setuptools
+, wheel
+
 # dependencies
 , fftw
 , fftwFloat
@@ -1078,13 +1117,18 @@ therefore we have to set `LDFLAGS` and `CFLAGS`.
 buildPythonPackage rec {
   pname = "pyFFTW";
   version = "0.9.2";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash = "sha256-9ru2r6kwhUCaskiFoaPNuJCfCVoUL01J40byvRt4kHQ=";
   };
 
+  nativeBuildInputs = [
+    setuptools
+    wheel
+  ];
+
   buildInputs = [
     fftw
     fftwFloat
@@ -1334,9 +1378,7 @@ instead of a dev dependency).
 
 Keep in mind that while the examples above are done with `requirements.txt`,
 `pythonRelaxDepsHook` works by modifying the resulting wheel file, so it should
-work in any of the formats supported by `buildPythonPackage` currently,
-with the exception of `other` (see `format` in
-[`buildPythonPackage` parameters](#buildpythonpackage-parameters) for more details).
+work with any of the existing [hooks](#setup-hooks).
 
 #### Using unittestCheckHook {#using-unittestcheckhook}
 
@@ -1461,18 +1503,26 @@ We first create a function that builds `toolz` in `~/path/to/toolz/release.nix`
 ```nix
 { lib
 , buildPythonPackage
+, fetchPypi
+, setuptools
+, wheel
 }:
 
 buildPythonPackage rec {
   pname = "toolz";
   version = "0.10.0";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
     hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
   };
 
+  nativeBuildInputs = [
+    setuptools
+    wheel
+  ];
+
   meta = with lib; {
     changelog = "https://github.com/pytoolz/toolz/releases/tag/${version}";
     homepage = "https://github.com/pytoolz/toolz/";
diff --git a/nixpkgs/doc/languages-frameworks/ruby.section.md b/nixpkgs/doc/languages-frameworks/ruby.section.md
index f1953500fa32..3a4c94de9292 100644
--- a/nixpkgs/doc/languages-frameworks/ruby.section.md
+++ b/nixpkgs/doc/languages-frameworks/ruby.section.md
@@ -32,7 +32,8 @@ Again, it's possible to launch the interpreter from the shell. The Ruby interpre
 
 #### Load Ruby environment from `.nix` expression {#load-ruby-environment-from-.nix-expression}
 
-As explained in the Nix manual, `nix-shell` can also load an expression from a `.nix` file. Say we want to have Ruby 2.6, `nokogori`, and `pry`. Consider a `shell.nix` file with:
+As explained [in the `nix-shell` section](https://nixos.org/manual/nix/stable/command-ref/nix-shell) of the Nix manual, `nix-shell` can also load an expression from a `.nix` file.
+Say we want to have Ruby 2.6, `nokogori`, and `pry`. Consider a `shell.nix` file with:
 
 ```nix
 with import <nixpkgs> {};