diff options
Diffstat (limited to 'nixpkgs/doc/build-helpers/images/dockertools.section.md')
| -rw-r--r-- | nixpkgs/doc/build-helpers/images/dockertools.section.md | 881 |
1 files changed, 718 insertions, 163 deletions
diff --git a/nixpkgs/doc/build-helpers/images/dockertools.section.md b/nixpkgs/doc/build-helpers/images/dockertools.section.md index e732e0472926..9317146b8f94 100644 --- a/nixpkgs/doc/build-helpers/images/dockertools.section.md +++ b/nixpkgs/doc/build-helpers/images/dockertools.section.md @@ -676,6 +676,7 @@ If our package sets `includeStorePaths` to `false`, we'll end up with only the f dockerTools.streamLayeredImage { name = "hello"; contents = [ hello ]; + includeStorePaths = false; } ``` @@ -714,301 +715,855 @@ dockerTools.streamLayeredImage { ``` ::: -## pullImage {#ssec-pkgs-dockerTools-fetchFromRegistry} +[]{#ssec-pkgs-dockerTools-fetchFromRegistry} +## pullImage {#ssec-pkgs-dockerTools-pullImage} -This function is analogous to the `docker pull` command, in that it can be used to pull a Docker image from a Docker registry. By default [Docker Hub](https://hub.docker.com/) is used to pull images. +This function is similar to the `docker pull` command, which means it can be used to pull a Docker image from a registry that implements the [Docker Registry HTTP API V2](https://distribution.github.io/distribution/spec/api/). +By default, the `docker.io` registry is used. -Its parameters are described in the example below: +The image will be downloaded as an uncompressed Docker-compatible repository tarball, which is suitable for use with other `dockerTools` functions such as [`buildImage`](#ssec-pkgs-dockerTools-buildImage), [`buildLayeredImage`](#ssec-pkgs-dockerTools-buildLayeredImage), and [`streamLayeredImage`](#ssec-pkgs-dockerTools-streamLayeredImage). + +This function requires two different types of hashes/digests to be specified: + +- One of them is used to identify a unique image within the registry (see the documentation for the `imageDigest` attribute). +- The other is used by Nix to ensure the contents of the output haven't changed (see the documentation for the `sha256` attribute). + +Both hashes are required because they must uniquely identify some content in two completely different systems (the Docker registry and the Nix store), but their values will not be the same. +See [](#ex-dockerTools-pullImage-nixprefetchdocker) for a tool that can help gather these values. + +### Inputs {#ssec-pkgs-dockerTools-pullImage-inputs} + +`pullImage` expects a single argument with the following attributes: + +`imageName` (String) + +: Specifies the name of the image to be downloaded, as well as the registry endpoint. + By default, the `docker.io` registry is used. + To specify a different registry, prepend the endpoint to `imageName`, separated by a slash (`/`). + See [](#ex-dockerTools-pullImage-differentregistry) for how to do that. + +`imageDigest` (String) + +: Specifies the digest of the image to be downloaded. + + :::{.tip} + **Why can't I specify a tag to pull from, and have to use a digest instead?** + + Tags are often updated to point to different image contents. + The most common example is the `latest` tag, which is usually updated whenever a newer image version is available. + + An image tag isn't enough to guarantee the contents of an image won't change, but a digest guarantees this. + Providing a digest helps ensure that you will still be able to build the same Nix code and get the same output even if newer versions of an image are released. + ::: + +`sha256` (String) + +: The hash of the image after it is downloaded. + Internally, this is passed to the [`outputHash`](https://nixos.org/manual/nix/stable/language/advanced-attributes#adv-attr-outputHash) attribute of the resulting derivation. + This is needed to provide a guarantee to Nix that the contents of the image haven't changed, because Nix doesn't support the value in `imageDigest`. + +`finalImageName` (String; _optional_) + +: Specifies the name that will be used for the image after it has been downloaded. + This only applies after the image is downloaded, and is not used to identify the image to be downloaded in the registry. + Use `imageName` for that instead. + + _Default value:_ the same value specified in `imageName`. + +`finalImageTag` (String; _optional_) + +: Specifies the tag that will be used for the image after it has been downloaded. + This only applies after the image is downloaded, and is not used to identify the image to be downloaded in the registry. + + _Default value:_ `"latest"`. + +`os` (String; _optional_) + +: Specifies the operating system of the image to pull. + If specified, its value should follow the [OCI Image Configuration Specification](https://github.com/opencontainers/image-spec/blob/main/config.md#properties), which should still be compatible with Docker. + According to the linked specification, all possible values for `$GOOS` in [the Go docs](https://go.dev/doc/install/source#environment) should be valid, but will commonly be one of `darwin` or `linux`. + + _Default value:_ `"linux"`. + +`arch` (String; _optional_) + +: Specifies the architecture of the image to pull. + If specified, its value should follow the [OCI Image Configuration Specification](https://github.com/opencontainers/image-spec/blob/main/config.md#properties), which should still be compatible with Docker. + According to the linked specification, all possible values for `$GOARCH` in [the Go docs](https://go.dev/doc/install/source#environment) should be valid, but will commonly be one of `386`, `amd64`, `arm`, or `arm64`. + + _Default value:_ the same value from `pkgs.go.GOARCH`. + +`tlsVerify` (Boolean; _optional_) + +: Used to enable or disable HTTPS and TLS certificate verification when communicating with the chosen Docker registry. + Setting this to `false` will make `pullImage` connect to the registry through HTTP. + + _Default value:_ `true`. + +`name` (String; _optional_) + +: The name used for the output in the Nix store path. + + _Default value:_ a value derived from `finalImageName` and `finalImageTag`, with some symbols replaced. + It is recommended to treat the default as an opaque value. + +### Examples {#ssec-pkgs-dockerTools-pullImage-examples} + +::: {.example #ex-dockerTools-pullImage-niximage} +# Pulling the nixos/nix Docker image from the default registry + +This example pulls the [`nixos/nix` image](https://hub.docker.com/r/nixos/nix) and saves it in the Nix store. ```nix -pullImage { +{ dockerTools }: +dockerTools.pullImage { imageName = "nixos/nix"; - imageDigest = - "sha256:473a2b527958665554806aea24d0131bacec46d23af09fef4598eeab331850fa"; + imageDigest = "sha256:b8ea88f763f33dfda2317b55eeda3b1a4006692ee29e60ee54ccf6d07348c598"; finalImageName = "nix"; - finalImageTag = "2.11.1"; - sha256 = "sha256-qvhj+Hlmviz+KEBVmsyPIzTB3QlVAFzwAY1zDPIBGxc="; - os = "linux"; - arch = "x86_64"; + finalImageTag = "2.19.3"; + sha256 = "zRwlQs1FiKrvHPaf8vWOR/Tlp1C5eLn1d9pE4BZg3oA="; } ``` +::: -- `imageName` specifies the name of the image to be downloaded, which can also include the registry namespace (e.g. `nixos`). This argument is required. - -- `imageDigest` specifies the digest of the image to be downloaded. This argument is required. +::: {.example #ex-dockerTools-pullImage-differentregistry} +# Pulling the nixos/nix Docker image from a specific registry -- `finalImageName`, if specified, this is the name of the image to be created. Note it is never used to fetch the image since we prefer to rely on the immutable digest ID. By default it's equal to `imageName`. +This example pulls the [`coreos/etcd` image](https://quay.io/repository/coreos/etcd) from the `quay.io` registry. -- `finalImageTag`, if specified, this is the tag of the image to be created. Note it is never used to fetch the image since we prefer to rely on the immutable digest ID. By default it's `latest`. +```nix +{ dockerTools }: +dockerTools.pullImage { + imageName = "quay.io/coreos/etcd"; + imageDigest = "sha256:24a23053f29266fb2731ebea27f915bb0fb2ae1ea87d42d890fe4e44f2e27c5d"; + finalImageName = "etcd"; + finalImageTag = "v3.5.11"; + sha256 = "Myw+85f2/EVRyMB3axECdmQ5eh9p1q77FWYKy8YpRWU="; +} +``` +::: -- `sha256` is the checksum of the whole fetched image. This argument is required. +::: {.example #ex-dockerTools-pullImage-nixprefetchdocker} +# Finding the digest and hash values to use for `dockerTools.pullImage` -- `os`, if specified, is the operating system of the fetched image. By default it's `linux`. +Since [`dockerTools.pullImage`](#ssec-pkgs-dockerTools-pullImage) requires two different hashes, one can run the `nix-prefetch-docker` tool to find out the values for the hashes. +The tool outputs some text for an attribute set which you can pass directly to `pullImage`. -- `arch`, if specified, is the cpu architecture of the fetched image. By default it's `x86_64`. +```shell +$ nix run nixpkgs#nix-prefetch-docker -- --image-name nixos/nix --image-tag 2.19.3 --arch amd64 --os linux +(some output removed for clarity) +Writing manifest to image destination +-> ImageName: nixos/nix +-> ImageDigest: sha256:498fa2d7f2b5cb3891a4edf20f3a8f8496e70865099ba72540494cd3e2942634 +-> FinalImageName: nixos/nix +-> FinalImageTag: latest +-> ImagePath: /nix/store/4mxy9mn6978zkvlc670g5703nijsqc95-docker-image-nixos-nix-latest.tar +-> ImageHash: 1q6cf2pdrasa34zz0jw7pbs6lvv52rq2aibgxccbwcagwkg2qj1q +{ + imageName = "nixos/nix"; + imageDigest = "sha256:498fa2d7f2b5cb3891a4edf20f3a8f8496e70865099ba72540494cd3e2942634"; + sha256 = "1q6cf2pdrasa34zz0jw7pbs6lvv52rq2aibgxccbwcagwkg2qj1q"; + finalImageName = "nixos/nix"; + finalImageTag = "latest"; +} +``` -`nix-prefetch-docker` command can be used to get required image parameters: +It is important to supply the `--arch` and `--os` arguments to `nix-prefetch-docker` to filter to a single image, in case there are multiple architectures and/or operating systems supported by the image name and tags specified. +By default, `nix-prefetch-docker` will set `os` to `linux` and `arch` to `amd64`. -```ShellSession -$ nix run nixpkgs#nix-prefetch-docker -- --image-name mysql --image-tag 5 +Run `nix-prefetch-docker --help` for a list of all supported arguments: +```shell +$ nix run nixpkgs#nix-prefetch-docker -- --help +(output removed for clarity) ``` +::: + +## exportImage {#ssec-pkgs-dockerTools-exportImage} -Since a given `imageName` may transparently refer to a manifest list of images which support multiple architectures and/or operating systems, you can supply the `--os` and `--arch` arguments to specify exactly which image you want. By default it will match the OS and architecture of the host the command is run on. +This function is similar to the `docker container export` command, which means it can be used to export an image's filesystem as an uncompressed tarball archive. +The difference is that `docker container export` is applied to containers, but `dockerTools.exportImage` applies to Docker images. +The resulting archive will not contain any image metadata (such as command to run with `docker container run`), only the filesystem contents. -```ShellSession -$ nix-prefetch-docker --image-name mysql --image-tag 5 --arch x86_64 --os linux +You can use this function to import an archive in Docker with `docker image import`. +See [](#ex-dockerTools-exportImage-importingDocker) to understand how to do that. + +:::{.caution} +`exportImage` works by unpacking the given image inside a VM. +Because of this, using this function requires the `kvm` device to be available, see [`system-features`](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-system-features). +::: + +### Inputs {#ssec-pkgs-dockerTools-exportImage-inputs} + +`exportImage` expects an argument with the following attributes: + +`fromImage` (Attribute Set or String) + +: The repository tarball of the image whose filesystem will be exported. + It must be a valid Docker image, such as one exported by `docker image save`, or another image built with the `dockerTools` utility functions. + + If `name` is not specified, `fromImage` must be an Attribute Set corresponding to a derivation, i.e. it can't be a path to a tarball. + If `name` is specified, `fromImage` can be either an Attribute Set corresponding to a derivation or simply a path to a tarball. + + See [](#ex-dockerTools-exportImage-naming) and [](#ex-dockerTools-exportImage-fromImagePath) to understand the connection between `fromImage`, `name`, and the name used for the output of `exportImage`. + +`fromImageName` (String or Null; _optional_) + +: Used to specify the image within the repository tarball in case it contains multiple images. + A value of `null` means that `exportImage` will use the first image available in the repository. + + :::{.note} + This must be used with `fromImageTag`. Using only `fromImageName` without `fromImageTag` will make `exportImage` use the first image available in the repository. + ::: + + _Default value:_ `null`. + +`fromImageTag` (String or Null; _optional_) + +: Used to specify the image within the repository tarball in case it contains multiple images. + A value of `null` means that `exportImage` will use the first image available in the repository. + + :::{.note} + This must be used with `fromImageName`. Using only `fromImageTag` without `fromImageName` will make `exportImage` use the first image available in the repository + ::: + + _Default value:_ `null`. + +`diskSize` (Number; _optional_) + +: Controls the disk size (in megabytes) of the VM used to unpack the image. + + _Default value:_ 1024. + +`name` (String; _optional_) + +: The name used for the output in the Nix store path. + + _Default value:_ the value of `fromImage.name`. + +### Examples {#ssec-pkgs-dockerTools-exportImage-examples} + +:::{.example #ex-dockerTools-exportImage-hello} +# Exporting a Docker image with `dockerTools.exportImage` + +This example first builds a layered image with [`dockerTools.buildLayeredImage`](#ssec-pkgs-dockerTools-buildLayeredImage), and then exports its filesystem with `dockerTools.exportImage`. + +```nix +{ dockerTools, hello }: +dockerTools.exportImage { + name = "hello"; + fromImage = dockerTools.buildLayeredImage { + name = "hello"; + contents = [ hello ]; + }; +} ``` -Desired image name and tag can be set using `--final-image-name` and `--final-image-tag` arguments: +When building the package above, we can see the layers of the Docker image being unpacked to produce the final output: -```ShellSession -$ nix-prefetch-docker --image-name mysql --image-tag 5 --final-image-name eu.gcr.io/my-project/mysql --final-image-tag prod +```shell +$ nix-build +(some output removed for clarity) +Unpacking base image... +From-image name or tag wasn't set. Reading the first ID. +Unpacking layer 5731199219418f175d1580dbca05677e69144425b2d9ecb60f416cd57ca3ca42/layer.tar +tar: Removing leading `/' from member names +Unpacking layer e2897bf34bb78c4a65736510204282d9f7ca258ba048c183d665bd0f3d24c5ec/layer.tar +tar: Removing leading `/' from member names +Unpacking layer 420aa5876dca4128cd5256da7dea0948e30ef5971712f82601718cdb0a6b4cda/layer.tar +tar: Removing leading `/' from member names +Unpacking layer ea5f4e620e7906c8ecbc506b5e6f46420e68d4b842c3303260d5eb621b5942e5/layer.tar +tar: Removing leading `/' from member names +Unpacking layer 65807b9abe8ab753fa97da8fb74a21fcd4725cc51e1b679c7973c97acd47ebcf/layer.tar +tar: Removing leading `/' from member names +Unpacking layer b7da2076b60ebc0ea6824ef641978332b8ac908d47b2d07ff31b9cc362245605/layer.tar +Executing post-mount steps... +Packing raw image... +[ 1.660036] reboot: Power down +/nix/store/x6a5m7c6zdpqz1d8j7cnzpx9glzzvd2h-hello ``` -## exportImage {#ssec-pkgs-dockerTools-exportImage} +The following command lists some of the contents of the output to verify that the structure of the archive is as expected: -This function is analogous to the `docker export` command, in that it can be used to flatten a Docker image that contains multiple layers. It is in fact the result of the merge of all the layers of the image. As such, the result is suitable for being imported in Docker with `docker import`. +```shell +$ tar --exclude '*/share/*' --exclude 'nix/store/*/*' -tvf /nix/store/x6a5m7c6zdpqz1d8j7cnzpx9glzzvd2h-hello +drwxr-xr-x root/0 0 1979-12-31 16:00 ./ +drwxr-xr-x root/0 0 1979-12-31 16:00 ./bin/ +lrwxrwxrwx root/0 0 1979-12-31 16:00 ./bin/hello -> /nix/store/h92a9jd0lhhniv2q417hpwszd4jhys7q-hello-2.12.1/bin/hello +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/05zbwhz8a7i2v79r9j21pl6m6cj0xi8k-libunistring-1.1/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/ayg5rhjhi9ic73hqw33mjqjxwv59ndym-xgcc-13.2.0-libgcc/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/h92a9jd0lhhniv2q417hpwszd4jhys7q-hello-2.12.1/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/m59xdgkgnjbk8kk6k6vbxmqnf82mk9s0-libidn2-2.3.4/ +dr-xr-xr-x root/0 0 1979-12-31 16:00 ./nix/store/p3jshbwxiwifm1py0yq544fmdyy98j8a-glibc-2.38-27/ +drwxr-xr-x root/0 0 1979-12-31 16:00 ./share/ +``` +::: -> **_NOTE:_** Using this function requires the `kvm` device to be available. +:::{.example #ex-dockerTools-exportImage-importingDocker} +# Importing an archive built with `dockerTools.exportImage` in Docker -The parameters of `exportImage` are the following: +We will use the same package from [](#ex-dockerTools-exportImage-hello) and import it into Docker. ```nix -exportImage { - fromImage = someLayeredImage; - fromImageName = null; - fromImageTag = null; +{ dockerTools, hello }: +dockerTools.exportImage { + name = "hello"; + fromImage = dockerTools.buildLayeredImage { + name = "hello"; + contents = [ hello ]; + }; +} +``` + +Building and importing it into Docker: + +```shell +$ nix-build +(output removed for clarity) +/nix/store/x6a5m7c6zdpqz1d8j7cnzpx9glzzvd2h-hello +$ docker image import /nix/store/x6a5m7c6zdpqz1d8j7cnzpx9glzzvd2h-hello +sha256:1d42dba415e9b298ea0decf6497fbce954de9b4fcb2984f91e307c8fedc1f52f +$ docker image ls +REPOSITORY TAG IMAGE ID CREATED SIZE +<none> <none> 1d42dba415e9 4 seconds ago 32.6MB +``` +::: - name = someLayeredImage.name; +:::{.example #ex-dockerTools-exportImage-naming} +# Exploring output naming with `dockerTools.exportImage` + +`exportImage` does not require a `name` attribute if `fromImage` is a derivation, which means that the following works: + +```nix +{ dockerTools, hello }: +dockerTools.exportImage { + fromImage = dockerTools.buildLayeredImage { + name = "hello"; + contents = [ hello ]; + }; } ``` -The parameters relative to the base image have the same synopsis as described in [buildImage](#ssec-pkgs-dockerTools-buildImage), except that `fromImage` is the only required argument in this case. +However, since [`dockerTools.buildLayeredImage`](#ssec-pkgs-dockerTools-buildLayeredImage)'s output ends with `.tar.gz`, the output of `exportImage` will also end with `.tar.gz`, even though the archive created with `exportImage` is uncompressed: + +```shell +$ nix-build +(output removed for clarity) +/nix/store/by3f40xvc4l6bkis74l0fj4zsy0djgkn-hello.tar.gz +$ file /nix/store/by3f40xvc4l6bkis74l0fj4zsy0djgkn-hello.tar.gz +/nix/store/by3f40xvc4l6bkis74l0fj4zsy0djgkn-hello.tar.gz: POSIX tar archive (GNU) +``` + +If the archive was actually compressed, the output of file would've mentioned that fact. +Because of this, it may be important to set a proper `name` attribute when using `exportImage` with other functions from `dockerTools`. +::: -The `name` argument is the name of the derivation output, which defaults to `fromImage.name`. +:::{.example #ex-dockerTools-exportImage-fromImagePath} +# Using `dockerTools.exportImage` with a path as `fromImage` -## Environment Helpers {#ssec-pkgs-dockerTools-helpers} +It is possible to use a path as the value of the `fromImage` attribute when calling `dockerTools.exportImage`. +However, when doing so, a `name` attribute **MUST** be specified, or you'll encounter an error when evaluating the Nix code. -Some packages expect certain files to be available globally. -When building an image from scratch (i.e. without `fromImage`), these files are missing. -`pkgs.dockerTools` provides some helpers to set up an environment with the necessary files. -You can include them in `copyToRoot` like this: +For this example, we'll assume a Docker tarball image named `image.tar.gz` exists in the same directory where our package is defined: ```nix -buildImage { - name = "environment-example"; - copyToRoot = with pkgs.dockerTools; [ - usrBinEnv - binSh - caCertificates - fakeNss - ]; +{ dockerTools }: +dockerTools.exportImage { + name = "filesystem.tar"; + fromImage = ./image.tar.gz; } ``` +Building this will give us the expected output: + +```shell +$ nix-build +(output removed for clarity) +/nix/store/w13l8h3nlkg0zv56k7rj0ai0l2zlf7ss-filesystem.tar +``` + +If you don't specify a `name` attribute, you'll encounter an evaluation error and the package won't build. +::: + +## Environment Helpers {#ssec-pkgs-dockerTools-helpers} + +When building Docker images with Nix, you might also want to add certain files that are expected to be available globally by the software you're packaging. +Simple examples are the `env` utility in `/usr/bin/env`, or trusted root TLS/SSL certificates. +Such files will most likely not be included if you're building a Docker image from scratch with Nix, and they might also not be included if you're starting from a Docker image that doesn't include them. +The helpers in this section are packages that provide some of these commonly-needed global files. + +Most of these helpers are packages, which means you have to add them to the list of contents to be included in the image (this changes depending on the function you're using to build the image). +[](#ex-dockerTools-helpers-buildImage) and [](#ex-dockerTools-helpers-buildLayeredImage) show how to include these packages on `dockerTools` functions that build an image. +For more details on how that works, see the documentation for the function you're using. + ### usrBinEnv {#sssec-pkgs-dockerTools-helpers-usrBinEnv} This provides the `env` utility at `/usr/bin/env`. +This is currently implemented by linking to the `env` binary from the `coreutils` package, but is considered an implementation detail that could change in the future. ### binSh {#sssec-pkgs-dockerTools-helpers-binSh} -This provides `bashInteractive` at `/bin/sh`. +This provides a `/bin/sh` link to the `bash` binary from the `bashInteractive` package. +Because of this, it supports cases such as running a command interactively in a container (for example by running `docker run -it <image_name>`). ### caCertificates {#sssec-pkgs-dockerTools-helpers-caCertificates} -This sets up `/etc/ssl/certs/ca-certificates.crt`. +This adds trusted root TLS/SSL certificates from the `cacert` package in multiple locations in an attempt to be compatible with binaries built for multiple Linux distributions. +The locations currently used are: +- `/etc/ssl/certs/ca-bundle.crt` +- `/etc/ssl/certs/ca-certificates.crt` +- `/etc/pki/tls/certs/ca-bundle.crt` + +[]{#ssec-pkgs-dockerTools-fakeNss} ### fakeNss {#sssec-pkgs-dockerTools-helpers-fakeNss} -Provides `/etc/passwd` and `/etc/group` that contain root and nobody. -Useful when packaging binaries that insist on using nss to look up -username/groups (like nginx). +This is a re-export of the `fakeNss` package from Nixpkgs. +See [](#sec-fakeNss). ### shadowSetup {#ssec-pkgs-dockerTools-shadowSetup} -This constant string is a helper for setting up the base files for managing users and groups, only if such files don't exist already. It is suitable for being used in a [`buildImage` `runAsRoot`](#ex-dockerTools-buildImage-runAsRoot) script for cases like in the example below: +This is a string containing a script that sets up files needed for [`shadow`](https://github.com/shadow-maint/shadow) to work (using the `shadow` package from Nixpkgs), and alters `PATH` to make all its utilities available in the same script. +It is intended to be used with other dockerTools functions in attributes that expect scripts. +After the script in `shadowSetup` runs, you'll then be able to add more commands that make use of the utilities in `shadow`, such as adding any extra users and/or groups. +See [](#ex-dockerTools-shadowSetup-buildImage) and [](#ex-dockerTools-shadowSetup-buildLayeredImage) to better understand how to use it. + +`shadowSetup` achieves a result similar to [`fakeNss`](#sssec-pkgs-dockerTools-helpers-fakeNss), but only sets up a `root` user with different values for the home directory and the shell to use, in addition to setting up files for [PAM](https://en.wikipedia.org/wiki/Linux_PAM) and a {manpage}`login.defs(5)` file. + +:::{.caution} +Using both `fakeNss` and `shadowSetup` at the same time will either cause your build to break or produce unexpected results. +Use either `fakeNss` or `shadowSetup` depending on your use case, but avoid using both. +::: + +:::{.note} +When used with [`buildLayeredImage`](#ssec-pkgs-dockerTools-buildLayeredImage) or [`streamLayeredImage`](#ssec-pkgs-dockerTools-streamLayeredImage), you will have to set the `enableFakechroot` attribute to `true`, or else the script in `shadowSetup` won't run properly. +See [](#ex-dockerTools-shadowSetup-buildLayeredImage). +::: + +### Examples {#ssec-pkgs-dockerTools-helpers-examples} + +:::{.example #ex-dockerTools-helpers-buildImage} +# Using `dockerTools`'s environment helpers with `buildImage` + +This example adds the [`binSh`](#sssec-pkgs-dockerTools-helpers-binSh) helper to a basic Docker image built with [`dockerTools.buildImage`](#ssec-pkgs-dockerTools-buildImage). +This helper makes it possible to enter a shell inside the container. +This is the `buildImage` equivalent of [](#ex-dockerTools-helpers-buildLayeredImage). ```nix -buildImage { +{ dockerTools, hello }: +dockerTools.buildImage { + name = "env-helpers"; + tag = "latest"; + + copyToRoot = [ + hello + dockerTools.binSh + ]; +``` + +After building the image and loading it in Docker, we can create a container based on it and enter a shell inside the container. +This is made possible by `binSh`. + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/2p0i3i04cgjlk71hsn7ll4kxaxxiv4qg-docker-image-env-helpers.tar.gz +$ docker load -i /nix/store/2p0i3i04cgjlk71hsn7ll4kxaxxiv4qg-docker-image-env-helpers.tar.gz +(output removed for clarity) +$ docker run --rm -it env-helpers:latest /bin/sh +sh-5.2# help +GNU bash, version 5.2.21(1)-release (x86_64-pc-linux-gnu) +(rest of output removed for clarity) +``` +::: + +:::{.example #ex-dockerTools-helpers-buildLayeredImage} +# Using `dockerTools`'s environment helpers with `buildLayeredImage` + +This example adds the [`binSh`](#sssec-pkgs-dockerTools-helpers-binSh) helper to a basic Docker image built with [`dockerTools.buildLayeredImage`](#ssec-pkgs-dockerTools-buildLayeredImage). +This helper makes it possible to enter a shell inside the container. +This is the `buildLayeredImage` equivalent of [](#ex-dockerTools-helpers-buildImage). + +```nix +{ dockerTools, hello }: +dockerTools.buildLayeredImage { + name = "env-helpers"; + tag = "latest"; + + contents = [ + hello + dockerTools.binSh + ]; + + config = { + Cmd = [ "/bin/hello" ]; + }; +} +``` + +After building the image and loading it in Docker, we can create a container based on it and enter a shell inside the container. +This is made possible by `binSh`. + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/rpf47f4z5b9qr4db4ach9yr4b85hjhxq-env-helpers.tar.gz +$ docker load -i /nix/store/rpf47f4z5b9qr4db4ach9yr4b85hjhxq-env-helpers.tar.gz +(output removed for clarity) +$ docker run --rm -it env-helpers:latest /bin/sh +sh-5.2# help +GNU bash, version 5.2.21(1)-release (x86_64-pc-linux-gnu) +(rest of output removed for clarity) +``` +::: + +:::{.example #ex-dockerTools-shadowSetup-buildImage} +# Using `dockerTools.shadowSetup` with `dockerTools.buildImage` + +This is an example that shows how to use `shadowSetup` with `dockerTools.buildImage`. +Note that the extra script in `runAsRoot` uses `groupadd` and `useradd`, which are binaries provided by the `shadow` package. +These binaries are added to the `PATH` by the `shadowSetup` script, but only for the duration of `runAsRoot`. + +```nix +{ dockerTools, hello }: +dockerTools.buildImage { name = "shadow-basic"; + tag = "latest"; + + copyToRoot = [ hello ]; runAsRoot = '' - #!${pkgs.runtimeShell} - ${pkgs.dockerTools.shadowSetup} - groupadd -r redis - useradd -r -g redis redis + ${dockerTools.shadowSetup} + groupadd -r hello + useradd -r -g hello hello mkdir /data - chown redis:redis /data + chown hello:hello /data ''; + + config = { + Cmd = [ "/bin/hello" ]; + WorkingDir = "/data"; + }; } ``` +::: -Creating base files like `/etc/passwd` or `/etc/login.defs` is necessary for shadow-utils to manipulate users and groups. - -## fakeNss {#ssec-pkgs-dockerTools-fakeNss} - -If your primary goal is providing a basic skeleton for user lookups to work, -and/or a lesser privileged user, adding `pkgs.fakeNss` to -the container image root might be the better choice than a custom script -running `useradd` and friends. - -It provides a `/etc/passwd` and `/etc/group`, containing `root` and `nobody` -users and groups. +:::{.example #ex-dockerTools-shadowSetup-buildLayeredImage} +# Using `dockerTools.shadowSetup` with `dockerTools.buildLayeredImage` -It also provides a `/etc/nsswitch.conf`, configuring NSS host resolution to -first check `/etc/hosts`, before checking DNS, as the default in the absence of -a config file (`dns [!UNAVAIL=return] files`) is quite unexpected. +It accomplishes the same thing as [](#ex-dockerTools-shadowSetup-buildImage), but using `buildLayeredImage` instead. -You can pair it with `binSh`, which provides `bin/sh` as a symlink -to `bashInteractive` (as `/bin/sh` is configured as a shell). +Note that the extra script in `fakeRootCommands` uses `groupadd` and `useradd`, which are binaries provided by the `shadow` package. +These binaries are added to the `PATH` by the `shadowSetup` script, but only for the duration of `fakeRootCommands`. ```nix -buildImage { +{ dockerTools, hello }: +dockerTools.buildLayeredImage { name = "shadow-basic"; + tag = "latest"; - copyToRoot = pkgs.buildEnv { - name = "image-root"; - paths = [ binSh pkgs.fakeNss ]; - pathsToLink = [ "/bin" "/etc" "/var" ]; + contents = [ hello ]; + + fakeRootCommands = '' + ${dockerTools.shadowSetup} + groupadd -r hello + useradd -r -g hello hello + mkdir /data + chown hello:hello /data + ''; + enableFakechroot = true; + + config = { + Cmd = [ "/bin/hello" ]; + WorkingDir = "/data"; }; } ``` +::: +[]{#ssec-pkgs-dockerTools-buildNixShellImage-arguments} ## buildNixShellImage {#ssec-pkgs-dockerTools-buildNixShellImage} -Create a Docker image that sets up an environment similar to that of running `nix-shell` on a derivation. -When run in Docker, this environment somewhat resembles the Nix sandbox typically used by `nix-build`, with a major difference being that access to the internet is allowed. -It additionally also behaves like an interactive `nix-shell`, running things like `shellHook` and setting an interactive prompt. -If the derivation is fully buildable (i.e. `nix-build` can be used on it), running `buildDerivation` inside such a Docker image will build the derivation, with all its outputs being available in the correct `/nix/store` paths, pointed to by the respective environment variables like `$out`, etc. +`buildNixShellImage` uses [`streamNixShellImage`](#ssec-pkgs-dockerTools-streamNixShellImage) underneath to build a compressed Docker-compatible repository tarball of an image that sets up an environment similar to that of running `nix-shell` on a derivation. +Basically, `buildNixShellImage` runs the script created by `streamNixShellImage` to save the compressed image in the Nix store. + +`buildNixShellImage` supports the same options as `streamNixShellImage`, see [`streamNixShellImage`](#ssec-pkgs-dockerTools-streamNixShellImage) for details. + +[]{#ssec-pkgs-dockerTools-buildNixShellImage-example} +### Examples {#ssec-pkgs-dockerTools-buildNixShellImage-examples} -::: {.warning} -The behavior doesn't match `nix-shell` or `nix-build` exactly and this function is known not to work correctly for e.g. fixed-output derivations, content-addressed derivations, impure derivations and other special types of derivations. +:::{.example #ex-dockerTools-buildNixShellImage-hello} +# Building a Docker image with `buildNixShellImage` with the build environment for the `hello` package + +This example shows how to build the `hello` package inside a Docker container built with `buildNixShellImage`. +The Docker image generated will have a name like `hello-<version>-env` and tag `latest`. +This example is the `buildNixShellImage` equivalent of [](#ex-dockerTools-streamNixShellImage-hello). + +```nix +{ dockerTools, hello }: +dockerTools.buildNixShellImage { + drv = hello; + tag = "latest"; +} +``` + +The result of building this package is a `.tar.gz` file that can be loaded into Docker: + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/pkj1sgzaz31wl0pbvbg3yp5b3kxndqms-hello-2.12.1-env.tar.gz + +$ docker load -i /nix/store/pkj1sgzaz31wl0pbvbg3yp5b3kxndqms-hello-2.12.1-env.tar.gz +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest +``` + +After starting an interactive container, the derivation can be built by running `buildDerivation`, and the output can be executed as expected: + +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ buildDerivation +Running phase: unpackPhase +unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz +source root is hello-2.12.1 +(some output removed for clarity) +Running phase: fixupPhase +shrinking RPATHs of ELF executables and libraries in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +shrinking /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin/hello +checking for references to /build/ in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1... +gzipping man pages under /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/share/man/ +patching script interpreter paths in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +stripping (with command strip and flags -S -p) in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin + +[nix-shell:~]$ $out/bin/hello +Hello, world! +``` ::: -### Arguments {#ssec-pkgs-dockerTools-buildNixShellImage-arguments} +## streamNixShellImage {#ssec-pkgs-dockerTools-streamNixShellImage} + +`streamNixShellImage` builds a **script** which, when run, will stream to stdout a Docker-compatible repository tarball of an image that sets up an environment similar to that of running `nix-shell` on a derivation. +This means that `streamNixShellImage` does not output an image into the Nix store, but only a script that builds the image, saving on IO and disk/cache space, particularly with large images. +See [](#ex-dockerTools-streamNixShellImage-hello) to understand how to load in Docker the image generated by this script. + +The environment set up by `streamNixShellImage` somewhat resembles the Nix sandbox typically used by `nix-build`, with a major difference being that access to the internet is allowed. +It also behaves like an interactive `nix-shell`, running things like `shellHook` (see [](#ex-dockerTools-streamNixShellImage-addingShellHook)) and setting an interactive prompt. +If the derivation is buildable (i.e. `nix-build` can be used on it), running `buildDerivation` in the container will build the derivation, with all its outputs being available in the correct `/nix/store` paths, pointed to by the respective environment variables (e.g. `$out`). -`drv` +::: {.caution} +The environment in the image doesn't match `nix-shell` or `nix-build` exactly, and this function is known not to work correctly for fixed-output derivations, content-addressed derivations, impure derivations and other special types of derivations. +::: -: The derivation on which to base the Docker image. +### Inputs {#ssec-pkgs-dockerTools-streamNixShellImage-inputs} - Adding packages to the Docker image is possible by e.g. extending the list of `nativeBuildInputs` of this derivation like +`streamNixShellImage` expects one argument with the following attributes: - ```nix - buildNixShellImage { - drv = someDrv.overrideAttrs (old: { - nativeBuildInputs = old.nativeBuildInputs or [] ++ [ - somethingExtra - ]; - }); - # ... - } - ``` +`drv` (Attribute Set) - Similarly, you can extend the image initialization script by extending `shellHook` +: The derivation for which the environment in the image will be set up. + Adding packages to the Docker image is possible by extending the list of `nativeBuildInputs` of this derivation. + See [](#ex-dockerTools-streamNixShellImage-extendingBuildInputs) for how to do that. + Similarly, you can extend the image initialization script by extending `shellHook`. + [](#ex-dockerTools-streamNixShellImage-addingShellHook) shows how to do that. -`name` _optional_ +`name` (String; _optional_) -: The name of the resulting image. +: The name of the generated image. - *Default:* `drv.name + "-env"` + _Default value:_ the value of `drv.name + "-env"`. -`tag` _optional_ +`tag` (String or Null; _optional_) : Tag of the generated image. + If `null`, the hash of the nix derivation that builds the Docker image will be used as the tag. - *Default:* the resulting image derivation output path's hash + _Default value:_ `null`. -`uid`/`gid` _optional_ +`uid` (Number; _optional_) -: The user/group ID to run the container as. This is like a `nixbld` build user. +: The user ID to run the container as. + This can be seen as a `nixbld` build user. - *Default:* 1000/1000 + _Default value:_ 1000. -`homeDirectory` _optional_ +`gid` (Number; _optional_) + +: The group ID to run the container as. + This can be seen as a `nixbld` build group. -: The home directory of the user the container is running as + _Default value:_ 1000. - *Default:* `/build` +`homeDirectory` (String; _optional_) -`shell` _optional_ +: The home directory of the user the container is running as. -: The path to the `bash` binary to use as the shell. This shell is started when running the image. + _Default value:_ `/build`. - *Default:* `pkgs.bashInteractive + "/bin/bash"` +`shell` (String; _optional_) -`command` _optional_ +: The path to the `bash` binary to use as the shell. + This shell is started when running the image. + This can be seen as an equivalent of the `NIX_BUILD_SHELL` [environment variable](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html#environment-variables) for {manpage}`nix-shell(1)`. -: Run this command in the environment of the derivation, in an interactive shell. See the `--command` option in the [`nix-shell` documentation](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html?highlight=nix-shell#options). + _Default value:_ the `bash` binary from the `bashInteractive` package. - *Default:* (none) +`command` (String or Null; _optional_) -`run` _optional_ +: If specified, this command will be run in the environment of the derivation in an interactive shell. + A call to `exit` will be added after the command if it is specified, so the shell will exit after it's finished running. + This can be seen as an equivalent of the `--command` option in {manpage}`nix-shell(1)`. -: Same as `command`, but runs the command in a non-interactive shell instead. See the `--run` option in the [`nix-shell` documentation](https://nixos.org/manual/nix/stable/command-ref/nix-shell.html?highlight=nix-shell#options). + _Default value:_ `null`. - *Default:* (none) +`run` (String or Null; _optional_) -### Example {#ssec-pkgs-dockerTools-buildNixShellImage-example} +: Similar to the `command` attribute, but runs the command in a non-interactive shell instead. + A call to `exit` will be added after the command if it is specified, so the shell will exit after it's finished running. + This can be seen as an equivalent of the `--run` option in {manpage}`nix-shell(1)`. + + _Default value:_ `null`. -The following shows how to build the `pkgs.hello` package inside a Docker container built with `buildNixShellImage`. +### Examples {#ssec-pkgs-dockerTools-streamNixShellImage-examples} + +:::{.example #ex-dockerTools-streamNixShellImage-hello} +# Building a Docker image with `streamNixShellImage` with the build environment for the `hello` package + +This example shows how to build the `hello` package inside a Docker container built with `streamNixShellImage`. +The Docker image generated will have a name like `hello-<version>-env` and tag `latest`. +This example is the `streamNixShellImage` equivalent of [](#ex-dockerTools-buildNixShellImage-hello). ```nix -with import <nixpkgs> {}; -dockerTools.buildNixShellImage { +{ dockerTools, hello }: +dockerTools.streamNixShellImage { drv = hello; + tag = "latest"; } ``` -Build the derivation: +The result of building this package is a script. +Running this script and piping it into `docker load` gives you the same image that was built in [](#ex-dockerTools-buildNixShellImage-hello). + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/8vhznpz2frqazxnd8pgdvf38jscdypax-stream-hello-2.12.1-env -```console -nix-build hello.nix +$ /nix/store/8vhznpz2frqazxnd8pgdvf38jscdypax-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest ``` - these 8 derivations will be built: - /nix/store/xmw3a5ln29rdalavcxk1w3m4zb2n7kk6-nix-shell-rc.drv - ... - Creating layer 56 from paths: ['/nix/store/crpnj8ssz0va2q0p5ibv9i6k6n52gcya-stdenv-linux'] - Creating layer 57 with customisation... - Adding manifests... - Done. - /nix/store/cpyn1lc897ghx0rhr2xy49jvyn52bazv-hello-2.12-env.tar.gz +After starting an interactive container, the derivation can be built by running `buildDerivation`, and the output can be executed as expected: -Load the image: +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ buildDerivation +Running phase: unpackPhase +unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz +source root is hello-2.12.1 +(some output removed for clarity) +Running phase: fixupPhase +shrinking RPATHs of ELF executables and libraries in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +shrinking /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin/hello +checking for references to /build/ in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1... +gzipping man pages under /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/share/man/ +patching script interpreter paths in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1 +stripping (with command strip and flags -S -p) in /nix/store/f2vs29jibd7lwxyj35r9h87h6brgdysz-hello-2.12.1/bin + +[nix-shell:~]$ $out/bin/hello +Hello, world! +``` +::: + +:::{.example #ex-dockerTools-streamNixShellImage-extendingBuildInputs} +# Adding extra packages to a Docker image built with `streamNixShellImage` + +This example shows how to add extra packages to an image built with `streamNixShellImage`. +In this case, we'll add the `cowsay` package. +The Docker image generated will have a name like `hello-<version>-env` and tag `latest`. +This example uses [](#ex-dockerTools-streamNixShellImage-hello) as a starting point. -```console -docker load -i result +```nix +{ dockerTools, cowsay, hello }: +dockerTools.streamNixShellImage { + tag = "latest"; + drv = hello.overrideAttrs (old: { + nativeBuildInputs = old.nativeBuildInputs or [] ++ [ + cowsay + ]; + }); +} ``` - 0d9f4c4cd109: Loading layer [==================================================>] 2.56MB/2.56MB - ... - ab1d897c0697: Loading layer [==================================================>] 10.24kB/10.24kB - Loaded image: hello-2.12-env:pgj9h98nal555415faa43vsydg161bdz +The result of building this package is a script which can be run and piped into `docker load` to load the generated image. + +```shell +$ nix-build +(some output removed for clarity) +/nix/store/h5abh0vljgzg381lna922gqknx6yc0v7-stream-hello-2.12.1-env + +$ /nix/store/h5abh0vljgzg381lna922gqknx6yc0v7-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest +``` -Run the container: +After starting an interactive container, we can verify the extra package is available by running `cowsay`: -```console -docker run -it hello-2.12-env:pgj9h98nal555415faa43vsydg161bdz +```shell +$ docker run -it hello-2.12.1-env:latest +[nix-shell:~]$ cowsay "Hello, world!" + _______________ +< Hello, world! > + --------------- + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || ``` +::: - [nix-shell:/build]$ +:::{.example #ex-dockerTools-streamNixShellImage-addingShellHook} +# Adding a `shellHook` to a Docker image built with `streamNixShellImage` -In the running container, run the build: +This example shows how to add a `shellHook` command to an image built with `streamNixShellImage`. +In this case, we'll simply output the string `Hello, world!`. +The Docker image generated will have a name like `hello-<version>-env` and tag `latest`. +This example uses [](#ex-dockerTools-streamNixShellImage-hello) as a starting point. -```console -buildDerivation +```nix +{ dockerTools, hello }: +dockerTools.streamNixShellImage { + tag = "latest"; + drv = hello.overrideAttrs (old: { + shellHook = '' + ${old.shellHook or ""} + echo "Hello, world!" + ''; + }); +} ``` - unpacking sources - unpacking source archive /nix/store/8nqv6kshb3vs5q5bs2k600xpj5bkavkc-hello-2.12.tar.gz - ... - patching script interpreter paths in /nix/store/z5wwy5nagzy15gag42vv61c2agdpz2f2-hello-2.12 - checking for references to /build/ in /nix/store/z5wwy5nagzy15gag42vv61c2agdpz2f2-hello-2.12... +The result of building this package is a script which can be run and piped into `docker load` to load the generated image. -Check the build result: +```shell +$ nix-build +(some output removed for clarity) +/nix/store/iz4dhdvgzazl5vrgyz719iwjzjy6xlx1-stream-hello-2.12.1-env -```console -$out/bin/hello +$ /nix/store/iz4dhdvgzazl5vrgyz719iwjzjy6xlx1-stream-hello-2.12.1-env | docker load +(some output removed for clarity) +Loaded image: hello-2.12.1-env:latest ``` - Hello, world! +After starting an interactive container, we can see the result of the `shellHook`: + +```shell +$ docker run -it hello-2.12.1-env:latest +Hello, world! + +[nix-shell:~]$ +``` +::: |