setup-hooks: Add autoPatchelfHook

I originally wrote this for packaging proprietary games in Vuizvui[1]
but I thought it would be generally useful as we have a fair amount of
proprietary software lurking around in nixpkgs, which are a bit tedious
to maintain, especially when the library dependencies change after an
update.

So this setup hook searches for all ELF executables and libraries in the
resulting output paths after install phase and uses patchelf to set the
RPATH and interpreter according to what dependencies are available
inside the builder.

For example consider something like this:

stdenv.mkDerivation {
  ...
  nativeBuildInputs = [ autoPatchelfHook ];
  buildInputs = [ mesa zlib ];
  ...
}

Whenever for example an executable requires mesa or zlib, the RPATH will
automatically be set to the lib dir of the corresponding dependency.

If the library dependency is required at runtime, an attribute called
runtimeDependencies can be used to list dependencies that are added to
all executables that are discovered unconditionally.

Beside this, it also makes initial packaging of proprietary software
easier, because one no longer has to manually figure out the
dependencies in the first place.

[1]: https://github.com/openlab-aux/vuizvui

Signed-off-by: aszlig <aszlig@nix.build>
Closes: #34506

1 files changed, 14 insertions, 0 deletions

diff --git a/doc/stdenv.xml b/doc/stdenv.xml
index 3a7b23baaa7e..2a3316b8d018 100644
--- a/doc/stdenv.xml
+++ b/doc/stdenv.xml
@@ -1802,6 +1802,20 @@ addEnvHooks "$hostOffset" myBashFunction
     disabled or patched to work with PaX.</para></listitem>
   </varlistentry>
 
+  <varlistentry>
+    <term>autoPatchelfHook</term>
+    <listitem><para>This is a special setup hook which helps in packaging
+    proprietary software in that it automatically tries to find missing shared
+    library dependencies of ELF files. All packages within the
+    <envar>runtimeDependencies</envar> environment variable are unconditionally
+    added to executables, which is useful for programs that use
+    <citerefentry>
+      <refentrytitle>dlopen</refentrytitle>
+      <manvolnum>3</manvolnum>
+    </citerefentry>
+    to load libraries at runtime.</para></listitem>
+  </varlistentry>
+
 </variablelist>
 
 </para>