diff options
author | Graham Christensen <graham@grahamc.com> | 2022-09-22 11:17:14 -0400 |
---|---|---|
committer | Graham Christensen <graham@grahamc.com> | 2022-09-22 16:01:23 -0400 |
commit | c2b898da7623a39e3c9b6265d311fe182aa526f0 (patch) | |
tree | 5b55dc2b33259ab9746c1f321a5357939dfcb363 /pkgs/tools | |
parent | a3f8caaf613cec9b11971851e07fd9f146d53c94 (diff) |
treewide: drop -l$NIX_BUILD_CORES
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load. For a build machine which is configured to run `$B` builds where each build gets `total cores / B` cores (`$C`), passing `-l $C` to make will improperly limit the load to `$C` instead of `$B * $C`. This effect becomes quite pronounced on machines with 80 cores, with 40 simultaneous builds and a cores limit of 2. On a machine with this configuration, Nix will run 40 builds and make will limit the overall system load to approximately 2. A build machine with this many cores can happily run with a load approaching 80. A non-solution is to oversubscribe the machine, by picking a larger `$C`. However, there is no way to divide the number of cores in a way which fairly subdivides the available cores when `$B` is greater than 1. There has been exploration of passing a jobserver in to the sandbox, or sharing a jobserver between all the builds. This is one option, but relatively complicated and only supports make. Lots of other software uses its own implementation of `-j` and doesn't support either `-l` or the Make jobserver. For the case of an interactive user machine, the user should limit overall system load using `$B`, `$C`, and optionally systemd's cpu/network/io limiting features. Making this change should significantly improve the utilization of our build farm, and improve the throughput of Hydra.
Diffstat (limited to 'pkgs/tools')
-rw-r--r-- | pkgs/tools/admin/tigervnc/default.nix | 2 | ||||
-rw-r--r-- | pkgs/tools/security/tracee/default.nix | 2 | ||||
-rw-r--r-- | pkgs/tools/typesetting/tex/texlive/bin.nix | 2 |
3 files changed, 3 insertions, 3 deletions
diff --git a/pkgs/tools/admin/tigervnc/default.nix b/pkgs/tools/admin/tigervnc/default.nix index 549096ec0e430..724301854b403 100644 --- a/pkgs/tools/admin/tigervnc/default.nix +++ b/pkgs/tools/admin/tigervnc/default.nix @@ -92,7 +92,7 @@ stdenv.mkDerivation rec { --with-xkb-path=${xkeyboard_config}/share/X11/xkb \ --with-xkb-bin-directory=${xorg.xkbcomp}/bin \ --with-xkb-output=$out/share/X11/xkb/compiled - make TIGERVNC_SRC=$src TIGERVNC_BUILDDIR=`pwd`/../.. -j$NIX_BUILD_CORES -l$NIX_BUILD_CORES + make TIGERVNC_SRC=$src TIGERVNC_BUILDDIR=`pwd`/../.. -j$NIX_BUILD_CORES popd '' + lib.optionalString stdenv.isDarwin '' make dmg diff --git a/pkgs/tools/security/tracee/default.nix b/pkgs/tools/security/tracee/default.nix index 48d102e619d4e..7d93e6d76f7de 100644 --- a/pkgs/tools/security/tracee/default.nix +++ b/pkgs/tools/security/tracee/default.nix @@ -63,7 +63,7 @@ buildGoModule rec { buildPhase = '' runHook preBuild - make $makeFlags ''${enableParallelBuilding:+-j$NIX_BUILD_CORES -l$NIX_BUILD_CORES} + make $makeFlags ''${enableParallelBuilding:+-j$NIX_BUILD_CORES runHook postBuild ''; diff --git a/pkgs/tools/typesetting/tex/texlive/bin.nix b/pkgs/tools/typesetting/tex/texlive/bin.nix index 8da3df2773f04..64eb16cda082f 100644 --- a/pkgs/tools/typesetting/tex/texlive/bin.nix +++ b/pkgs/tools/typesetting/tex/texlive/bin.nix @@ -202,7 +202,7 @@ core-big = stdenv.mkDerivation { #TODO: upmendex if [[ "$path" =~ "libs/pplib" ]]; then # TODO: revert for texlive 2022 # ../../../texk/web2c/luatexdir/luamd5/md5lib.c:197:10: fatal error: 'utilsha.h' file not found - make ''${enableParallelBuilding:+-j''${NIX_BUILD_CORES} -l''${NIX_BUILD_CORES}} + make ''${enableParallelBuilding:+-j''${NIX_BUILD_CORES}} fi ) done |