summary refs log tree commit diff
path: root/pkgs/development/embedded
diff options
context:
space:
mode:
authorGraham Christensen <graham@grahamc.com>2022-09-22 11:17:14 -0400
committerGraham Christensen <graham@grahamc.com>2022-09-22 16:01:23 -0400
commitc2b898da7623a39e3c9b6265d311fe182aa526f0 (patch)
tree5b55dc2b33259ab9746c1f321a5357939dfcb363 /pkgs/development/embedded
parenta3f8caaf613cec9b11971851e07fd9f146d53c94 (diff)
treewide: drop -l$NIX_BUILD_CORES
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.

For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.

This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.

A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.

There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.

For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.

Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
Diffstat (limited to 'pkgs/development/embedded')
-rwxr-xr-xpkgs/development/embedded/blackmagic/helper.sh2
1 files changed, 1 insertions, 1 deletions
diff --git a/pkgs/development/embedded/blackmagic/helper.sh b/pkgs/development/embedded/blackmagic/helper.sh
index bae57f633cfbf..9185253b39d4b 100755
--- a/pkgs/development/embedded/blackmagic/helper.sh
+++ b/pkgs/development/embedded/blackmagic/helper.sh
@@ -11,7 +11,7 @@ out=${out:-/tmp}
 ################################################################################
 export CFLAGS=$NIX_CFLAGS_COMPILE
 export MAKEFLAGS="\
-  ${enableParallelBuilding:+-j${NIX_BUILD_CORES} -l${NIX_BUILD_CORES}}"
+  ${enableParallelBuilding:+-j${NIX_BUILD_CORES}}"
 
 ################################################################################
 PRODUCTS="blackmagic.bin blackmagic.hex blackmagic_dfu.bin blackmagic_dfu.hex"