summary refs log tree commit diff
path: root/pkgs/stdenv
AgeCommit message (Collapse)AuthorFilesLines
2023-11-12stdenv: Improve performanceArtturin1-82/+119
| stat | before | after | Δ | Δ% | |------------------------|-----------------|-----------------|-----------------|---------| | cpuTime | 513.67 | 507.77 | ↘ 5.90 | -1.15% | | envs-bytes | 20,682,847,968 | 20,628,961,616 | ↘ 53,886,352 | -0.26% | | envs-elements | 1,054,735,104 | 1,051,395,620 | ↘ 3,339,484 | -0.32% | | envs-number | 765,310,446 | 763,612,291 | ↘ 1,698,155 | -0.22% | | gc-heapSize | 53,439,602,688 | 51,711,545,344 | ↘ 1,728,057,344 | -3.23% | | gc-totalBytes | 113,062,066,672 | 112,139,998,240 | ↘ 922,068,432 | -0.82% | | list-bytes | 3,118,249,784 | 3,118,249,784 | 0 | | | list-concats | 52,834,140 | 52,834,140 | 0 | | | list-elements | 389,781,223 | 389,781,223 | 0 | | | nrAvoided | 968,097,988 | 991,889,795 | ↗ 23,791,807 | 2.46% | | nrFunctionCalls | 697,259,792 | 697,259,792 | 0 | | | nrLookups | 510,257,062 | 338,275,331 | ↘ 171,981,731 | -33.70% | | nrOpUpdateValuesCopied | 1,446,690,216 | 1,446,690,216 | 0 | | | nrOpUpdates | 68,504,034 | 68,504,034 | 0 | | | nrPrimOpCalls | 429,464,805 | 429,464,805 | 0 | | | nrThunks | 1,009,240,391 | 982,109,100 | ↘ 27,131,291 | -2.69% | | sets-bytes | 33,524,722,928 | 33,524,722,928 | 0 | | | sets-elements | 1,938,309,212 | 1,938,309,212 | 0 | | | sets-number | 156,985,971 | 156,985,971 | 0 | | | sizes-Attr | 16 | 16 | 0 | | | sizes-Bindings | 16 | 16 | 0 | | | sizes-Env | 16 | 16 | 0 | | | sizes-Value | 24 | 24 | 0 | | | symbols-bytes | 2,151,298 | 2,151,298 | 0 | | | symbols-number | 159,707 | 159,707 | 0 | | | values-bytes | 30,218,194,248 | 29,567,043,264 | ↘ 651,150,984 | -2.15% | | values-number | 1,259,091,427 | 1,231,960,136 | ↘ 27,131,291 | -2.15% | > Accessing the lexical scope directly should be more efficient, yes, because it changes from a binary search (many lookups) to just two memory accesses > correction: one short linked list + one array access > oh and you had to do the lexical scope lookup anyway for lib itself > so it really does save a binary search at basically no extra cost - roberth after seeing the stats > Oooh nice. I did not consider that more of the maybeThunk optimization becomes effective (nrAvoided). Those lookups also caused allocations! - roberth Left `lib.generators` and `lib.strings` alone because they're only used once.
2023-10-31check-meta.nix: Fix flake noteRobert Hensing1-3/+2
- These new-cli commands can be used with `-f`, in which case they're evaluated with pure evaluation disabled. - Nix 2.4+ is not part of the condition; "flakes" is fully descriptive and more relatable. - Don't suggest that it only enables this variable. - Just don't say too much.
2023-10-25stdenv.mkDerivation: Reject MD5 hashesnicoo1-0/+11
While there is no fetcher or builder (in nixpkgs) that takes an `md5` parameter, for some inscrutable reason the nix interpreter accepts the following: ```nix fetchurl { url = "https://www.perdu.com"; hash = "md5-rrdBU2a35b2PM2ZO+n/zGw=="; } ``` Note that neither MD5 nor SHA1 are allowed by the syntax of SRI hashes.
2023-10-21stdenv: fix typoArtturin1-1/+1
2023-10-19stdenvAdapters.useMoldLinker: set `allowedRequisites` to `null`Pavel Sobolev1-10/+4
2023-10-13stdenvAdapters.useMoldLinker: automatically add `allowedRequisites`Artturin1-1/+4
Co-Authored-By: Pavel Sobolev <paveloom@riseup.net>
2023-10-13mold: add tests for `mold-wrapped` and `useMoldLinker` adapterArtturin1-2/+2
Make both ready for cross with prefixes Currently `pkgsCross.aarch64-multiplatform.mold.passthru.tests.{wrapped,adapter}` fail with ``` Testing running the 'hello' binary which should be linked with 'mold' Hello, world! Checking for mold in the '.comment' section No mention of 'mold' detected in the '.comment' section The command was: aarch64-unknown-linux-gnu-readelf -p .comment ...bin/hello The output was: String dump of section '.comment': [ 0] GCC: (GNU) 12.3.0 ```
2023-10-07release.nix: namespace bootstrap tools with triplesAlyssa Ross1-1/+1
This will allow buliding bootstrap tools for platforms with non-default libcs, like *-unknown-linux-musl. This gets rid of limitedSupportSystems/systemsWithAnySupport. There was no need to use systemsWithAnySupport for supportDarwin, because it was always equivalent to supportedSystems for that purpose, and the only other way it was used was for determining which platforms to build the bootstrap tools for, so we might as well use a more explicit parameter for that, and then we can change how it works without affecting the rest of the Hydra jobs. Not affecting the rest of the Hydra jobs is important, because if we changed all jobs to use config triples, we'd end up renaming every Hydra job. That might still be worth thinking about at some point, but it's unnecessary at this point (and would be a lot of work). I've checked by running nix-eval-jobs --force-recurse pkgs/top-level/release.nix that the actual bootstrap tools derivations are unaffected by this change, and that the only other jobs that change are ones that depend on the hash of all of Nixpkgs. Of the other jobset entrypoints that end up importing pkgs/top-level/release.nix, none used the limitedSupportedSystems parameter, so they should all be unaffected as well.
2023-10-03bootstrap-tools-cross: Rename bootstrap file jobs to be more descriptiveArtturin16-27/+27
Rename files in `bootstrap-files/` to match jobs in `make-bootstrap-tools-cross`. Should make automating the bootstrap files easier. ```nix $ nix repl --file ./pkgs/top-level/release-cross.nix nix-repl> bootstrapTools { aarch64-unknown-linux-gnu = { ... }; aarch64-unknown-linux-musl = { ... }; armv5tel-unknown-linux-gnueabi = { ... }; armv6l-unknown-linux-gnueabihf = { ... }; armv6l-unknown-linux-musleabihf = { ... }; armv7l-unknown-linux-gnueabihf = { ... }; mips64el-unknown-linux-gnuabi64 = {... }; mips64el-unknown-linux-gnuabin32 = { ... }; mipsel-unknown-linux-gnu = { ... }; powerpc64le-unknown-linux-gnu = { ... }; riscv64-unknown-linux-gnu = { ... }; x86_64-unknown-linux-musl = { ... }; } ``` Additionally if non-linux bootstrap files are added to `make-bootstrap-tools-cross` then there won't be any renaming needed.
2023-10-02Merge pull request #258476 from Artturin/bootstrapthingies2Artturi2-54/+10
2023-10-02bootstrap-tools-cross: Add a note about what should be hereArtturin1-1/+3
2023-10-02stdenv: remove unused loongson2f bootstrap filesArtturin1-44/+0
The last use of it was removed in (linux/default.nix: use mipsel.nix instead of longson.nix for mips32)[e8b10284f32b32469d5f54433e5c5a75448b327b].
2023-10-01bootstrap-tools-cross: sortArtturin1-5/+8
2023-10-01bootstrap-tools-cross: remove bootstrap files for systems which are not used ↵Artturin1-5/+0
in stdenv/linux/default.nix
2023-09-30Revert "Revert "Merge pull request #253760 from chivay/bootstrap-scrt""Artturin1-1/+2
This reverts commit 3f46cdcb5b903fe1c07707fea9ee24ae09d964b2.
2023-09-07Merge staging-next into staginggithub-actions[bot]1-1/+1
2023-09-07freshBootstrapTools.bootstrapTools: fix eval on darwinArtturin1-1/+1
``` error: undefined variable 'system' at /home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/stdenv/darwin/make-bootstrap-tools.nix:213:32: 212| 213| bootstrapTools = derivation { | ^ 214| inherit system; ``` fixes eval of `freshBootstrapTools.bootstrapTools` in darwin repl
2023-09-05Merge staging-next into staginggithub-actions[bot]1-1/+1
2023-09-04pkgs/stdenv: fix typo in booter.nix docsMartin Zacho1-1/+1
2023-09-04Merge pull request #245909 from Artturin/setupshchanges2Artturi1-1/+1
2023-09-02stdenv: Fix possible issues discovered withArtturin1-1/+1
``` set -o errexit -o nounset -o pipefail shopt -s inherit_errexit ``` in `stdenv/default-builder.sh`
2023-08-29stdenv: Add hack to fix cmake canExecute cross-compilationArtturin1-0/+2
Fixes `pkgsCross.musl64.llvmPackages_16.clang.cc` on `x86_64-linux`, which used to fail with `/bin/sh: clang-tblgen: not found`. Same hack is used in other projects: https://github.com/search?q=%2FCMAKE_CROSSCOMPILING_EMULATOR.%2B%5C%2Fusr%5C%2Fbin%5C%2Fenv%2F+NOT+is%3Afork&type=code Comment from https://github.com/vsrinivas/fuchsia/blob/30435a9d0f0b67c94e3c70760b522c9f7fbbd6be/build/cmake/HostLinuxToolchain.cmake#L64 > Required to run host Linux executables during the build itself. > An example would be https://gitub.com/KhronosGroup/Vulkan-Loader and > its "asm_offset" program. > > NOTE: Alternatives have been tried unsuccessfully, i.e.: > > With $(set CMAKE_CROSSCOMPILING_EMULATOR), the build fails because > the CMake ninja/Make script tries to find the executable in the current > path, as in: > > [3/16] Generating gen_defines.asm > FAILED: loader/gen_defines.asm > cd /tmp/cc/build-Vulkan-Loader/loader && asm_offset GAS > /bin/sh: asm_offset: command not found > ninja: build stopped: subcommand failed. > > With $(set CMAKE_CROSSCOMPILING_EMULATOR ""), the build fails because > the shell cannot find the "" program as in: > > [3/16] Generating gen_defines.asm > FAILED: loader/gen_defines.asm > cd /tmp/cc/build-Vulkan-Loader/loader && "" /tmp/cc/build-Vulkan-Loader/loader/asm_offset GAS > /bin/sh: : command not found > ninja: build stopped: subcommand failed. > > It seems that the root of the problem comes from how the CMake function > cmCustomCommandGenerator::GetArgc0Location() computes the target > executable's location. At this point it's unclear whether this is a CMake > bug or a feature. Risicle discovered this hack. Co-authored-by: Robert Scott <code@humanleg.org.uk>
2023-08-10stdenv: Print `_allFlags` debug output to stderrArtturin1-1/+1
In the default `fixupPhase` the output of `substituteAllStream` is streamed to setup-hook. `stdenv.cc.bintools.overrideAttrs { NIX_DEBUG = 6; }` With `NIX_DEBUG` contains: ``` @expandResponseParams@ -> /nix/store/yl01rd58vp4m8bbhkihpk132cprfmx6f-expand-response-params/bin/expand-response-params ... ```
2023-08-06Merge pull request #219421 from risicle/ris-fortify-headers-autoRobert Scott1-0/+3
cc-wrapper: include fortify-headers before libc includes for musl
2023-08-06cc-wrapper: include fortify-headers before libc includes for muslRobert Scott1-0/+3
2023-08-06Merge staging-next into staginggithub-actions[bot]1-1/+1
2023-08-06Merge master into staging-nextgithub-actions[bot]1-1/+1
2023-08-05treewide: fix some comments (#247365)Jan Malakhovski1-1/+1
* pkgs/top-level/splice.nix: fix broken comment * stdenv: use a value that actually works in the comment there
2023-08-01codesign_allocate: reference cctoolsSimon Žlender1-0/+4
2023-08-01Merge pull request #245640 from reckenrode/revert-stdenv-no-responseWeijia Wang1-48/+17
darwin-stdenv: revert `NIX_CC_NO_RESPONSE_FILE` logic
2023-07-31Revert "stdenv: use improved strip.sh for aarch64-linux"Vladimír Čunát1-4/+1
This reverts commit 39919b8f215110c1516f5c6b300f5ee69df23fd4. The parent merge resolved this more properly.
2023-07-30stdenv: use improved strip.sh for aarch64-linuxVladimír Čunát1-1/+4
Adapted from PR #246164 TODO: clean up / use it everywhere on the next rebuild.
2023-07-28stdenv: Make condition clearerArtturin1-1/+1
-z checks for a empty string -n checks for a not empty string It makes more sense to run the chmod if the string is not empty
2023-07-25darwin-stdenv: revert `NIX_CC_NO_RESPONSE_FILE` logicRandy Eckenrode1-48/+17
To work around intermitent build failures with clang 16, the stdenv attempted to pass arguments on the command-line on newer versions of macOS. Unfortunately, the larger `ARG_MAX` is still not large enough to build qtwebengine. This commit reverts the `NIX_CC_NO_RESPONSE_FILE` logic in the stdenv. The changes to cc-wrapper in #245282 are needed for clang 16 to prevent the above-mentioned build failures.
2023-07-22Merge master into staging-nextgithub-actions[bot]1-4/+1
2023-07-21stdenvAdapters: Remove 6 year old commentArtturin1-3/+0
Comment was added in 3c36dcc and hasn't been needed because makeStaticBinaries already adds glibc.static
2023-07-21stdenvAdapters: Fix conditionArtturin1-1/+1
Fixes issue 244232
2023-07-17Merge pull request #241025 from uri-canva/fix-nativeUri Baghin1-2/+5
stdenv: fix stagesNative
2023-07-15freshBootstrapTools: fix build on darwinWeijia Wang1-6/+6
2023-07-14stdenvBootstrapTools: in darwin, only run install_name_tool on Mach-O filesTheodore Ni1-2/+2
2023-07-09darwin.stdenv: allow `patchShebangs` during the bootstrapRandy Eckenrode1-12/+26
This fixes pyicu (and any other package that uses `icu-config` instead of the CMake or some other module to get the build flags). What happened here is the bootstrap disables `patchShebangs` to avoid propagating the bootstrap tools to the final stdenv (due to `sh` and `bash` being on the `PATH` from the bootstrap tools). Because of that, the `#!/bin/sh` line in `icu-config` was not updated, causing it to invoke the system bash on Darwin. While that is undesirable in its own right, when the system bash is invoked as `sh`, `echo -n` will print `-n`, resulting in the breakage see in https://github.com/NixOS/nixpkgs/pull/241951#issuecomment-1627604354. The fix is to build bash earlier in the bootstrap while making sure it is picked up over the one in the bootstrap tools. That allows `patchShebangs` to be enabled during the bootstrap. Any package with scripts that is included in the final stdenv should now have its scripts’ shebang lines properly patched.
2023-07-09Merge master into staging-nextgithub-actions[bot]2-2/+3
2023-07-08darwin.stdenv: only run `install_name_tool` on filesRandy Eckenrode1-1/+1
2023-07-08darwin.stdenv: fix portable libsystem hook with sandboxingRandy Eckenrode2-1/+2
When sandboxing is enabled, the hook tries to run `install_name_tool` and fails because the system one is inaccessible. Having it use `targetPrefix` allows it to find and use the cross-install_name_tool.
2023-07-06Merge #224822: hardening flags: enable fortify3 by defaultVladimír Čunát1-6/+3
...into staging
2023-07-05Merge pull request #240433 from reckenrode/darwin-stdenv-reworkWeijia Wang3-558/+1149
darwin.stdenv: Darwin stdenv rework
2023-07-03stdenvAdapters: don't use lib.optional with a listfigsoda1-1/+1
2023-07-03stdenv: fix overriding with attrset when finalAttrs isn't usedArtturin1-1/+1
``` nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname error: … while evaluating a branch condition at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7: 85| in 86| if builtins.isAttrs result then | ^ 87| result // { … while calling the 'isAttrs' builtin at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10: 85| in 86| if builtins.isAttrs result then | ^ 87| result // { (stack trace truncated; use '--show-trace' to show the full trace) error: attempt to call something which is not a function but a set at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21: 57| f = self: super: 58| let x = f0 super; | ^ 59| in ```
2023-07-02swift-corelibs: don’t link against libcurlRandy Eckenrode1-53/+33
swift-corelibs uses libcurl to implement `NSURLSession` in Foundation via the symbols exported by CF. Foundation is not build on Darwin, and these symbols are not exported by the system CoreFoundation. By not linking against libcurl, this breaks a cycle between CF and libcurl. That should allow libcurl to drop the patch disabling linking against the SystemConfiguration and restore NAT64 support. Unfortunately, the Darwin stdenv bootstrap still needs to build dependencies that use `fetchFromGitHub`. While it can drop curl from the final stdenv, it still needs to use it during the stdenv bootstrap.
2023-07-02darwin.stdenv: refactor stdenv definitionRandy Eckenrode3-556/+1167
In preparation for bumping the LLVM used by Darwin, this change refactors and reworks the stdenv build process. When it made sense, existing behaviors were kept to avoid causing any unwanted breakage. However, there are some differences. The reasoning and differences are discussed below. - Improved cycle times - Working on the Darwin stdenv was a tedious process because `allowedRequisites` determined what was allowed between stages. If you made a mistake, you might have to wait a considerable amount of time for the build to fail. Using assertions makes many errors fail at evaluation time and makes moving things around safer and easier to do. - Decoupling from bootstrap tools - The stdenv build process builds as much as it can in the early stages to remove the requirement that the bootstrap tools need bumped in order to bump the stdenv itself. This should lower the barrier to updates and make it easier to bump in the future. It also allows changes to be made without requiring additional tools be added to the bootstrap tools. - Patterned after the Linux stdenv - I tried to follow the patterns established in the Linux stdenv with adaptations made to Darwin’s needs. My hope is this makes the Darwin stdenv more approable for non-Darwin developers who made need to interact with it. It also allowed some of the hacks to be removed. - Documentation - Comments were added explaining what was happening and why things were being done. This is particular important for some stages that might not be obvious (such as the sysctl stage). - Cleanup - Converting the intermediate `allowedRequisites` to assertions revealed that many packages were being referenced that no longer exist or have been renamed. Removing them reduces clutter and should help make the stdenv bootstrap process be more understandable.