| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
directory
The change moves definition of bootstrap files slightly closer to
`linux` structure to eventually allow those to update in bulk:
https://github.com/NixOS/nixpkgs/issues/253713
|
|
The aarch64 musl bootstrap tools are woefully outdated. Just getting
them to the point of being able to build new versions of themselves
required a number of hacks[1] that can be reverted once we have new
bootstrap tools, and before that it stdenv didn't even build for the
preceding three years.
[1]: https://github.com/NixOS/nixpkgs/pull/169764
So, following the script established by previous bootstrap tools
updates:
Files came from this Hydra build:
https://hydra.nixos.org/build/246470544
…which used nixpkgs revision dd5621df6dcb90122b50da5ec31c411a0de3e538
to instantiate:
/nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv
…and then built:
/nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools
I downloaded these files from Hydra and prefetched them into the Nix
store with the following commands:
STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix --extra-experimental-features nix-command store prefetch-file \
file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix --extra-experimental-features nix-command store prefetch-file --executable \
file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
These commands produced the following output:
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE=').
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs=').
I used the hashes from the output above to create the fetchurl
invocation which is part of this commit.
I then started the bootstrap with the following command:
nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello
As @lovesegfault requested, here are the the sha256sums of all the
on-server components for extra verification:
sha256sum /nix/store/${STOREPATH}/on-server/*
…which produced the following output:
658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01 /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
|
|
|
|
This PR updates the bootstrap tarballs for riscv64-linux with new Hydra-generated ones.
Fixes #275848 (bootstrap assembler too old to build gcc 13).
I'll be following the script used in #151399, #168199, #183487, and #188334.
Files came from [this](https://hydra.nixos.org/build/246376732#tabs-summary) Hydra build, which used nixpkgs revision 160cedc144aced7a35a91440b46b74ffacd52682 to instantiate:
```
/nix/store/cpiajh4l83b08pynwiwkpxj53d78pcxr-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu.drv
```
and then built:
```
/nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
```
I downloaded these files from Hydra and prefetched them into the nix store with the following commands:
```
STOREPATH=8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
file://$(nix store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```
These commands produced the following output:
```
Downloaded 'file:///nix/store/xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' to '/nix/store/3fal4gikp92013kac6rdmfbrch2s859b-xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' (hash 'sha256-0LxRd7fdafQezNJ+N2tuOfm0KEwgfRSts5fhP0e0r0s=').
Downloaded 'file:///nix/store/9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' to '/nix/store/kb7wyy30y1gxcmdajljr26kxxac606qa-9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' (hash 'sha256-OGO96QUzs2n5pGipn/V87AxzUY9OWKZl417nE8HdZIE=').
```
I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.
I then started the bootstrap with the following command:
```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.riscv64' hello
```
As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:
```
sha256sum /nix/store/${STOREPATH}/on-server/*
```
which produced the following output:
```
d0bc5177b7dd69f41eccd27e376b6e39f9b4284c207d14adb397e13f47b4af4b /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
65f9433abb598f63c932d33351b14f686551512b1cece1e64c2d0e76aa0ec52e /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/busybox
```
|
|
|
|
--replace-{fail,warn,quiet}
|
|
stdenv.cc may throw, e.g. in the case of pkgsCross.ghcjs where we must
not force it for the purpose of attribute accessing (`or` doesn't
implicitly tryEval…).
Regression introduced in 1a5bd697adecf27385b69352485baa52a6e02fe9.
|
|
|
|
|
|
stdenv.adapters: fix `overrideLibcxx` to refer to existing `compiler…
|
|
|
|
|
|
Before the change `pkgsLLVM` attributes were failing to pull in
`compiler-rt` on `x86_64-linux`:
$ nix build --no-link -f. pkgsLLVM.asciidoc-full
error:
error: attribute 'llvmPackages_13' missing
at pkgs/stdenv/adapters.nix:86:32:
85| inherit libcxx;
86| extraPackages = [ cxxabi pkgs.pkgsTargetTarget."llvmPackages_${lib.versions.major llvmLibcxxVersion}".compiler-rt ];
| ^
87| });
It happens because `pkgs.pkgsTargetTarget` are always empty for
cross-packages like `pkgsLLVM.`, `pkgsCross.*.` or
`--arg crossSystem '...'`.
|
|
|
|
stdenv/check-meta: Use bespoke type checking
|
|
|
|
Before the change `pkgsMusl.adobe-reader` was failing the interpreter:
$ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
error:
error: evaluation aborted with the following error message: 'unsupported platform for the pure Linux stdenv'
After the change `pkgsMusl.adobe-reader` returns catchable excation:
$ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
{ success = false; value = false; }
Noticed when was exploring `nixpkgs` for uncatchable evaluation errors.
Ideally those should only happen when there is a code bug in the use
site. In this case it's just a package with incompatible constraints for
`musl`.
Changed uncatchable `abort` to `throw`.
|
|
Aka `checkMeta` goes brrr.
Using the module system type checking works OK & generates good error messages.
The performance of using it however is terrible because of the value merging it does being very allocation heavy.
By implementing a very minimal type checker we can drastically improve the performance when nixpkgs is evaluated with `checkMeta = true`.
|
|
|
|
Without the change bootstrapTools build fails as:
https://cache.nixos.org/log/g5wyq9xqshan6m3kl21bjn1z88hx48rh-stdenv-bootstrap-tools.drv
error: install_name_tool: changing install names or rpaths can't be redone for: /nix/store/0hxg356h7lnl2hck23wrdpbib3cckx41-stdenv-bootstrap-tools/bin/tac (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)
|
|
|
|
stdenv: Avoid allocating intermediate attrset when checking meta validity
|
|
adisbladis/stdenv-meta-tolist-license-allocations
stdenv: Avoid some list allocations in check-meta when checking licenses
|
|
This is a small performance optimization. It should be impercetible to most.
Benchmarks:
- Before
``` json
{
"cpuTime": 0.2777960002422333,
"envs": {
"bytes": 3832648,
"elements": 189513,
"number": 144784
},
"gc": {
"heapSize": 402915328,
"totalBytes": 50229344
},
"list": {
"bytes": 655304,
"concats": 3249,
"elements": 81913
},
"nrAvoided": 218962,
"nrFunctionCalls": 127718,
"nrLookups": 40946,
"nrOpUpdateValuesCopied": 1563978,
"nrOpUpdates": 8542,
"nrPrimOpCalls": 113032,
"nrThunks": 329605,
"sets": {
"bytes": 29774864,
"elements": 1824537,
"number": 36392
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 16,
"Value": 24
},
"symbols": {
"bytes": 235909,
"number": 24432
},
"values": {
"bytes": 9691392,
"number": 403808
}
}
```
- After
```
{
"cpuTime": 0.2615779936313629,
"envs": {
"bytes": 3833832,
"elements": 189661,
"number": 144784
},
"gc": {
"heapSize": 402915328,
"totalBytes": 50212960
},
"list": {
"bytes": 655304,
"concats": 3249,
"elements": 81913
},
"nrAvoided": 218814,
"nrFunctionCalls": 127718,
"nrLookups": 40798,
"nrOpUpdateValuesCopied": 1563978,
"nrOpUpdates": 8542,
"nrPrimOpCalls": 113032,
"nrThunks": 329457,
"sets": {
"bytes": 29765392,
"elements": 1824093,
"number": 36244
},
"sizes": {
"Attr": 16,
"Bindings": 16,
"Env": 16,
"Value": 24
},
"symbols": {
"bytes": 235909,
"number": 24432
},
"values": {
"bytes": 9687840,
"number": 403660
}
}
```
|
|
|
|
|
|
bintools-wrapper
this makes it a lot easier to create a modified stdenv with a
different set of defaultHardeningFlags and as a bonus allows us
to inject the correct defaultHardeningFlags into toolchain wrapper
scripts, reducing repetition.
while most hardening flags are arguably more of a compiler thing,
it works better to put them in bintools-wrapper because cc-wrapper
can easily refer to bintools but not vice-versa.
mkDerivation can still easily refer to either when it is constructed.
this also switches fortran-hook.sh to use the same defaults for
NIX_HARDENING_ENABLE as for C. previously NIX_HARDENING_ENABLE
defaults were apparently used to avoid passing problematic flags
to a fortran compiler, but this falls apart as soon as mkDerivation
sets its own NIX_HARDENING_ENABLE - cc.hardeningUnsupportedFlags
is a more appropriate mechanism for this as it actively filters
out flags from being used by the wrapper, so switch to using that
instead.
this is still an imperfect mechanism because it doesn't handle a
compiler which has both langFortran *and* langC very well - applying
the superset of the two's hardeningUnsupportedFlags to either
compiler's invocation. however this is nothing new - cc-wrapper
already poorly handles a langFortran+langC compiler, applying two
setup hooks that have contradictory options.
|
|
Conflicts:
pkgs/tools/networking/ofono/default.nix
|
|
stdenv: enable multithreading for `xz` decompression
|
|
|
|
treewide: remove unreferenced patch files
|
|
|
|
Found with `fd \\.patch$ pkgs/ -x bash -c 'rg -F "{/}" pkgs/ -q || echo {}'`
|
|
If a CMake target has a non-default LINKER_LANGUAGE set, CMake will
manually add the libraries it has detected that language's compiler as
linking implicitly. When it does this, it'll pass -Bstatic and
-Bdynamic options based on the vibes it gets from each such detected
library. This in itself isn't a problem, because the compiler
toolchain, or our wrapper, or something, seems to be smart enough to
ignore -Bdynamic for those libraries. But it does create a problem if
the compiler adds extra libraries to the linker command line after
that final -Bdynamic, because those will be linked dynamically. Since
our compiler is static by default, CMake should reset to -Bstatic
after it's done manually specifying libraries, but CMake didn't
actually know that our compiler is static by default. The fix for
that is to tell it, like so.
Until recently, this problem was difficult to notice, because it would
result binaries that worked, but that were dynamically linked. Since
e08ce498f03f ("cc-wrapper: Account for NIX_LDFLAGS and NIX_CFLAGS_LINK
in linkType"), though, -Wl,-dynamic-linker is no longer mistakenly
passed for executables that are supposed to be static, so they end up
created with a /lib interpreter path, and so don't run at all on
NixOS.
This fixes pkgsStatic.graphite2.
|
|
|
|
|
|
stdenv: run patchShebangs on the configure script when it's a file
|
|
if the configure script has a `/usr/bin/env` or some other shebang which
is not in the sandbox then there will be errors such as
`...-stdenv-linux/setup: line 1299: ./configure: cannot execute: required file not found`
There are 250 files which `patchShebangs` `./configure`
https://github.com/search?q=NOT+is%3Afork+lang%3Anix+%2FpatchShebangs+.%5C%2Fconfigure%2F&type=code
|
|
|
|
freshBootstrapTools: Overlay the package set with the desired LLVM
|
|
|
|
Make phases header consistent for all phases.
`Running phase:` is from an old nix ux doc from 2020
https://github.com/tweag/nix-ux/blob/master/first_steps_with_nix_v2.md
Co-authored-by: Artturin <Artturin@artturin.com>
|
|
possible performance improvement
|
|
|
|
See c3c31aa798551a2808981d14537d04de8f51dff8
|
|
|
|
As reported in #241692, since the `llvmPackages` bump the
bootstrap-tools started failing to build due to a mismatch in LLVM
versions used to build certain tools.
By overlaying the imported package set to specify `llvmPackages`, we get
everything built with the expected LLVM version.
|
|
| stat | before | after | Δ | Δ% |
|------------------------|-----------------|-----------------|-----------------|---------|
| cpuTime | 513.67 | 507.77 | ↘ 5.90 | -1.15% |
| envs-bytes | 20,682,847,968 | 20,628,961,616 | ↘ 53,886,352 | -0.26% |
| envs-elements | 1,054,735,104 | 1,051,395,620 | ↘ 3,339,484 | -0.32% |
| envs-number | 765,310,446 | 763,612,291 | ↘ 1,698,155 | -0.22% |
| gc-heapSize | 53,439,602,688 | 51,711,545,344 | ↘ 1,728,057,344 | -3.23% |
| gc-totalBytes | 113,062,066,672 | 112,139,998,240 | ↘ 922,068,432 | -0.82% |
| list-bytes | 3,118,249,784 | 3,118,249,784 | 0 | |
| list-concats | 52,834,140 | 52,834,140 | 0 | |
| list-elements | 389,781,223 | 389,781,223 | 0 | |
| nrAvoided | 968,097,988 | 991,889,795 | ↗ 23,791,807 | 2.46% |
| nrFunctionCalls | 697,259,792 | 697,259,792 | 0 | |
| nrLookups | 510,257,062 | 338,275,331 | ↘ 171,981,731 | -33.70% |
| nrOpUpdateValuesCopied | 1,446,690,216 | 1,446,690,216 | 0 | |
| nrOpUpdates | 68,504,034 | 68,504,034 | 0 | |
| nrPrimOpCalls | 429,464,805 | 429,464,805 | 0 | |
| nrThunks | 1,009,240,391 | 982,109,100 | ↘ 27,131,291 | -2.69% |
| sets-bytes | 33,524,722,928 | 33,524,722,928 | 0 | |
| sets-elements | 1,938,309,212 | 1,938,309,212 | 0 | |
| sets-number | 156,985,971 | 156,985,971 | 0 | |
| sizes-Attr | 16 | 16 | 0 | |
| sizes-Bindings | 16 | 16 | 0 | |
| sizes-Env | 16 | 16 | 0 | |
| sizes-Value | 24 | 24 | 0 | |
| symbols-bytes | 2,151,298 | 2,151,298 | 0 | |
| symbols-number | 159,707 | 159,707 | 0 | |
| values-bytes | 30,218,194,248 | 29,567,043,264 | ↘ 651,150,984 | -2.15% |
| values-number | 1,259,091,427 | 1,231,960,136 | ↘ 27,131,291 | -2.15% |
> Accessing the lexical scope directly should be more efficient, yes, because it changes from a binary search (many lookups) to just two memory accesses
> correction: one short linked list + one array access
> oh and you had to do the lexical scope lookup anyway for lib itself
> so it really does save a binary search at basically no extra cost
- roberth
after seeing the stats
> Oooh nice. I did not consider that more of the maybeThunk optimization becomes effective (nrAvoided). Those lookups also caused allocations!
- roberth
Left `lib.generators` and `lib.strings` alone because they're only used
once.
|
|
darwin.stdenv: use CoreFoundation instead of CF
|
|
|
