about summary refs log tree commit diff
path: root/nixos/lib/testing
AgeCommit message (Collapse)AuthorFilesLines
2024-05-14nixos/testing: fix markdown link in enableOCR descriptionDonovan Glover1-1/+1
2024-04-12NixOS Integration tests: Re-enable for macOSJacek Galowicz1-1/+1
2024-04-10Revert "NixOS Integration Tests: Enable again for darwin"Cole Helbling1-1/+1
2024-04-10NixOS Integration Tests: Enable again for darwinJacek Galowicz1-1/+1
2024-04-03nixos/tests/installer: avoid create_machine, clean upK9001-0/+7
- use normal VM nodes for target, with some extra trickery - rename preBootCommands to postBootCommands to match its actual intent - rename VMs to installer and target, so they're not all called machine - set platforms on non-UEFI tests properly - add missing packages for systemd-boot test - fix initrd secrets leaking into the store and having wrong paths
2024-04-01nixos/lib, doc: remove references to mdDoc (#300738)Philip Taron8-37/+35
* doc: remove references to mdDoc in nixos/doc/manual/development/option-declarations.section.md * nixos/lib: remove mdDoc in nixos/lib/make-options-doc/default.nix * nixos/lib: remove mdDoc in nixos/lib/systemd-types.nix * nixos/lib: remove mdDoc in nixos/lib/systemd-unit-options.nix * nixos/lib: remove mdDoc in nixos/lib/testing/driver.nix * nixos/lib: remove mdDoc in nixos/lib/testing/interactive.nix * nixos/lib: remove mdDoc in nixos/lib/testing/meta.nix * nixos/lib: remove mdDoc in nixos/lib/testing/name.nix * nixos/lib: remove mdDoc in nixos/lib/testing/network.nix * nixos/lib: remove mdDoc in nixos/lib/testing/nodes.nix * nixos/lib: remove mdDoc in nixos/lib/testing/run.nix * nixos/lib: remove mdDoc in nixos/lib/testing/testScript.nix
2024-03-02macOS support for NixOS tests (#282401)Gabriella Gonzalez3-3/+29
Closes #193336 Closes #261694 Related to #108984 The goal here was to get the following flake to build and run on `aarch64-darwin`: ```nix { inputs.nixpkgs.url = <this branch>; outputs = { nixpkgs, ... }: { checks.aarch64-darwin.default = nixpkgs.legacyPackages.aarch64-darwin.nixosTest { name = "test"; nodes.machine = { }; testScript = ""; }; }; } ``` … and after this change it does. There's no longer a need for the user to set `nodes.*.nixpkgs.pkgs` or `nodes.*.virtualisation.host.pkgs` as the correct values are inferred from the host system.
2024-02-27nixos/lib/testing: remove yet another source of unnecessary test rebuildsK9001-0/+1
2024-02-27nixos/lib/testing: remove another source of unnecessary test rebuildsK9001-1/+4
2023-11-06nixos/testing/nodes: Do allow aliasesRobert Hensing1-3/+0
Aliases exist for a reason. Sure it is nice to make sure that some aliases aren't used within Nixpkgs, but this creates two problems which are far worse than your failing to meet your neatness compulsions. - Users encounter missing attributes, https://github.com/NixOS/nixpkgs/issues/264577 wasting their time, stalling their progress, and even occupying others time that would be better spent on fixing *real* issues. - Hydra doesn't treat evaluation errors seriously enough, with the effect that actual relevant test failures are masked by evaluation failures such as those caused by this no aliases business. - We don't even have the infrastructure to get rid of aliases, because all warnings in package attributes are disallowed by Nixpkgs CI tooling, last I checked. Before re-disabling this, make sure that - An actually helpful deprecation process is in place. - Aliases are still allowed when `nixos-lib.runTests` and `pkgs.testers.runNixOSTest` are invoked by external projects. For instance, `all-tests.nix` could provide such an override (e.g. with `newScope`).
2023-10-29nixos/lib/testing/run: expose `rawTestDerivation`Raito Bezarius1-13/+23
For `testBuildFailure` and similar functions, we need a full blown derivation and not a lazy one. This is an internal option for test framework developers.
2023-10-29nixos/test-driver: provide a global timeoutRaito Bezarius1-0/+13
Since the debut of the test-driver, we didn't obtain a race timer with the test execution to ensure that tests doesn't run beyond a certain amount of time. This is particularly important when you are running into hanging tests which cannot be detected by current facilities (requires more pvpanic wiring up, QMP API stuff, etc.). Two easy examples: - Some QEMU tests may get stuck in some situation and run for more than 24 hours → we default to 1 hour max. - Some QEMU tests may panic in the wrong place, e.g. UEFI firmware or worse → end users can set a "reasonable" amount of time And then, we should let the retry logic retest them until they succeed and adjust their global timeouts. Of course, this does not help with the fact that the timeout may need to be a function of the actual busyness of the machine running the tests. This is only one step towards increased reliability.
2023-10-06nixos/testing: fix eval for tests created without make-python-test.nixK9001-7/+9
We have too many test entrypoints and this is a problem.
2023-10-06nixosTest: fix tests defined with pkgs.nixosTestK9001-2/+2
2023-09-03nixos/testing/driver: Copy cross fix from `modules/misc/nixpkgs.nix`Artturin1-1/+6
f7077ba1311a49c91493a6696f4ec0dd568f297a Fixes `error: makeWrapper/makeShellWrapper must be in nativeBuildInputs` `pkgsCross.aarch64-multiplatform.nixosTests.sway`
2023-07-10integration test driver: Auto-generate integration test driver's machineJacek Galowicz1-2/+2
method documentation of nixos docs from python doc strings
2023-07-09integration test driver: Adapt test script checking outputJacek Galowicz1-1/+5
2023-06-29nixosTest: adds support for lib.extendArthur Gautier1-0/+1
When lib overrides were used, before this commit, they would not be made available in the configuration evaluation of nixosTest's nodes. Sample code: ``` nix let pkgs = import ./. { overlays = [ (new: old: { lib = old.lib.extend (self: super: { sorry_dave = builtins.trace "There are no pod bay doors" "sorry dave"; }); }) ]; }; in pkgs.testers.nixosTest { name = "demo lib overlay"; nodes = { machine = { lib, ... }: { environment.etc."got-lib-overlay".text = lib.sorry_dave; }; }; testScript = { nodes }: '' start_all() machine.succeed('grep dave /etc/got-lib-overlay') ''; } ```
2023-05-24nixos/qemu-vm: quoted string reformatGraham Dennis1-1/+1
2023-05-24nixos/qemu-vm: add option for named network interfacesGraham Dennis2-16/+32
Adds a new option to the virtualisation modules that enables specifying explicitly named network interfaces in QEMU VMs. The existing `virtualisation.vlans` option is still supported for cases where the name of the network interface is irrelevant.
2023-05-11nixos/testing/nodes.nix: Do not rely on disabledModulesRobert Hensing1-2/+2
It's just not necessary.
2023-05-11nixos/testing: Add node.pkgsReadOnly escape hatchRobert Hensing1-2/+23
By adding this option indirection, a test can declare all by itself that it needs a custom nixpkgs. This is a more convenient way of going about this when the caller of the test framework receives a `node.pkgs` unconditionally.
2023-05-11nixos/testing: Add node.pkgs optionRobert Hensing1-1/+21
By factoring out this logic, it's easier for other projects to make use of it this optimization too (and do it correctly).
2023-05-11nixos/all-tests.nix: Add readOnlyPkgs moduleRobert Hensing1-0/+1
2023-05-10nixos/all-tests.nix: Set nixpkgs.systemRobert Hensing1-5/+8
2023-05-06lib/modules: Move class out of specialArgsRobert Hensing1-1/+1
2023-05-06nixos,nixpkgs: Add module classesRobert Hensing1-1/+4
This allows modules that declare their class to be checked. While that's not most user modules, frameworks can take advantage of this by setting declaring the module class for their users. That way, the mistake of importing a module into the wrong hierarchy can be reported more clearly in some cases.
2023-04-25nixosTest: remove hostname limitationsAlyssa Ross1-20/+11
2023-02-16nixos/lib/testing: set default timeout for VM testsK9001-1/+1
2023-01-30Revert #178290: nixos/virtualisation: add optionVladimír Čunát2-30/+15
...for explicitly named network interfaces This reverts commit 6ae3e7695e27a4f7afb1d2017f5967d5e82f4c00. (and evaluation fixups 08d26bbb726 7aed90a969c) Some of the tests fail or time out after the merge.
2023-01-22nixos/virtualisation: add option for explicitly named network interfacesAndrew Hoff2-15/+30
Adds a new option to the virtualisation modules that enables specifying explicitly named network interfaces in QEMU VMs. The existing `virtualisation.vlans` is still supported for cases where the name of the network interface is irrelevant.
2023-01-22nixos/documentation: deprecate docbook option docspennae1-1/+1
following the plan in https://github.com/NixOS/nixpkgs/pull/189318#discussion_r961764451 also adds an activation script to print the warning during activation instead of during build, otherwise folks using the new CLI that hides build logs by default might never see the warning.
2022-12-22nixos/tests: remove minimal-kernel moduleNaïm Favier2-10/+3
It's not used, doesn't build, and seems like the only reason to have `manualConfig` take `stdenv` as an argument.
2022-10-16nixos/lib/testing: Delay nodes.machine.~config~ migrationRobert Hensing1-1/+1
Provide a window during which both solutions are valid without warnings, in order to fight warning fatigue, and not to push 3rd party repo maintainers to add unnecessary compat code.
2022-09-29nixos/doc: Disambiguate test option idsRobert Hensing4-10/+10
Changing this later on would break external links into the manual.
2022-09-29nixos/testing: Fix release.nix tests evaluationRobert Hensing2-6/+2
Fixes the problem introduced by 12b3066aae46a8ccc3d21f54f668a3f4be344332 which caused nixos/release.nix to return the wrong attributes, while intending to only affect nixos/lib's runTest. This also removes callTest from the test options, because callTest is only ever invoked by all-tests.nix.
2022-09-28lib.nixos.runTest: Return the test itselfRobert Hensing1-1/+1
(NB: the lib in the flake) Correcting a mistake I made when removing the test matrix code from https://github.com/NixOS/nixpkgs/pull/191540
2022-09-24nixos/testing/network.nix: Add network config to specialisationsRobert Hensing2-1/+2
2022-09-24nixos/testing: Improve interactive docsRobert Hensing1-1/+21
2022-09-24nixos/testing: Add nodes.config backcompat to nodes module argumentRobert Hensing1-1/+1
2022-09-24nixos/testing: Improve option docsRobert Hensing8-26/+109
2022-09-24nixos/testing: Support mypy through regular mechanismsRobert Hensing1-3/+1
Rebase / forward port of 2c8bbf33fd84d2fd9de70d66c1f50ac1b6123dd8
2022-09-24nixos/testing/meta.nix: Add options, some optionalRobert Hensing1-3/+19
2022-09-24nixos/testing/network.nix: Fix specialisations onlyShorthandRobert Hensing1-5/+9
2022-09-24nixos/testing: Embrace callTestRobert Hensing1-9/+11
My conception of its input was wrong. It is quite a useful construct, even if its name is a bit weird.
2022-09-24nixos/testing: Extract nixos-test-base.nix NixOS moduleRobert Hensing2-12/+24
2022-09-24nixos/testing: Move entrypoint to nixos/lib + docRobert Hensing1-0/+24
2022-09-21nixos/testing/network.nix: Fix for specialisations and specialArgs.nameRobert Hensing1-56/+93
Before this, it relied on being able to `imports` from the `name` module argument, which wasn't really necessary and required a potentially quite breaking change. We can revert that now.
2022-09-21nixos/testing: Add pkgs parameterRobert Hensing1-0/+11
This parameter is for packages to use in VMs, unlike hostPkgs.
2022-09-21nixos/testing/network.nix: Avoid deprecated .configRobert Hensing1-1/+1