doc: recommend versionCheckHook whenever relevant

3 files changed, 14 insertions, 34 deletions

diff --git a/doc/build-helpers/testers.chapter.md b/doc/build-helpers/testers.chapter.md
index 34cfc00a4953c..a10e60de8c6d7 100644
--- a/doc/build-helpers/testers.chapter.md
+++ b/doc/build-helpers/testers.chapter.md
@@ -120,9 +120,10 @@ It has two modes:
 
 Checks that the output from running a command contains the specified version string in it as a whole word.
 
-Although simplistic, this test assures that the main program can run.
-While there's no substitute for a real test case, it does catch dynamic linking errors and such.
-It also provides some protection against accidentally building the wrong version, for example when using an "old" hash in a fixed-output derivation.
+NOTE: In most cases, [`versionCheckHook`](#versioncheckhook) should be preferred, but this function is provided and documented here anyway. The motivation for adding either tests would be:
+
+- Catch dynamic linking errors and such and missing environment variables that should be added by wrapping.
+- Probable protection against accidentally building the wrong version, for example when using an "old" hash in a fixed-output derivation.
 
 By default, the command to be run will be inferred from the given `package` attribute:
 it will check `meta.mainProgram` first, and fall back to `pname` or `name`.
diff --git a/doc/stdenv/passthru.chapter.md b/doc/stdenv/passthru.chapter.md
index 008b908b2f48e..142b978d58808 100644
--- a/doc/stdenv/passthru.chapter.md
+++ b/doc/stdenv/passthru.chapter.md
@@ -75,40 +75,17 @@ The Nixpkgs systems for continuous integration [Hydra](https://hydra.nixos.org/)
 #### Package tests {#var-passthru-tests-packages}
 []{#var-meta-tests-packages} <!-- legacy anchor -->
 
-Tests that are part of the source package, if they run quickly, are typically executed in the [`installCheckPhase`](#var-stdenv-phases).
-This phase is also suitable for performing a `--version` test for packages that support such flag.
-Most programs distributed by Nixpkgs support such a `--version` flag, and successfully calling the program with that flag indicates that the package at least got compiled properly.
+Besides tests provided by upstream, that you run in the [`checkPhase`](#ssec-check-phase), you may want to define tests derivations in the `passthru.tests` attribute, which won't change the build. `passthru.tests` have several advantages over running tests during any of the [standard phases](#sec-stdenv-phases):
 
-:::{.example #ex-checking-build-installCheckPhase}
+- They access the package as consumers would, independently from the environment in which it was built
+- They can be run and debugged without rebuilding the package, which is useful if that takes a long time
+- They don't add overhead to each build, as opposed checks added to the [`distPhase`](#ssec-distribution-phase), such as [`versionCheckHook`](#versioncheckhook).
 
-## Checking builds with `installCheckPhase`
+It is also possible to use `passthru.tests` to test the version with [`testVersion`](#tester-testVersion), but since that is pretty trivial and recommended thing to do, we recommend using [`versionCheckHook`](#versioncheckhook) for that, which has the following advantages over `passthru.tests`:
 
-When building `git`, a rudimentary test for successful compilation would be running `git --version`:
-
-```nix
-stdenv.mkDerivation (finalAttrs: {
-  pname = "git";
-  version = "1.2.3";
-  # ...
-  doInstallCheck = true;
-  installCheckPhase = ''
-    runHook preInstallCheck
-    echo checking if 'git --version' mentions ${finalAttrs.version}
-    $out/bin/git --version | grep ${finalAttrs.version}
-    runHook postInstallCheck
-  '';
-  # ...
-})
-```
-:::
-
-However, tests that are non-trivial will better fit into `passthru.tests` because they:
-
-- Access the package as consumers would, independently from the environment in which it was built
-- Can be run and debugged without rebuilding the package, which is useful if that takes a long time
-- Don't add overhad to each build, as opposed to `installCheckPhase`
-
-It is also possible to use `passthru.tests` to test the version with [`testVersion`](#tester-testVersion).
+- If the `versionCheckPhase` (the phase defined by [`versionCheckHook`](#versioncheckhook)) fails, it triggers a failure which can't be ignored if you use the package, or if you find out about it in a [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) report.
+- Sometimes packages become silently broken - meaning they fail to launch but their build passes because they don't perform any tests in the `checkPhase`. If you use this tool infrequently, such a silent breakage may rot in your system / profile configuration, and you will not notice the failure until you will want to use this package. Testing such basic functionality ensures you have to deal with the failure when you update your system / profile.
+- When you open a PR, [ofborg](https://github.com/NixOS/ofborg)'s CI _will_ run `passthru.tests` of [packages that are directly changed by your PR (according to your commits' messages)](https://github.com/NixOS/ofborg?tab=readme-ov-file#automatic-building), but if you'd want to use the [`@ofborg build`](https://github.com/NixOS/ofborg?tab=readme-ov-file#build) command for dependent packages, you won't have to specify in addition the `.tests` attribute of the packages you want to build, and no body will be able to avoid these tests.
 
 <!-- NOTE(@fricklerhandwerk): one may argue whether that testing guide should rather be in the user's manual -->
 For more on how to write and run package tests for Nixpkgs, see the [testing section in the package contributor guide](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests).
diff --git a/doc/stdenv/stdenv.chapter.md b/doc/stdenv/stdenv.chapter.md
index 400fa2de1e764..6d26f1dad3d46 100644
--- a/doc/stdenv/stdenv.chapter.md
+++ b/doc/stdenv/stdenv.chapter.md
@@ -762,6 +762,8 @@ Before and after running `make`, the hooks `preBuild` and `postBuild` are called
 
 The check phase checks whether the package was built correctly by running its test suite. The default `checkPhase` calls `make $checkTarget`, but only if the [`doCheck` variable](#var-stdenv-doCheck) is enabled.
 
+It is highly recommended, for packages' sources that are not distributed with any tests, to at least use [`versionCheckHook`](#versioncheckhook) to test that the resulting executable is basically functional.
+
 #### Variables controlling the check phase {#variables-controlling-the-check-phase}
 
 ##### `doCheck` {#var-stdenv-doCheck}