summary refs log tree commit diff
path: root/doc/builders
diff options
context:
space:
mode:
Diffstat (limited to 'doc/builders')
-rw-r--r--doc/builders/fetchers.chapter.md14
-rw-r--r--doc/builders/special.xml1
-rw-r--r--doc/builders/special/invalidateFetcherByDrvHash.section.md31
3 files changed, 43 insertions, 3 deletions
diff --git a/doc/builders/fetchers.chapter.md b/doc/builders/fetchers.chapter.md
index 30d0653448528..e36724f295f9c 100644
--- a/doc/builders/fetchers.chapter.md
+++ b/doc/builders/fetchers.chapter.md
@@ -1,8 +1,16 @@
 # Fetchers {#chap-pkgs-fetchers}
 
-When using Nix, you will frequently need to download source code and other files from the internet. Nixpkgs comes with a few helper functions that allow you to fetch fixed-output derivations in a structured way.
+When using Nix, you will frequently need to download source code and other files from the internet. For this purpose, Nix provides the [_fixed output derivation_](https://nixos.org/manual/nix/stable/#fixed-output-drvs) feature and Nixpkgs provides various functions that implement the actual fetching from various protocols and services.
 
-The two fetcher primitives are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
+## Caveats
+
+Because fixed output derivations are _identified_ by their hash, a common mistake is to update a fetcher's URL or a version parameter, without updating the hash. **This will cause the old contents to be used.** So remember to always invalidate the hash argument.
+
+For those who develop and maintain fetchers, a similar problem arises with changes to the implementation of a fetcher. These may cause a fixed output derivation to fail, but won't normally be caught by tests because the supposed output is already in the store or cache. For the purpose of testing, you can use a trick that is embodied by the [`invalidateFetcherByDrvHash`](#sec-pkgs-invalidateFetcherByDrvHash) function. It uses the derivation `name` to create a unique output path per fetcher implementation, defeating the caching precisely where it would be harmful.
+
+## `fetchurl` and `fetchzip` {#fetchurl}
+
+Two basic fetchers are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
 
 ```nix
 { stdenv, fetchurl }:
@@ -20,7 +28,7 @@ The main difference between `fetchurl` and `fetchzip` is in how they store the c
 
 `fetchpatch` works very similarly to `fetchurl` with the same arguments expected. It expects patch files as a source and performs normalization on them before computing the checksum. For example it will remove comments or other unstable parts that are sometimes added by version control systems and can change over time.
 
-Other fetcher functions allow you to add source code directly from a VCS such as subversion or git. These are mostly straightforward nambes based on the name of the command used with the VCS system. Because they give you a working repository, they act most like `fetchzip`.
+Most other fetchers return a directory rather than a single file.
 
 ## `fetchsvn` {#fetchsvn}
 
diff --git a/doc/builders/special.xml b/doc/builders/special.xml
index 8902ce5c81329..2f84599cdd4f3 100644
--- a/doc/builders/special.xml
+++ b/doc/builders/special.xml
@@ -7,4 +7,5 @@
  </para>
  <xi:include href="special/fhs-environments.section.xml" />
  <xi:include href="special/mkshell.section.xml" />
+ <xi:include href="special/invalidateFetcherByDrvHash.section.xml" />
 </chapter>
diff --git a/doc/builders/special/invalidateFetcherByDrvHash.section.md b/doc/builders/special/invalidateFetcherByDrvHash.section.md
new file mode 100644
index 0000000000000..7c2f03a64b7b3
--- /dev/null
+++ b/doc/builders/special/invalidateFetcherByDrvHash.section.md
@@ -0,0 +1,31 @@
+
+## `invalidateFetcherByDrvHash` {#sec-pkgs-invalidateFetcherByDrvHash}
+
+Use the derivation hash to invalidate the output via name, for testing.
+
+Type: `(a@{ name, ... } -> Derivation) -> a -> Derivation`
+
+Normally, fixed output derivations can and should be cached by their output
+hash only, but for testing we want to re-fetch everytime the fetcher changes.
+
+Changes to the fetcher become apparent in the drvPath, which is a hash of
+how to fetch, rather than a fixed store path.
+By inserting this hash into the name, we can make sure to re-run the fetcher
+every time the fetcher changes.
+
+This relies on the assumption that Nix isn't clever enough to reuse its
+database of local store contents to optimize fetching.
+
+You might notice that the "salted" name derives from the normal invocation,
+not the final derivation. `invalidateFetcherByDrvHash` has to invoke the fetcher
+function twice: once to get a derivation hash, and again to produce the final
+fixed output derivation.
+
+Example:
+
+    tests.fetchgit = invalidateFetcherByDrvHash fetchgit {
+      name = "nix-source";
+      url = "https://github.com/NixOS/nix";
+      rev = "9d9dbe6ed05854e03811c361a3380e09183f4f4a";
+      sha256 = "sha256-7DszvbCNTjpzGRmpIVAWXk20P0/XTrWZ79KSOGLrUWY=";
+    };