diff options
author | annalee <150648636+a-n-n-a-l-e-e@users.noreply.github.com> | 2024-03-28 13:02:34 +0000 |
---|---|---|
committer | annalee <150648636+a-n-n-a-l-e-e@users.noreply.github.com> | 2024-03-28 13:02:34 +0000 |
commit | 675fb1648e6925ca87437bc98fce2077648e8be3 (patch) | |
tree | 7f48f889770fe72ecdc74d853af022bb8bc214d7 /doc/languages-frameworks/python.section.md | |
parent | 035b7f36aa1367c00845f669ef0617e55d31104f (diff) | |
parent | 69de1e1beb4fa322688a2bac5761e59a2b17e15e (diff) |
Merge remote-tracking branch 'upstream/staging-next' into staging
Conflicts: pkgs/development/python-modules/apsw/default.nix pkgs/development/python-modules/deal/default.nix pkgs/development/python-modules/kubernetes/default.nix
Diffstat (limited to 'doc/languages-frameworks/python.section.md')
-rw-r--r-- | doc/languages-frameworks/python.section.md | 116 |
1 files changed, 78 insertions, 38 deletions
diff --git a/doc/languages-frameworks/python.section.md b/doc/languages-frameworks/python.section.md index 6bd1ad74435e4..3b737333308da 100644 --- a/doc/languages-frameworks/python.section.md +++ b/doc/languages-frameworks/python.section.md @@ -254,17 +254,19 @@ The next example shows a non trivial overriding of the `blas` implementation to be used through out all of the Python package set: ```nix -python3MyBlas = pkgs.python3.override { - packageOverrides = self: super: { - # We need toPythonModule for the package set to evaluate this - blas = super.toPythonModule(super.pkgs.blas.override { - blasProvider = super.pkgs.mkl; - }); - lapack = super.toPythonModule(super.pkgs.lapack.override { - lapackProvider = super.pkgs.mkl; - }); +{ + python3MyBlas = pkgs.python3.override { + packageOverrides = self: super: { + # We need toPythonModule for the package set to evaluate this + blas = super.toPythonModule(super.pkgs.blas.override { + blasProvider = super.pkgs.mkl; + }); + lapack = super.toPythonModule(super.pkgs.lapack.override { + lapackProvider = super.pkgs.mkl; + }); + }; }; -}; +} ``` This is particularly useful for numpy and scipy users who want to gain speed with other blas implementations. @@ -322,7 +324,9 @@ python3Packages.buildPythonApplication rec { This is then added to `all-packages.nix` just as any other application would be. ```nix -luigi = callPackage ../applications/networking/cluster/luigi { }; +{ + luigi = callPackage ../applications/networking/cluster/luigi { }; +} ``` Since the package is an application, a consumer doesn't need to care about @@ -342,7 +346,9 @@ the attribute in `python-packages.nix`, and the `toPythonApplication` shall be applied to the reference: ```nix -youtube-dl = with python3Packages; toPythonApplication youtube-dl; +{ + youtube-dl = with python3Packages; toPythonApplication youtube-dl; +} ``` #### `toPythonModule` function {#topythonmodule-function} @@ -354,10 +360,12 @@ bindings should be made available from `python-packages.nix`. The modifications. ```nix -opencv = toPythonModule (pkgs.opencv.override { - enablePython = true; - pythonPackages = self; -}); +{ + opencv = toPythonModule (pkgs.opencv.override { + enablePython = true; + pythonPackages = self; + }); +} ``` Do pay attention to passing in the right Python version! @@ -1162,7 +1170,8 @@ a good indication that the package is not in a valid state. Pytest is the most common test runner for python repositories. A trivial test run would be: -``` +```nix +{ nativeCheckInputs = [ pytest ]; checkPhase = '' runHook preCheck @@ -1171,6 +1180,7 @@ test run would be: runHook postCheck ''; +} ``` However, many repositories' test suites do not translate well to nix's build @@ -1178,7 +1188,8 @@ sandbox, and will generally need many tests to be disabled. To filter tests using pytest, one can do the following: -``` +```nix +{ nativeCheckInputs = [ pytest ]; # avoid tests which need additional data or touch network checkPhase = '' @@ -1188,6 +1199,7 @@ To filter tests using pytest, one can do the following: runHook postCheck ''; +} ``` `--ignore` will tell pytest to ignore that file or directory from being @@ -1213,7 +1225,8 @@ when a package may need many items disabled to run the test suite. Using the example above, the analogous `pytestCheckHook` usage would be: -``` +```nix +{ nativeCheckInputs = [ pytestCheckHook ]; @@ -1233,12 +1246,14 @@ Using the example above, the analogous `pytestCheckHook` usage would be: disabledTestPaths = [ "tests/test_failing.py" ]; +} ``` This is especially useful when tests need to be conditionally disabled, for example: -``` +```nix +{ disabledTests = [ # touches network "download" @@ -1250,6 +1265,7 @@ for example: # can fail when building with other packages "socket" ]; +} ``` Trying to concatenate the related strings to disable tests in a regular @@ -1263,20 +1279,24 @@ all packages have test suites that can be run easily, and some have none at all. To help ensure the package still works, [`pythonImportsCheck`](#using-pythonimportscheck) can attempt to import the listed modules. -``` +```nix +{ pythonImportsCheck = [ "requests" "urllib" ]; +} ``` roughly translates to: -``` +```nix +{ postCheck = '' PYTHONPATH=$out/${python.sitePackages}:$PYTHONPATH python -c "import requests; import urllib" ''; +} ``` However, this is done in its own phase, and not dependent on whether [`doCheck = true;`](#var-stdenv-doCheck). @@ -1307,7 +1327,8 @@ pkg3>=1.0,<=2.0 we can do: -``` +```nix +{ nativeBuildInputs = [ pythonRelaxDepsHook ]; @@ -1318,6 +1339,7 @@ we can do: pythonRemoveDeps = [ "pkg2" ]; +} ``` which would result in the following `requirements.txt` file: @@ -1330,9 +1352,11 @@ pkg3 Another option is to pass `true`, that will relax/remove all dependencies, for example: -``` +```nix +{ nativeBuildInputs = [ pythonRelaxDepsHook ]; pythonRelaxDeps = true; +} ``` which would result in the following `requirements.txt` file: @@ -1357,7 +1381,8 @@ work with any of the [existing hooks](#setup-hooks). `unittestCheckHook` is a hook which will substitute the setuptools `test` command for a [`checkPhase`](#ssec-check-phase) which runs `python -m unittest discover`: -``` +```nix +{ nativeCheckInputs = [ unittestCheckHook ]; @@ -1365,6 +1390,7 @@ work with any of the [existing hooks](#setup-hooks). unittestFlagsArray = [ "-s" "tests" "-v" ]; +} ``` #### Using sphinxHook {#using-sphinxhook} @@ -1374,7 +1400,8 @@ using the popular Sphinx documentation generator. It is setup to automatically find common documentation source paths and render them using the default `html` style. -``` +```nix +{ outputs = [ "out" "doc" @@ -1383,13 +1410,15 @@ render them using the default `html` style. nativeBuildInputs = [ sphinxHook ]; +} ``` The hook will automatically build and install the artifact into the `doc` output, if it exists. It also provides an automatic diversion for the artifacts of the `man` builder into the `man` target. -``` +```nix +{ outputs = [ "out" "doc" @@ -1401,14 +1430,17 @@ for the artifacts of the `man` builder into the `man` target. "singlehtml" "man" ]; +} ``` Overwrite `sphinxRoot` when the hook is unable to find your documentation source root. -``` +```nix +{ # Configure sphinxRoot for uncommon paths sphinxRoot = "weird/docs/path"; +} ``` The hook is also available to packages outside the python ecosystem by @@ -1753,6 +1785,7 @@ folder and not downloaded again. If you need to change a package's attribute(s) from `configuration.nix` you could do: ```nix +{ nixpkgs.config.packageOverrides = super: { python3 = super.python3.override { packageOverrides = python-self: python-super: { @@ -1767,6 +1800,7 @@ If you need to change a package's attribute(s) from `configuration.nix` you coul }; }; }; +} ``` `python3Packages.twisted` is now globally overridden. @@ -1779,11 +1813,13 @@ To modify only a Python package set instead of a whole Python derivation, use this snippet: ```nix +{ myPythonPackages = python3Packages.override { overrides = self: super: { - twisted = ...; + twisted = <...>; }; - } + }; +} ``` ### How to override a Python package using overlays? {#how-to-override-a-python-package-using-overlays} @@ -1819,7 +1855,7 @@ final: prev: { ( python-final: python-prev: { foo = python-prev.foo.overridePythonAttrs (oldAttrs: { - ... + # ... }); } ) @@ -1846,7 +1882,7 @@ The Python interpreters are by default not built with optimizations enabled, bec the builds are in that case not reproducible. To enable optimizations, override the interpreter of interest, e.g using -``` +```nix let pkgs = import ./. {}; mypython = pkgs.python3.override { @@ -1864,17 +1900,21 @@ Some packages define optional dependencies for additional features. With `extras-require`, while PEP 621 calls these `optional-dependencies`. ```nix -optional-dependencies = { - complete = [ distributed ]; -}; +{ + optional-dependencies = { + complete = [ distributed ]; + }; +} ``` and letting the package requiring the extra add the list to its dependencies ```nix -dependencies = [ - ... -] ++ dask.optional-dependencies.complete; +{ + dependencies = [ + # ... + ] ++ dask.optional-dependencies.complete; +} ``` This method is using `passthru`, meaning that changing `optional-dependencies` of a package won't cause it to rebuild. |