diff options
Diffstat (limited to 'nixos')
238 files changed, 5238 insertions, 1675 deletions
diff --git a/nixos/doc/manual/README b/nixos/doc/manual/README index 587f6275197ae..120c127d7af20 100644 --- a/nixos/doc/manual/README +++ b/nixos/doc/manual/README @@ -1,12 +1,3 @@ -To build the manual, you need Nix installed on your system (no need -for NixOS). To install Nix, follow the instructions at +Moved to: ./contributing-to-this-manual.xml. Link: - https://nixos.org/nix/download.html - -When you have Nix on your system, in the root directory of the project -(i.e., `nixpkgs`), run: - - nix-build nixos/release.nix -A manual.x86_64-linux - -When this command successfully finishes, it will tell you where the -manual got generated. +https://nixos.org/manual/nixos/unstable/#chap-contributing diff --git a/nixos/doc/manual/administration/service-mgmt.xml b/nixos/doc/manual/administration/service-mgmt.xml index 1c5d48a5bcf07..863b0d47f6c7b 100644 --- a/nixos/doc/manual/administration/service-mgmt.xml +++ b/nixos/doc/manual/administration/service-mgmt.xml @@ -6,7 +6,7 @@ <title>Service Management</title> <para> In NixOS, all system services are started and monitored using the systemd - program. Systemd is the “init” process of the system (i.e. PID 1), the + program. systemd is the “init” process of the system (i.e. PID 1), the parent of all other processes. It manages a set of so-called “units”, which can be things like system services (programs), but also mount points, swap files, devices, targets (groups of units) and more. Units can have @@ -16,10 +16,17 @@ dependencies of this unit cause all system services to be started, file systems to be mounted, swap files to be activated, and so on. </para> - <para> - The command <command>systemctl</command> is the main way to interact with - <command>systemd</command>. Without any arguments, it shows the status of - active units: + <section xml:id="sect-nixos-systemd-general"> + <title>Interacting with a running systemd</title> + <para> + The command <command>systemctl</command> is the main way to interact with + <command>systemd</command>. The following paragraphs demonstrate ways to + interact with any OS running systemd as init system. NixOS is of no + exception. The <link xlink:href="#sect-nixos-systemd-nixos">next section + </link> explains NixOS specific things worth knowing. + </para> + <para> + Without any arguments, <literal>systmctl</literal> the status of active units: <screen> <prompt>$ </prompt>systemctl -.mount loaded active mounted / @@ -28,10 +35,10 @@ sshd.service loaded active running SSH Daemon graphical.target loaded active active Graphical Interface <replaceable>...</replaceable> </screen> - </para> - <para> - You can ask for detailed status information about a unit, for instance, the - PostgreSQL database service: + </para> + <para> + You can ask for detailed status information about a unit, for instance, the + PostgreSQL database service: <screen> <prompt>$ </prompt>systemctl status postgresql.service postgresql.service - PostgreSQL Server @@ -62,11 +69,72 @@ Jan 07 15:55:57 hagbard systemd[1]: Started PostgreSQL Server. <prompt># </prompt>systemctl start postgresql.service <prompt># </prompt>systemctl restart postgresql.service </screen> - These operations are synchronous: they wait until the service has finished - starting or stopping (or has failed). Starting a unit will cause the - dependencies of that unit to be started as well (if necessary). - </para> -<!-- - cgroups: each service and user session is a cgroup + These operations are synchronous: they wait until the service has finished + starting or stopping (or has failed). Starting a unit will cause the + dependencies of that unit to be started as well (if necessary). + </para> + <!-- TODO: document cgroups, draft: + each service and user session is a cgroup -- cgroup resource management --> + - cgroup resource management --> + </section> + <section xml:id="sect-nixos-systemd-nixos"> + <title>systemd in NixOS</title> + <para> + Packages in Nixpkgs sometimes provide systemd units with them, usually in + e.g <literal>#pkg-out#/lib/systemd/</literal>. Putting such a package in + <literal>environment.systemPackages</literal> doesn't make the service + available to users or the system. + </para> + <para> + In order to enable a systemd <emphasis>system</emphasis> service with + provided upstream package, use (e.g): +<programlisting> +<xref linkend="opt-systemd.packages"/> = [ pkgs.packagekit ]; +</programlisting> + </para> + <para> + Usually NixOS modules written by the community do the above, plus take care of + other details. If a module was written for a service you are interested in, + you'd probably need only to use + <literal>services.#name#.enable = true;</literal>. These services are defined + in Nixpkgs' + <link xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/modules"> + <literal>nixos/modules/</literal> directory </link>. In case the service is + simple enough, the above method should work, and start the service on boot. + </para> + <para> + <emphasis>User</emphasis> systemd services on the other hand, should be + treated differently. Given a package that has a systemd unit file at + <literal>#pkg-out#/lib/systemd/user/</literal>, using + <xref linkend="opt-systemd.packages"/> will make you able to start the service via + <literal>systemctl --user start</literal>, but it won't start automatically on login. + <!-- TODO: Document why systemd.packages doesn't work for user services or fix this. + https://github.com/NixOS/nixpkgs/blob/2cd6594a8710a801038af2b72348658f732ce84a/nixos/modules/system/boot/systemd-lib.nix#L177-L198 + + This has been talked over at https://discourse.nixos.org/t/how-to-enable-upstream-systemd-user-services-declaratively/7649/5 + --> + However, You can imperatively enable it by adding the package's attribute to + <link linkend="opt-environment.systemPackages"> + <literal>systemd.packages</literal></link> and then do this (e.g): +<screen> +<prompt>$ </prompt>mkdir -p ~/.config/systemd/user/default.target.wants +<prompt>$ </prompt>ln -s /run/current-system/sw/lib/systemd/user/syncthing.service ~/.config/systemd/user/default.target.wants/ +<prompt>$ </prompt>systemctl --user daemon-reload +<prompt>$ </prompt>systemctl --user enable syncthing.service +</screen> + If you are interested in a timer file, use <literal>timers.target.wants</literal> + instead of <literal>default.target.wants</literal> in the 1st and 2nd command. + </para> + <para> + Using <literal>systemctl --user enable syncthing.service</literal> instead of + the above, will work, but it'll use the absolute path of + <literal>syncthing.service</literal> for the symlink, and this path is in + <literal>/nix/store/.../lib/systemd/user/</literal>. Hence + <link xlink:href="#sec-nix-gc">garbage collection</link> will remove that file + and you will wind up with a broken symlink in your systemd configuration, which + in turn will not make the service / timer start on login. + </para> + </section> </chapter> + diff --git a/nixos/doc/manual/configuration/configuration.xml b/nixos/doc/manual/configuration/configuration.xml index 6eb8f50bacac6..44ff1f1b571a7 100644 --- a/nixos/doc/manual/configuration/configuration.xml +++ b/nixos/doc/manual/configuration/configuration.xml @@ -18,6 +18,7 @@ <xi:include href="user-mgmt.xml" /> <xi:include href="file-systems.xml" /> <xi:include href="x-windows.xml" /> + <xi:include href="wayland.xml" /> <xi:include href="gpu-accel.xml" /> <xi:include href="xfce.xml" /> <xi:include href="networking.xml" /> diff --git a/nixos/doc/manual/configuration/gpu-accel.xml b/nixos/doc/manual/configuration/gpu-accel.xml index 95ee13f4796e8..3695a5ff445a4 100644 --- a/nixos/doc/manual/configuration/gpu-accel.xml +++ b/nixos/doc/manual/configuration/gpu-accel.xml @@ -65,16 +65,16 @@ Platform Vendor Advanced Micro Devices, Inc.</screen> <title>AMD</title> <para> - Modern AMD <link - xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics - Core Next</link> (GCN) GPUs are supported through the - <package>rocm-opencl-icd</package> package. Adding this package to - <xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL - support: - - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - rocm-opencl-icd -];</programlisting> + Modern AMD <link + xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics + Core Next</link> (GCN) GPUs are supported through the + <package>rocm-opencl-icd</package> package. Adding this package to + <xref linkend="opt-hardware.opengl.extraPackages"/> enables OpenCL + support: + + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + rocm-opencl-icd + ];</programlisting> </para> </section> @@ -100,9 +100,9 @@ Platform Vendor Advanced Micro Devices, Inc.</screen> support. For example, for Gen8 and later GPUs, the following configuration can be used: - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - intel-compute-runtime -];</programlisting> + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + intel-compute-runtime + ];</programlisting> </para> </section> @@ -173,31 +173,31 @@ GPU1: <title>AMD</title> <para> - Modern AMD <link - xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics - Core Next</link> (GCN) GPUs are supported through either radv, which is - part of <package>mesa</package>, or the <package>amdvlk</package> package. - Adding the <package>amdvlk</package> package to - <xref linkend="opt-hardware.opengl.extraPackages"/> makes both drivers - available for applications and lets them choose. A specific driver can - be forced as follows: - - <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ - pkgs.<package>amdvlk</package> -]; - -# To enable Vulkan support for 32-bit applications, also add: -<xref linkend="opt-hardware.opengl.extraPackages32"/> = [ - pkgs.driversi686Linux.<package>amdvlk</package> -]; - -# For amdvlk -<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = - "/run/opengl-driver/share/vulkan/icd.d/amd_icd64.json"; -# For radv -<xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = - "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json"; -</programlisting> + Modern AMD <link + xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics + Core Next</link> (GCN) GPUs are supported through either radv, which is + part of <package>mesa</package>, or the <package>amdvlk</package> package. + Adding the <package>amdvlk</package> package to + <xref linkend="opt-hardware.opengl.extraPackages"/> makes both drivers + available for applications and lets them choose. A specific driver can + be forced as follows: + + <programlisting><xref linkend="opt-hardware.opengl.extraPackages"/> = [ + pkgs.<package>amdvlk</package> + ]; + + # To enable Vulkan support for 32-bit applications, also add: + <xref linkend="opt-hardware.opengl.extraPackages32"/> = [ + pkgs.driversi686Linux.<package>amdvlk</package> + ]; + + # For amdvlk + <xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = + "/run/opengl-driver/share/vulkan/icd.d/amd_icd64.json"; + # For radv + <xref linkend="opt-environment.variables"/>.VK_ICD_FILENAMES = + "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json"; + </programlisting> </para> </section> </section> diff --git a/nixos/doc/manual/configuration/user-mgmt.xml b/nixos/doc/manual/configuration/user-mgmt.xml index cbec83814c9a3..e83e7b75ef542 100644 --- a/nixos/doc/manual/configuration/user-mgmt.xml +++ b/nixos/doc/manual/configuration/user-mgmt.xml @@ -38,7 +38,7 @@ assigned by setting the user's <link linkend="opt-users.users._name_.hashedPassword">hashedPassword</link> option. A hashed password can be generated using <command>mkpasswd -m - sha-512</command> after installing the <literal>mkpasswd</literal> package. + sha-512</command>. </para> <para> A user ID (uid) is assigned automatically. You can also specify a uid diff --git a/nixos/doc/manual/configuration/wayland.xml b/nixos/doc/manual/configuration/wayland.xml new file mode 100644 index 0000000000000..231c219cf0811 --- /dev/null +++ b/nixos/doc/manual/configuration/wayland.xml @@ -0,0 +1,23 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-wayland"> + <title>Wayland</title> + + <para> + While X11 (see <xref linkend="sec-x11"/>) is still the primary display + technology on NixOS, Wayland support is steadily improving. + Where X11 separates the X Server and the window manager, on Wayland those + are combined: a Wayland Compositor is like an X11 window manager, but also + embeds the Wayland 'Server' functionality. This means it is sufficient to + install a Wayland Compositor such as <package>sway</package> without + separately enabling a Wayland server: +<programlisting> +<xref linkend="opt-programs.sway.enable"/> = true; +</programlisting> + This installs the <package>sway</package> compositor along with some + essential utilities. Now you can start <package>sway</package> from the TTY + console. + </para> +</chapter> diff --git a/nixos/doc/manual/contributing-to-this-manual.xml b/nixos/doc/manual/contributing-to-this-manual.xml new file mode 100644 index 0000000000000..935dd66bc141c --- /dev/null +++ b/nixos/doc/manual/contributing-to-this-manual.xml @@ -0,0 +1,22 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xml:id="chap-contributing"> + <title>Contributing to this documentation</title> + <para> + The DocBook sources of NixOS' manual are in the <filename +xlink:href="https://github.com/NixOS/nixpkgs/tree/master/nixos/doc/manual"> +nixos/doc/manual</filename> subdirectory of the <link +xlink:href="https://github.com/NixOS/nixpkgs">Nixpkgs</link> repository. + </para> + <para> + You can quickly check your edits with the following: + </para> +<screen> +<prompt>$ </prompt>cd /path/to/nixpkgs +<prompt>$ </prompt>nix-build nixos/release.nix -A manual.x86_64-linux +</screen> + <para> + If the build succeeds, the manual will be in + <filename>./result/share/doc/nixos/index.html</filename>. + </para> +</chapter> diff --git a/nixos/doc/manual/default.nix b/nixos/doc/manual/default.nix index 6ca75f869f45c..af7a2e08220c1 100644 --- a/nixos/doc/manual/default.nix +++ b/nixos/doc/manual/default.nix @@ -63,6 +63,7 @@ let "--stringparam html.script './highlightjs/highlight.pack.js ./highlightjs/loader.js'" "--param xref.with.number.and.title 1" "--param toc.section.depth 0" + "--param generate.consistent.ids 1" "--stringparam admon.style ''" "--stringparam callout.graphics.extension .svg" "--stringparam current.docid manual" diff --git a/nixos/doc/manual/installation/installing-from-other-distro.xml b/nixos/doc/manual/installation/installing-from-other-distro.xml index f10a7d6588798..43f69b923d14d 100644 --- a/nixos/doc/manual/installation/installing-from-other-distro.xml +++ b/nixos/doc/manual/installation/installing-from-other-distro.xml @@ -161,6 +161,13 @@ nixpkgs https://nixos.org/channels/nixpkgs-unstable</screen> existing systems without the help of a rescue USB drive or similar. </para> </warning> + <note> + <para> + On some distributions there are separate PATHS for programs intended only for root. + In order for the installation to succeed, you might have to use <literal>PATH="$PATH:/usr/sbin:/sbin"</literal> + in the following command. + </para> + </note> <screen><prompt>$ </prompt>sudo PATH="$PATH" NIX_PATH="$NIX_PATH" `which nixos-install` --root /mnt</screen> <para> Again, please refer to the <literal>nixos-install</literal> step in diff --git a/nixos/doc/manual/installation/upgrading.xml b/nixos/doc/manual/installation/upgrading.xml index 08780051d5f60..15ba5db9a37be 100644 --- a/nixos/doc/manual/installation/upgrading.xml +++ b/nixos/doc/manual/installation/upgrading.xml @@ -14,7 +14,7 @@ <para> <emphasis>Stable channels</emphasis>, such as <literal - xlink:href="https://nixos.org/channels/nixos-20.03">nixos-20.03</literal>. + xlink:href="https://nixos.org/channels/nixos-20.09">nixos-20.09</literal>. These only get conservative bug fixes and package upgrades. For instance, a channel update may cause the Linux kernel on your system to be upgraded from 4.19.34 to 4.19.38 (a minor bug fix), but not from @@ -38,7 +38,7 @@ <para> <emphasis>Small channels</emphasis>, such as <literal - xlink:href="https://nixos.org/channels/nixos-20.03-small">nixos-20.03-small</literal> + xlink:href="https://nixos.org/channels/nixos-20.09-small">nixos-20.09-small</literal> or <literal xlink:href="https://nixos.org/channels/nixos-unstable-small">nixos-unstable-small</literal>. @@ -63,8 +63,8 @@ <para> When you first install NixOS, you’re automatically subscribed to the NixOS channel that corresponds to your installation source. For instance, if you - installed from a 20.03 ISO, you will be subscribed to the - <literal>nixos-20.03</literal> channel. To see which NixOS channel you’re + installed from a 20.09 ISO, you will be subscribed to the + <literal>nixos-20.09</literal> channel. To see which NixOS channel you’re subscribed to, run the following as root: <screen> <prompt># </prompt>nix-channel --list | grep nixos @@ -75,13 +75,13 @@ nixos https://nixos.org/channels/nixos-unstable <prompt># </prompt>nix-channel --add https://nixos.org/channels/<replaceable>channel-name</replaceable> nixos </screen> (Be sure to include the <literal>nixos</literal> parameter at the end.) For - instance, to use the NixOS 20.03 stable channel: + instance, to use the NixOS 20.09 stable channel: <screen> -<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03 nixos +<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09 nixos </screen> If you have a server, you may want to use the “small” channel instead: <screen> -<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.03-small nixos +<prompt># </prompt>nix-channel --add https://nixos.org/channels/nixos-20.09-small nixos </screen> And if you want to live on the bleeding edge: <screen> @@ -132,7 +132,7 @@ nixos https://nixos.org/channels/nixos-unstable kernel, initrd or kernel modules. You can also specify a channel explicitly, e.g. <programlisting> -<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.03; +<xref linkend="opt-system.autoUpgrade.channel"/> = https://nixos.org/channels/nixos-20.09; </programlisting> </para> </section> diff --git a/nixos/doc/manual/man-nixos-install.xml b/nixos/doc/manual/man-nixos-install.xml index b205e23096875..91542d37cbd55 100644 --- a/nixos/doc/manual/man-nixos-install.xml +++ b/nixos/doc/manual/man-nixos-install.xml @@ -50,6 +50,12 @@ </arg> <arg> + <group choice='req'> + <arg choice='plain'><option>--impure</option></arg> + </group> + </arg> + + <arg> <arg choice='plain'> <option>--channel</option> </arg> @@ -102,6 +108,12 @@ <arg> <arg choice='plain'> + <option>--keep-going</option> + </arg> + </arg> + + <arg> + <arg choice='plain'> <option>--help</option> </arg> </arg> @@ -297,6 +309,17 @@ </varlistentry> <varlistentry> <term> + <option>--keep-going</option> + </term> + <listitem> + <para> + Causes Nix to continue building derivations as far as possible + in the face of failed builds. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> <option>--help</option> </term> <listitem> diff --git a/nixos/doc/manual/manual.xml b/nixos/doc/manual/manual.xml index 18a67a2dd9416..db9e7313831da 100644 --- a/nixos/doc/manual/manual.xml +++ b/nixos/doc/manual/manual.xml @@ -19,5 +19,6 @@ <xi:include href="./generated/options-db.xml" xpointer="configuration-variable-list" /> </appendix> + <xi:include href="contributing-to-this-manual.xml" /> <xi:include href="release-notes/release-notes.xml" /> </book> diff --git a/nixos/doc/manual/release-notes/rl-2009.xml b/nixos/doc/manual/release-notes/rl-2009.xml index 04086275d7c6a..afb09d7c5d262 100644 --- a/nixos/doc/manual/release-notes/rl-2009.xml +++ b/nixos/doc/manual/release-notes/rl-2009.xml @@ -3,8 +3,11 @@ xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-release-20.09"> - <title>Release 20.09 (“Nightingale”, 2020.09/??)</title> + <title>Release 20.09 (“Nightingale”, 2020.10/27)</title> + <para> + Support is planned until the end of April 2021, handing over to 21.03. + </para> <section xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" @@ -13,88 +16,611 @@ <title>Highlights</title> <para> - In addition to numerous new and upgraded packages, this release has the + In addition to 7349 new, 14442 updated, and 8181 removed packages, this release has the following highlights: </para> <itemizedlist> <listitem> <para> - Support is planned until the end of April 2021, handing over to 21.03. + Core version changes: </para> + <itemizedlist> + <listitem> + <para> + gcc: 9.2.0 -> 9.3.0 + </para> + </listitem> + <listitem> + <para> + glibc: 2.30 -> 2.31 + </para> + </listitem> + <listitem> + <para> + linux: still defaults to 5.4.x, all supported kernels available + </para> + </listitem> + <listitem> + <para> + mesa: 19.3.5 -> 20.1.7 + </para> + </listitem> + </itemizedlist> </listitem> <listitem> - <para>GNOME desktop environment was upgraded to 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link>.</para> - </listitem> - <listitem> - <para> - The Cinnamon desktop environment (v4.6) has been added. <varname>services.xserver.desktopManager.cinnamon.enable = true;</varname> to try it out! - Remember that, with any new feature it's possible you could run into issues, so please send all support requests to <link xlink:href="https://github.com/NixOS/nixpkgs/issues">github.com/NixOS/nixpkgs</link> to notify the maintainers. - </para> + <para> + Desktop Environments: + </para> + <itemizedlist> + <listitem> + <para> + plasma5: 5.17.5 -> 5.18.5 + </para> + </listitem> + <listitem> + <para> + kdeApplications: 19.12.3 -> 20.08.1 + </para> + </listitem> + <listitem> + <para> + gnome3: 3.34 -> 3.36, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.36/">release notes</link> + </para> + </listitem> + <listitem> + <para> + cinnamon: added at 4.6 + </para> + </listitem> + <listitem> + <para> + NixOS now distributes an official <link xlink:href="https://nixos.org/download.html#nixos-iso">GNOME ISO</link> + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - Quickly configure a complete, private, self-hosted video - conferencing solution with the new Jitsi Meet module. + Programming Languages and Frameworks: </para> + <itemizedlist> + + <listitem> + <para> + Agda ecosystem was heavily reworked (see more details below) + </para> + </listitem> + <listitem> + <para> + PHP now defaults to PHP 7.4, updated from 7.3 + </para> + </listitem> + <listitem> + <para> + PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release + </para> + </listitem> + <listitem> + <para> + Python 3 now defaults to Python 3.8 instead of 3.7 + </para> + </listitem> + <listitem> + <para> + Python 3.5 reached its upstream EOL at the end of September 2020: it + has been removed from the list of available packages + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module. - Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module. + Databases and Service Monitoring: </para> + <itemizedlist> + <listitem> + <para> + MariaDB has been updated to 10.4, MariaDB Galera to 26.4. Please read the related upgrade instructions under <link linkend="sec-release-20.09-incompatibilities">backwards incompatibilities</link> before upgrading. + </para> + </listitem> + <listitem> + <para> + Zabbix now defaults to 5.0, updated from 4.4. Please read related sections under <link linkend="sec-release-20.09-incompatibilities">backwards compatibilities</link> before upgrading. + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - We now distribute a GNOME ISO. + Major module changes: </para> + <itemizedlist> + <listitem> + <para> + Quickly configure a complete, private, self-hosted video + conferencing solution with the new Jitsi Meet module. + </para> + </listitem> + <listitem> + <para> + Two new options, <link linkend="opt-services.openssh.authorizedKeysCommand">authorizedKeysCommand</link> + and <link linkend="opt-services.openssh.authorizedKeysCommandUser">authorizedKeysCommandUser</link>, have + been added to the <literal>openssh</literal> module. If you have <literal>AuthorizedKeysCommand</literal> + in your <link linkend="opt-services.openssh.extraConfig">services.openssh.extraConfig</link> you should + make use of these new options instead. + </para> + </listitem> + <listitem> + <para> + There is a new module for Podman (<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line. + </para> + </listitem> + <listitem> + <para> + The new <varname>virtualisation.containers</varname> module manages configuration shared by the CRI-O and Podman modules. + </para> + </listitem> + <listitem> + <para> + Declarative Docker containers are renamed from <varname>docker-containers</varname> to <varname>virtualisation.oci-containers.containers</varname>. + This is to make it possible to use <literal>podman</literal> instead of <literal>docker</literal>. + </para> + </listitem> + <listitem> + <para> + The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link> + has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities + like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of + the NixOS configuration: since this can be expensive when a large number of packages are installed, the + feature is disabled by default. + </para> + </listitem> + <listitem> + <para> + <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities. + </para> + </listitem> + <listitem> + <para> + The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds. + </para> + </listitem> + <listitem> + <para> + Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the <link xlink:href="https://sourceforge.net/projects/g15daemon/">g15daemon project</link>. + </para> + </listitem> + <listitem> + <para> + The GRUB module gained support for basic password protection, which + allows to restrict non-default entries in the boot menu to one or more + users. The users and passwords are defined via the option + <option>boot.loader.grub.users</option>. + Note: Password support is only available in GRUB version 2. + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - PHP now defaults to PHP 7.4, updated from 7.3. + NixOS module changes: </para> + <itemizedlist> + <listitem> + <para> + The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them. + </para> + </listitem> + <listitem> + <para> + Following its deprecation in 20.03, the Perl NixOS test driver has been removed. + All remaining tests have been ported to the Python test framework. + Code outside nixpkgs using <filename>make-test.nix</filename> or + <filename>testing.nix</filename> needs to be ported to + <filename>make-test-python.nix</filename> and + <filename>testing-python.nix</filename> respectively. + </para> + </listitem> + <listitem> + <para> + Subordinate GID and UID mappings are now set up automatically for all normal users. + This will make container tools like Podman work as non-root users out of the box. + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - PHP 7.2 is no longer supported due to upstream not supporting this version for the entire lifecycle of the 20.09 release. + Starting with this release, the hydra-build-result + <literal>nixos-<replaceable>YY.MM</replaceable></literal> + branches no longer exist in the <link + xlink:href="https://github.com/nixos/nixpkgs-channels">deprecated + nixpkgs-channels repository</link>. These branches are now in + <link xlink:href="https://github.com/nixos/nixpkgs">the main nixpkgs + repository</link>. </para> </listitem> + </itemizedlist> + </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-new-services"> + <title>New Services</title> + + <para> + In addition to 1119 new, 118 updated, and 476 removed options; 61 new modules were added since the last release: + </para> + + <itemizedlist> <listitem> <para> - Python 3 now defaults to Python 3.8 instead of 3.7. + Hardware: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-hardware.system76.firmware-daemon.enable" /> adds easy support of system76 firmware + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.uinput.enable" /> loads uinput kernel module + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.video.hidpi.enable" /> enable good defaults for HiDPI displays + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.wooting.enable" /> support for Wooting keyboards + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-hardware.xpadneo.enable" /> xpadneo driver for Xbox One wireless controllers + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - Python 3.5 has reached its upstream EOL at the end of September 2020: it - has been removed from the list of available packages. + Programs: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-programs.hamster.enable" /> enable hamster time tracking + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-programs.steam.enable" /> adds easy enablement of steam and related system configuration + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - Two new options, <link linkend="opt-services.openssh.authorizedKeysCommand">authorizedKeysCommand</link> - and <link linkend="opt-services.openssh.authorizedKeysCommandUser">authorizedKeysCommandUser</link>, have - been added to the <literal>openssh</literal> module. If you have <literal>AuthorizedKeysCommand</literal> - in your <link linkend="opt-services.openssh.extraConfig">services.openssh.extraConfig</link> you should - make use of these new options instead. + Security: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-security.doas.enable" /> alternative to sudo, allows non-root users to execute commands as root + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-security.tpm2.enable" /> add Trusted Platform Module 2 support + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - There is a new module for Podman(<varname>virtualisation.podman</varname>), a drop-in replacement for the Docker command line. + System: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-boot.initrd.network.openvpn.enable" /> start an OpenVPN client during initrd boot + </para> + </listitem> + </itemizedlist> </listitem> <listitem> <para> - The new <varname>virtualisation.containers</varname> module manages configuration shared by the CRI-O and Podman modules. + Virtualization: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-boot.enableContainers" /> use nixos-containers + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-virtualisation.oci-containers.containers" /> run OCI (Docker) containers + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-virtualisation.podman.enable" /> daemonless container engine + </para> + </listitem> + </itemizedlist> </listitem> + <listitem> <para> - Declarative Docker containers are renamed from <varname>docker-containers</varname> to <varname>virtualisation.oci-containers.containers</varname>. - This is to make it possible to use <literal>podman</literal> instead of <literal>docker</literal>. + Services: </para> + <itemizedlist> + <listitem> + <para> + <xref linkend="opt-services.ankisyncd.enable" /> Anki sync server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.bazarr.enable" /> Subtitle manager for Sonarr and Radarr + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.biboumi.enable" /> Biboumi XMPP gateway to IRC + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.blockbook-frontend" /> Blockbook-frontend, a service for the Trezor wallet + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.cage.enable" /> Wayland cage service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.convos.enable" /> IRC daemon, which can be accessed throught the browser + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.engelsystem.enable" /> Tool for coordinating volunteers and shifts on large events + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.espanso.enable" /> text-expander written in rust + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.foldingathome.enable" /> Folding@home client + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.gerrit.enable" /> Web-based team code collaboration tool + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.go-neb.enable" /> Matrix bot + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.hardware.xow.enable" /> xow as a systemd service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.hercules-ci-agent.enable" /> Hercules CI build agent + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jicofo.enable" /> Jitsi Conference Focus, component of Jitsi Meet + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jirafeau.enable" /> A web file repository + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jitsi-meet.enable" /> Secure, simple and scalable video conferences + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jitsi-videobridge.enable" /> Jitsi Videobridge, a WebRTC compatible router + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.jupyterhub.enable" /> Jupyterhub development server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.k3s.enable" /> Lightweight Kubernetes distribution + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.magic-wormhole-mailbox-server.enable" /> Magic Wormhole Mailbox Server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.malcontent.enable" /> Parental Control support + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.matrix-appservice-discord.enable" /> Matrix and Discord bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mautrix-telegram.enable" /> Matrix-Telegram puppeting/relaybot bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mirakurun.enable" /> Japanese DTV Tuner Server Service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.molly-brown.enable" /> Molly-Brown Gemini server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.mullvad-vpn.enable" /> Mullvad VPN daemon + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.ncdns.enable" /> Namecoin to DNS bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.nextdns.enable" /> NextDNS to DoH Proxy service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.nix-store-gcs-proxy" /> Google storage bucket to be used as a nix store + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.onedrive.enable" /> OneDrive sync service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.pinnwand.enable" /> Pastebin-like service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.pixiecore.enable" /> Manage network booting of machines + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.privacyidea.enable" /> Privacy authentication server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.quorum.enable" /> Quorum blockchain daemon + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.robustirc-bridge.enable" /> RobustIRC bridge + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.rss-bridge.enable" /> Generate RSS and Atom feeds + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.rtorrent.enable" /> rTorrent service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.smartdns.enable" /> SmartDNS DNS server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.sogo.enable" /> SOGo groupware + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.teeworlds.enable" /> Teeworlds game server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.torque.mom.enable" /> torque computing node + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.torque.server.enable" /> torque server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.tuptime.enable" /> A total uptime service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.urserver.enable" /> X11 remote server + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.wasabibackend.enable" /> Wasabi backend service + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.yubikey-agent.enable" /> Yubikey agent + </para> + </listitem> + <listitem> + <para> + <xref linkend="opt-services.zigbee2mqtt.enable" /> Zigbee to MQTT bridge + </para> + </listitem> + </itemizedlist> </listitem> + + </itemizedlist> + + </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-incompatibilities"> + <title>Backward Incompatibilities</title> + + <para> + When upgrading from a previous release, please be aware of the following + incompatible changes: + </para> + + <itemizedlist> <listitem> <para> MariaDB has been updated to 10.4, MariaDB Galera to 26.4. @@ -144,36 +670,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; from the default of <literal>mysql</literal> to a different user please change <literal>'mysql'@'localhost'</literal> to the corresponding user instead. </para> </listitem> - <listitem> - <para> - The new option <link linkend="opt-documentation.man.generateCaches">documentation.man.generateCaches</link> - has been added to automatically generate the <literal>man-db</literal> caches, which are needed by utilities - like <command>whatis</command> and <command>apropos</command>. The caches are generated during the build of - the NixOS configuration: since this can be expensive when a large number of packages are installed, the - feature is disabled by default. - </para> - </listitem> - <listitem> - <para> - <varname>services.postfix.sslCACert</varname> was replaced by <varname>services.postfix.tlsTrustedAuthorities</varname> which now defaults to system certificate authorities. - </para> - </listitem> - <listitem> - <para> - Subordinate GID and UID mappings are now set up automatically for all normal users. - This will make container tools like Podman work as non-root users out of the box. - </para> - </listitem> - <listitem> - <para> - The various documented workarounds to use steam have been converted to a module. <varname>programs.steam.enable</varname> enables steam, controller support and the workarounds. - </para> - </listitem> - <listitem> - <para> - Support for built-in LCDs in various pieces of Logitech hardware (keyboards and USB speakers). <varname>hardware.logitech.lcd.enable</varname> enables support for all hardware supported by the g15daemon project. - </para> - </listitem> + <listitem> <para> Zabbix now defaults to 5.0, updated from 4.4. Please carefully read through @@ -208,72 +705,13 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; </programlisting> </para> </listitem> - <listitem> - <para> - The NixOS module system now supports freeform modules as a mix between <literal>types.attrsOf</literal> and <literal>types.submodule</literal>. These allow you to explicitly declare a subset of options while still permitting definitions without an associated option. See <xref linkend='sec-freeform-modules'/> for how to use them. - </para> - </listitem> - <listitem> - <para> - The GRUB module gained support for basic password protection, which - allows to restrict non-default entries in the boot menu to one or more - users. The users and passwords are defined via the option - <option>boot.loader.grub.users</option>. - Note: Password support is only avaiable in GRUB version 2. - </para> - </listitem> - <listitem> - <para> - Following its deprecation in 20.03, the Perl NixOS test driver has been removed. - All remaining tests have been ported to the Python test framework. - Code outside nixpkgs using <filename>make-test.nix</filename> or - <filename>testing.nix</filename> needs to be ported to - <filename>make-test-python.nix</filename> and - <filename>testing-python.nix</filename> respectively. - </para> - </listitem> - </itemizedlist> - </section> - - <section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-release-20.09-new-services"> - <title>New Services</title> - - <para> - The following new services were added since the last release: - </para> - <itemizedlist> <listitem> <para> - There is a new <xref linkend="opt-security.doas.enable"/> module that provides <command>doas</command>, a lighter alternative to <command>sudo</command> with many of the same features. - </para> - </listitem> - <listitem> - <para> - <link xlink:href="https://hercules-ci.com">Hercules CI</link> Agent is a specialized build agent for projects built with Nix. See the <link xlink:href="https://nixos.org/nixos/options.html#services.hercules-ci-agent">options</link> and <link xlink:href="https://docs.hercules-ci.com/hercules-ci/getting-started/#deploy-agent">setup</link>. + <package>maxx</package> package removed along with <varname>services.xserver.desktopManager.maxx</varname> module. + Please migrate to <package>cdesktopenv</package> and <varname>services.xserver.desktopManager.cde</varname> module. </para> </listitem> - </itemizedlist> - - </section> - - <section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xmlns:xi="http://www.w3.org/2001/XInclude" - version="5.0" - xml:id="sec-release-20.09-incompatibilities"> - <title>Backward Incompatibilities</title> - - <para> - When upgrading from a previous release, please be aware of the following - incompatible changes: - </para> - - <itemizedlist> <listitem> <para> The <link linkend="opt-services.matrix-synapse.enable">matrix-synapse</link> module no longer includes optional dependencies by default, they have to be added through the <link linkend="opt-services.matrix-synapse.plugins">plugins</link> option. @@ -300,7 +738,7 @@ GRANT ALL PRIVILEGES ON *.* TO 'mysql'@'localhost' WITH GRANT OPTION; It can still be enabled by providing <literal>phantomJsSupport = true</literal> to the package instantiation: <programlisting>{ services.grafana.package = pkgs.grafana.overrideAttrs (oldAttrs: rec { - phantomJsSupport = false; + phantomJsSupport = true; }); }</programlisting> </para> @@ -451,12 +889,23 @@ php.override { <listitem> <para> Nginx web server now starting with additional sandbox/hardening options. By default, write access - to <literal>services.nginx.stateDir</literal> is allowed. To allow writing to other folders, + to <literal>/var/log/nginx</literal> and <literal>/var/cache/nginx</literal> is allowed. To allow writing to other folders, use <literal>systemd.services.nginx.serviceConfig.ReadWritePaths</literal> <programlisting> systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ]; </programlisting> </para> + <para> + Nginx is also started with the systemd option <literal>ProtectHome = mkDefault true;</literal> + which forbids it to read anything from <literal>/home</literal>, <literal>/root</literal> + and <literal>/run/user</literal> (see + <link xlink:href="https://www.freedesktop.org/software/systemd/man/systemd.exec.html#ProtectHome=">ProtectHome docs</link> + for details). + If you require serving files from home directories, you may choose to set e.g. +<programlisting> +systemd.services.nginx.serviceConfig.ProtectHome = "read-only"; +</programlisting> + </para> </listitem> <listitem> <para> @@ -643,6 +1092,13 @@ systemd.services.nginx.serviceConfig.ReadWritePaths = [ "/var/www" ]; In the <literal>resilio</literal> module, <xref linkend="opt-services.resilio.httpListenAddr"/> has been changed to listen to <literal>[::1]</literal> instead of <literal>0.0.0.0</literal>. </para> </listitem> + <listitem> + <para> + <literal>sslh</literal> has been updated to version + <literal>1.21</literal>. The <literal>ssl</literal> probe must be + renamed to <literal>tls</literal> in <xref linkend="opt-services.sslh.appendConfig"/>. + </para> + </listitem> <listitem> <para> Users of <link xlink:href="http://openafs.org">OpenAFS 1.6</link> must @@ -1147,30 +1603,30 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; <para> Agda has been heavily reworked. <itemizedlist> - <listitem> - <para> - <literal>agda.mkDerivation</literal> has been heavily changed and - is now located at <package>agdaPackages.mkDerivation</package>. - </para> - </listitem> - <listitem> - <para> - New top-level packages <package>agda</package> and - <literal>agda.withPackages</literal> have been added, the second - of which sets up agda with access to chosen libraries. - </para> - </listitem> - <listitem> - <para> - All agda libraries now live under - <literal>agdaPackages</literal>. - </para> - </listitem> - <listitem> - <para> - Many broken libraries have been removed. - </para> - </listitem> + <listitem> + <para> + <literal>agda.mkDerivation</literal> has been heavily changed and + is now located at <package>agdaPackages.mkDerivation</package>. + </para> + </listitem> + <listitem> + <para> + New top-level packages <package>agda</package> and + <literal>agda.withPackages</literal> have been added, the second + of which sets up agda with access to chosen libraries. + </para> + </listitem> + <listitem> + <para> + All agda libraries now live under + <literal>agdaPackages</literal>. + </para> + </listitem> + <listitem> + <para> + Many broken libraries have been removed. + </para> + </listitem> </itemizedlist> See the <link xlink:href="https://nixos.org/nixpkgs/manual/#agda">new @@ -1183,7 +1639,7 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; nixpkgs. It was a work in progress to package the <link xlink:href="https://www.deepin.org/en/dde/">Deepin Desktop Environment (DDE)</link>, including libraries, tools and applications, and it was still - missing a service to lauch the desktop environment. It has shown + missing a service to launch the desktop environment. It has shown to no longer be a feasible goal due to reasons discussed in <link xlink:href="https://github.com/NixOS/nixpkgs/issues/94870">issue #94870</link>. The package <literal>netease-cloud-music</literal> has also been @@ -1228,4 +1684,131 @@ services.transmission.settings.rpc-bind-address = "0.0.0.0"; </listitem> </itemizedlist> </section> + + <section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="sec-release-20.09-contributions"> + <title>Contributions</title> + <para> + I, Jonathan Ringer, would like to thank the following individuals for their work on nixpkgs. This release could not be done without the hard work of the NixOS community. There were 31282 contributions across 1313 contributors. + </para> + <orderedlist> + <para> + Top contributors to NixOS/Nixpkgs from the 20.03 release to the 20.09 release: + </para> + <listitem> + <para> + 2288 Mario Rodas + </para> + </listitem> + <listitem> + <para> + 1837 Frederik Rietdijk + </para> + </listitem> + <listitem> + <para> + 946 Jörg Thalheim + </para> + </listitem> + <listitem> + <para> + 925 Maximilian Bosch + </para> + </listitem> + <listitem> + <para> + 687 Jonathan Ringer + </para> + </listitem> + <listitem> + <para> + 651 Jan Tojnar + </para> + </listitem> + <listitem> + <para> + 622 Daniël de Kok + </para> + </listitem> + <listitem> + <para> + 605 WORLDofPEACE + </para> + </listitem> + <listitem> + <para> + 597 Florian Klink + </para> + </listitem> + <listitem> + <para> + 528 José Romildo Malaquias + </para> + </listitem> + </orderedlist> + + <orderedlist> + <para> + Top contributors to stabilizing this release (Zero Hydra Failures period): + </para> + <listitem> + <para> + 281 volth + </para> + </listitem> + <listitem> + <para> + 101 Robert Scott + </para> + </listitem> + <listitem> + <para> + 86 Tim Steinbach + </para> + </listitem> + <listitem> + <para> + 76 WORLDofPEACE + </para> + </listitem> + <listitem> + <para> + 49 Maximilian Bosch + </para> + </listitem> + <listitem> + <para> + 42 Thomas Tuegel + </para> + </listitem> + <listitem> + <para> + 37 Doron Behar + </para> + </listitem> + <listitem> + <para> + 36 Vladimír Čunát + </para> + </listitem> + <listitem> + <para> + 27 Jonathan Ringer + </para> + </listitem> + <listitem> + <para> + 27 Maciej Krüger + </para> + </listitem> + </orderedlist> + + <para> + I, Jonathan Ringer, would also like to personally thank @WORLDofPEACE for their help in mentoring me on the release process. Special thanks also goes to Thomas Tuegel for helping immensely with stabilizing Qt, KDE, and Plasma5; I would also like to thank Robert Scott for his numerous fixes and pull request reviews. + </para> + + </section> </section> diff --git a/nixos/doc/manual/release-notes/rl-2103.xml b/nixos/doc/manual/release-notes/rl-2103.xml index c160ab5783d3b..5c017c65a2534 100644 --- a/nixos/doc/manual/release-notes/rl-2103.xml +++ b/nixos/doc/manual/release-notes/rl-2103.xml @@ -23,6 +23,9 @@ Support is planned until the end of October 2021, handing over to 21.09. </para> </listitem> + <listitem> + <para>GNOME desktop environment was upgraded to 3.38, see its <link xlink:href="https://help.gnome.org/misc/release-notes/3.38/">release notes</link>.</para> + </listitem> </itemizedlist> </section> @@ -39,7 +42,19 @@ <itemizedlist> <listitem> - <para /> + <para> + <link xlink:href="https://www.keycloak.org/">Keycloak</link>, + an open source identity and access management server with + support for <link + xlink:href="https://openid.net/connect/">OpenID Connect</link>, + <link xlink:href="https://oauth.net/2/">OAUTH 2.0</link> and + <link xlink:href="https://en.wikipedia.org/wiki/SAML_2.0">SAML + 2.0</link>. + </para> + <para> + See the <link linkend="module-services-keycloak">Keycloak + section of the NixOS manual</link> for more information. + </para> </listitem> </itemizedlist> @@ -75,6 +90,18 @@ </para> </listitem> <listitem> + <para> + <literal>rubyMinimal</literal> was removed due to being unused and + unusable. The default ruby interpreter includes JIT support, which makes + it reference it's compiler. Since JIT support is probably needed by some + Gems, it was decided to enable this feature with all cc references by + default, and allow to build a Ruby derivation without references to cc, + by setting <literal>jitSupport = false;</literal> in an overlay. See + <link xlink:href="https://github.com/NixOS/nixpkgs/pull/90151">#90151</link> + for more info. + </para> + </listitem> + <listitem> <para> The option <option>fonts.enableFontDir</option> has been renamed to <xref linkend="opt-fonts.fontDir.enable"/>. The path of font directory @@ -99,6 +126,52 @@ to <literal>/run/pdns-recursor</literal> to match upstream. </para> </listitem> + <listitem> + <para> + PowerDNS has been updated from <literal>4.2.x</literal> to <literal>4.3.x</literal>. Please + be sure to review the <link xlink:href="https://doc.powerdns.com/authoritative/upgrading.html#x-to-4-3-0">Upgrade Notes</link> + provided by upstream before upgrading. Worth specifically noting is that the service now runs + entirely as a dedicated <literal>pdns</literal> user, instead of starting as <literal>root</literal> + and dropping privileges, as well as the default <literal>socket-dir</literal> location changing from + <literal>/var/lib/powerdns</literal> to <literal>/run/pdns</literal>. + </para> + </listitem> + <listitem> + <para> + <package>btc1</package> has been abandoned upstream, and removed. + </para> + </listitem> + <listitem> + <para> + <package>riak-cs</package> package removed along with <varname>services.riak-cs</varname> module. + </para> + </listitem> + <listitem> + <para> + <package>stanchion</package> package removed along with <varname>services.stanchion</varname> module. + </para> + </listitem> + <listitem> + <para> + <package>mutt</package> has been updated to a new major version (2.x), which comes with + some backward incompatible changes that are described in the + <link xlink:href="http://www.mutt.org/relnotes/2.0/">release notes for Mutt 2.0</link>. + </para> + </listitem> + <listitem> + <para> + <literal>vim</literal> switched to Python 3, dropping all Python 2 support. + </para> + </listitem> + <listitem> + <para> + <link linkend="opt-boot.zfs.forceImportAll">boot.zfs.forceImportAll</link> + previously did nothing, but has been fixed. However its default has been + changed to <literal>false</literal> to preserve the existing default + behaviour. If you have this explicitly set to <literal>true</literal>, + please note that your non-root pools will now be forcibly imported. + </para> + </listitem> </itemizedlist> </section> @@ -122,6 +195,75 @@ to <package>nextcloud20</package>. </para> </listitem> + <listitem> + <para> + The setting <xref linkend="opt-services.redis.bind" /> defaults to <literal>127.0.0.1</literal> now, making Redis listen on the loopback interface only, and not all public network interfaces. + </para> + </listitem> + <listitem> + <para> + NixOS now emits a deprecation warning if systemd's <literal>StartLimitInterval</literal> setting is used in a <literal>serviceConfig</literal> section instead of in a <literal>unitConfig</literal>; that setting is deprecated and now undocumented for the service section by systemd upstream, but still effective and somewhat buggy there, which can be confusing. See <link xlink:href="https://github.com/NixOS/nixpkgs/issues/45785">#45785</link> for details. + </para> + <para> + All services should use <xref linkend="opt-systemd.services._name_.startLimitIntervalSec" /> or <literal>StartLimitIntervalSec</literal> in <xref linkend="opt-systemd.services._name_.unitConfig" /> instead. + </para> + </listitem> + <listitem> + <para> + The Unbound DNS resolver service (<literal>services.unbound</literal>) has been refactored to allow reloading, control sockets and to fix startup ordering issues. + </para> + + <para> + It is now possible to enable a local UNIX control socket for unbound by setting the <xref linkend="opt-services.unbound.localControlSocketPath" /> + option. + </para> + + <para> + Previously we just applied a very minimal set of restrictions and + trusted unbound to properly drop root privs and capabilities. + </para> + + <para> + As of this we are (for the most part) just using the upstream + example unit file for unbound. The main difference is that we start + unbound as <literal>unbound</literal> user with the required capabilities instead of + letting unbound do the chroot & uid/gid changes. + </para> + + <para> + The upstream unit configuration this is based on is a lot stricter with + all kinds of permissions then our previous variant. It also came with + the default of having the <literal>Type</literal> set to <literal>notify</literal>, therefore we are now also + using the <literal>unbound-with-systemd</literal> package here. Unbound will start up, + read the configuration files and start listening on the configured ports + before systemd will declare the unit <literal>active (running)</literal>. + This will likely help with startup order and the occasional race condition during system + activation where the DNS service is started but not yet ready to answer + queries. Services depending on <literal>nss-lookup.target</literal> or <literal>unbound.service</literal> + are now be able to use unbound when those targets have been reached. + </para> + + <para> + Aditionally to the much stricter runtime environmet the + <literal>/dev/urandom</literal> mount lines we previously had in the code (that would + randomly failed during the stop-phase) have been removed as systemd will take care of those for us. + </para> + + <para> + The <literal>preStart</literal> script is now only required if we enabled the trust + anchor updates (which are still enabled by default). + </para> + + <para> + Another benefit of the refactoring is that we can now issue reloads via + either <literal>pkill -HUP unbound</literal> and <literal>systemctl reload unbound</literal> to reload the + running configuration without taking the daemon offline. A prerequisite + of this was that unbound configuration is available on a well known path + on the file system. We are using the path <literal>/etc/unbound/unbound.conf</literal> as that is the + default in the CLI tooling which in turn enables us to use + <literal>unbound-control</literal> without passing a custom configuration location. + </para> + </listitem> </itemizedlist> </section> </section> diff --git a/nixos/doc/varlistentry-fixer.rb b/nixos/doc/varlistentry-fixer.rb index 6c7cc1e6439b1..02168016b554b 100755 --- a/nixos/doc/varlistentry-fixer.rb +++ b/nixos/doc/varlistentry-fixer.rb @@ -15,8 +15,8 @@ require "rexml/document" include REXML if ARGV.length < 1 then - $stderr.puts "Needs a filename." - exit 1 + $stderr.puts "Needs a filename." + exit 1 end filename = ARGV.shift @@ -51,17 +51,17 @@ $touched = false # Generates: --optionnamevalue # ^^ ^^ doc.elements.each("//varlistentry/term") do |term| - ["varname", "function", "option", "replaceable"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["varname", "function", "option", "replaceable"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end @@ -75,17 +75,17 @@ end # Generates: -Ipath # ^^ doc.elements.each("//cmdsynopsis/arg") do |term| - ["option", "replaceable"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["option", "replaceable"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end # <cmdsynopsis> @@ -104,21 +104,21 @@ end # Generates: [{--profile-name | -p }name] # ^^^^ doc.elements.each("//cmdsynopsis/arg") do |term| - ["group"].each do |prev_name| - term.elements.each(prev_name) do |el| - if el.next_element and - el.next_element.name == "replaceable" and - el.next_sibling_node.class == Element - then - $touched = true - term.insert_after(el, Text.new(" ")) - end - end - end + ["group"].each do |prev_name| + term.elements.each(prev_name) do |el| + if el.next_element and + el.next_element.name == "replaceable" and + el.next_sibling_node.class == Element + then + $touched = true + term.insert_after(el, Text.new(" ")) + end + end + end end if $touched then - doc.context[:attribute_quote] = :quote - doc.write(output: File.open(filename, "w")) + doc.context[:attribute_quote] = :quote + doc.write(output: File.open(filename, "w")) end diff --git a/nixos/lib/build-vms.nix b/nixos/lib/build-vms.nix index b1575fc13bbdf..ebbb0296bef63 100644 --- a/nixos/lib/build-vms.nix +++ b/nixos/lib/build-vms.nix @@ -18,9 +18,6 @@ rec { inherit pkgs; - qemu = pkgs.qemu_test; - - # Build a virtual network from an attribute set `{ machine1 = # config1; ... machineN = configN; }', where `machineX' is the # hostname and `configX' is a NixOS system configuration. Each @@ -39,7 +36,6 @@ rec { [ ../modules/virtualisation/qemu-vm.nix ../modules/testing/test-instrumentation.nix # !!! should only get added for automated test runs { key = "no-manual"; documentation.nixos.enable = false; } - { key = "qemu"; system.build.qemu = qemu; } { key = "nodes"; _module.args.nodes = nodes; } ] ++ optional minimal ../modules/testing/minimal-kernel.nix; }; diff --git a/nixos/lib/make-disk-image.nix b/nixos/lib/make-disk-image.nix index 8aa606a56af80..a4a488a1b3ea9 100644 --- a/nixos/lib/make-disk-image.nix +++ b/nixos/lib/make-disk-image.nix @@ -28,6 +28,9 @@ # partition of reasonable size is created in addition to the root partition. # For "legacy", the msdos partition table is used and a single large root # partition is created. + # For "legacy+gpt", the GPT partition table is used, a 1MiB no-fs partition for + # use by the bootloader is created, and a single large root partition is + # created. # For "hybrid", the GPT partition table is used and a mandatory ESP # partition of reasonable size is created in addition to the root partition. # Also a legacy MBR will be present. @@ -54,7 +57,7 @@ format ? "raw" }: -assert partitionTableType == "legacy" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none"; +assert partitionTableType == "legacy" || partitionTableType == "legacy+gpt" || partitionTableType == "efi" || partitionTableType == "hybrid" || partitionTableType == "none"; # We use -E offset=X below, which is only supported by e2fsprogs assert partitionTableType != "none" -> fsType == "ext4"; @@ -75,6 +78,7 @@ let format' = format; in let rootPartition = { # switch-case legacy = "1"; + "legacy+gpt" = "2"; efi = "2"; hybrid = "3"; }.${partitionTableType}; @@ -85,6 +89,16 @@ let format' = format; in let mklabel msdos \ mkpart primary ext4 1MiB -1 ''; + "legacy+gpt" = '' + parted --script $diskImage -- \ + mklabel gpt \ + mkpart no-fs 1MB 2MB \ + set 1 bios_grub on \ + align-check optimal 1 \ + mkpart primary ext4 2MB -1 \ + align-check optimal 2 \ + print + ''; efi = '' parted --script $diskImage -- \ mklabel gpt \ diff --git a/nixos/lib/make-iso9660-image.nix b/nixos/lib/make-iso9660-image.nix index 6a0e0e7c635a4..549530965f6e1 100644 --- a/nixos/lib/make-iso9660-image.nix +++ b/nixos/lib/make-iso9660-image.nix @@ -48,7 +48,7 @@ assert usbBootable -> isohybridMbrImage != ""; stdenv.mkDerivation { name = isoName; builder = ./make-iso9660-image.sh; - buildInputs = [ xorriso syslinux zstd libossp_uuid ]; + nativeBuildInputs = [ xorriso syslinux zstd libossp_uuid ]; inherit isoName bootable bootImage compressImage volumeID efiBootImage efiBootable isohybridMbrImage usbBootable; diff --git a/nixos/lib/make-system-tarball.nix b/nixos/lib/make-system-tarball.nix index dee91a6ce3f49..dab168f4a4813 100644 --- a/nixos/lib/make-system-tarball.nix +++ b/nixos/lib/make-system-tarball.nix @@ -37,7 +37,7 @@ in stdenv.mkDerivation { name = "tarball"; builder = ./make-system-tarball.sh; - buildInputs = extraInputs; + nativeBuildInputs = extraInputs; inherit fileName extraArgs extraCommands compressCommand; diff --git a/nixos/lib/test-driver/test-driver.py b/nixos/lib/test-driver/test-driver.py index 156392ad1e320..68bd35dd98e93 100644 --- a/nixos/lib/test-driver/test-driver.py +++ b/nixos/lib/test-driver/test-driver.py @@ -110,7 +110,6 @@ def create_vlan(vlan_nr: str) -> Tuple[str, str, "subprocess.Popen[bytes]", Any] pty_master, pty_slave = pty.openpty() vde_process = subprocess.Popen( ["vde_switch", "-s", vde_socket, "--dirmode", "0700"], - bufsize=1, stdin=pty_slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE, @@ -748,7 +747,6 @@ class Machine: self.process = subprocess.Popen( self.script, - bufsize=1, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, diff --git a/nixos/lib/testing-python.nix b/nixos/lib/testing-python.nix index 302c7f78bf8e7..13abfb9a111d4 100644 --- a/nixos/lib/testing-python.nix +++ b/nixos/lib/testing-python.nix @@ -3,13 +3,13 @@ # Use a minimal kernel? , minimal ? false # Ignored -, config ? {} +, config ? { } # !!! See comment about args in lib/modules.nix -, specialArgs ? {} +, specialArgs ? { } # Modules to add to each VM -, extraConfigurations ? [] }: +, extraConfigurations ? [ ] +}: -with import ./build-vms.nix { inherit system pkgs minimal specialArgs extraConfigurations; }; with pkgs; rec { @@ -17,42 +17,41 @@ rec { inherit pkgs; - mkTestDriver = let - testDriverScript = ./test-driver/test-driver.py; - in qemu_pkg: stdenv.mkDerivation { - name = "nixos-test-driver"; - - nativeBuildInputs = [ makeWrapper ]; - buildInputs = [ (python3.withPackages (p: [ p.ptpython ])) ]; - checkInputs = with python3Packages; [ pylint black mypy ]; - - dontUnpack = true; + mkTestDriver = + let + testDriverScript = ./test-driver/test-driver.py; + in + qemu_pkg: stdenv.mkDerivation { + name = "nixos-test-driver"; - preferLocalBuild = true; + nativeBuildInputs = [ makeWrapper ]; + buildInputs = [ (python3.withPackages (p: [ p.ptpython ])) ]; + checkInputs = with python3Packages; [ pylint black mypy ]; - doCheck = true; - checkPhase = '' - mypy --disallow-untyped-defs \ - --no-implicit-optional \ - --ignore-missing-imports ${testDriverScript} - pylint --errors-only ${testDriverScript} - black --check --diff ${testDriverScript} - ''; + dontUnpack = true; - installPhase = - '' - mkdir -p $out/bin - cp ${testDriverScript} $out/bin/nixos-test-driver - chmod u+x $out/bin/nixos-test-driver - # TODO: copy user script part into this file (append) + preferLocalBuild = true; - wrapProgram $out/bin/nixos-test-driver \ - --prefix PATH : "${lib.makeBinPath [ qemu_pkg vde2 netpbm coreutils ]}" \ + doCheck = true; + checkPhase = '' + mypy --disallow-untyped-defs \ + --no-implicit-optional \ + --ignore-missing-imports ${testDriverScript} + pylint --errors-only ${testDriverScript} + black --check --diff ${testDriverScript} ''; - }; - testDriver = mkTestDriver qemu_test; - testDriverInteractive = mkTestDriver qemu_kvm; + installPhase = + '' + mkdir -p $out/bin + cp ${testDriverScript} $out/bin/nixos-test-driver + chmod u+x $out/bin/nixos-test-driver + # TODO: copy user script part into this file (append) + + wrapProgram $out/bin/nixos-test-driver \ + --prefix PATH : "${lib.makeBinPath [ qemu_pkg vde2 netpbm coreutils ]}" \ + ''; + }; # Run an automated test suite in the given virtual network. # `driver' is the script that runs the network. @@ -75,11 +74,10 @@ rec { { testScript , enableOCR ? false , name ? "unnamed" - # Skip linting (mainly intended for faster dev cycles) + # Skip linting (mainly intended for faster dev cycles) , skipLint ? false , ... } @ t: - let # A standard store path to the vm monitor is built like this: # /tmp/nix-build-vm-test-run-$name.drv-0/vm-state-machine/monitor @@ -88,25 +86,7 @@ rec { maxTestNameLen = 50; testNameLen = builtins.stringLength name; - testDriverName = with builtins; - if testNameLen > maxTestNameLen then - abort ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " + - "it's currently ${toString testNameLen} characters long.") - else - "nixos-test-driver-${name}"; - - nodes = buildVirtualNetwork ( - t.nodes or (if t ? machine then { machine = t.machine; } else { })); - testScript' = - # Call the test script with the computed nodes. - if lib.isFunction testScript - then testScript { inherit nodes; } - else testScript; - - vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes); - - vms = map (m: m.config.system.build.vm) (lib.attrValues nodes); ocrProg = tesseract4.override { enableLanguages = [ "eng" ]; }; @@ -115,78 +95,124 @@ rec { # Generate convenience wrappers for running the test driver # interactively with the specified network, and for starting the # VMs from the command line. - driver = testDriver: + mkDriver = qemu_pkg: let + build-vms = import ./build-vms.nix { + inherit system pkgs minimal specialArgs; + extraConfigurations = extraConfigurations ++ (pkgs.lib.optional (qemu_pkg != null) + { + virtualisation.qemu.package = qemu_pkg; + } + ); + }; + + # FIXME: get this pkg from the module system + testDriver = mkTestDriver (if qemu_pkg == null then pkgs.qemu_test else qemu_pkg); + + nodes = build-vms.buildVirtualNetwork ( + t.nodes or (if t ? machine then { machine = t.machine; } else { }) + ); + vlans = map (m: m.config.virtualisation.vlans) (lib.attrValues nodes); + vms = map (m: m.config.system.build.vm) (lib.attrValues nodes); + + testScript' = + # Call the test script with the computed nodes. + if lib.isFunction testScript + then testScript { inherit nodes; } + else testScript; + + testDriverName = with builtins; + if testNameLen > maxTestNameLen then + abort + ("The name of the test '${name}' must not be longer than ${toString maxTestNameLen} " + + "it's currently ${toString testNameLen} characters long.") + else + "nixos-test-driver-${name}"; + warn = if skipLint then lib.warn "Linting is disabled!" else lib.id; in warn (runCommand testDriverName - { buildInputs = [ makeWrapper]; - testScript = testScript'; - preferLocalBuild = true; - testName = name; - } - '' - mkdir -p $out/bin - - echo -n "$testScript" > $out/test-script - ${lib.optionalString (!skipLint) '' - ${python3Packages.black}/bin/black --check --diff $out/test-script - ''} - - ln -s ${testDriver}/bin/nixos-test-driver $out/bin/ - vms=($(for i in ${toString vms}; do echo $i/bin/run-*-vm; done)) - wrapProgram $out/bin/nixos-test-driver \ - --add-flags "''${vms[*]}" \ - ${lib.optionalString enableOCR - "--prefix PATH : '${ocrProg}/bin:${imagemagick_tiff}/bin'"} \ - --run "export testScript=\"\$(${coreutils}/bin/cat $out/test-script)\"" \ - --set VLANS '${toString vlans}' - ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-run-vms - wrapProgram $out/bin/nixos-run-vms \ - --add-flags "''${vms[*]}" \ - ${lib.optionalString enableOCR "--prefix PATH : '${ocrProg}/bin'"} \ - --set tests 'start_all(); join_all();' \ - --set VLANS '${toString vlans}' \ - ${lib.optionalString (builtins.length vms == 1) "--set USE_SERIAL 1"} - ''); # " + { + buildInputs = [ makeWrapper ]; + testScript = testScript'; + preferLocalBuild = true; + testName = name; + passthru = { + inherit nodes; + }; + } + '' + mkdir -p $out/bin + + echo -n "$testScript" > $out/test-script + ${lib.optionalString (!skipLint) '' + ${python3Packages.black}/bin/black --check --diff $out/test-script + ''} + + ln -s ${testDriver}/bin/nixos-test-driver $out/bin/ + vms=($(for i in ${toString vms}; do echo $i/bin/run-*-vm; done)) + wrapProgram $out/bin/nixos-test-driver \ + --add-flags "''${vms[*]}" \ + ${lib.optionalString enableOCR + "--prefix PATH : '${ocrProg}/bin:${imagemagick_tiff}/bin'"} \ + --run "export testScript=\"\$(${coreutils}/bin/cat $out/test-script)\"" \ + --set VLANS '${toString vlans}' + ln -s ${testDriver}/bin/nixos-test-driver $out/bin/nixos-run-vms + wrapProgram $out/bin/nixos-run-vms \ + --add-flags "''${vms[*]}" \ + ${lib.optionalString enableOCR "--prefix PATH : '${ocrProg}/bin'"} \ + --set tests 'start_all(); join_all();' \ + --set VLANS '${toString vlans}' \ + ${lib.optionalString (builtins.length vms == 1) "--set USE_SERIAL 1"} + ''); # " passMeta = drv: drv // lib.optionalAttrs (t ? meta) { - meta = (drv.meta or {}) // t.meta; + meta = (drv.meta or { }) // t.meta; }; - test = passMeta (runTests (driver testDriver)); + driver = mkDriver null; + driverInteractive = mkDriver pkgs.qemu; - nodeNames = builtins.attrNames nodes; + test = passMeta (runTests driver); + + nodeNames = builtins.attrNames driver.nodes; invalidNodeNames = lib.filter - (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null) nodeNames; + (node: builtins.match "^[A-z_]([A-z0-9_]+)?$" node == null) + nodeNames; in - if lib.length invalidNodeNames > 0 then - throw '' - Cannot create machines out of (${lib.concatStringsSep ", " invalidNodeNames})! - All machines are referenced as python variables in the testing framework which will break the - script when special characters are used. + if lib.length invalidNodeNames > 0 then + throw '' + Cannot create machines out of (${lib.concatStringsSep ", " invalidNodeNames})! + All machines are referenced as python variables in the testing framework which will break the + script when special characters are used. - Please stick to alphanumeric chars and underscores as separation. - '' - else - test // { - inherit nodes test; - driver = driver testDriver; - driverInteractive = driver testDriverInteractive; - }; + Please stick to alphanumeric chars and underscores as separation. + '' + else + test // { + inherit test driver driverInteractive; + inherit (driver) nodes; + }; runInMachine = { drv , machine , preBuild ? "" , postBuild ? "" + , qemu ? pkgs.qemu_test , ... # ??? }: let - vm = buildVM { } - [ machine - { key = "run-in-machine"; + build-vms = import ./build-vms.nix { + inherit system pkgs minimal specialArgs extraConfigurations; + }; + + vm = build-vms.buildVM { } + [ + machine + { + key = "run-in-machine"; networking.hostName = "client"; nix.readOnlyStore = false; virtualisation.writableStore = false; @@ -229,20 +255,20 @@ rec { unset xchg export tests='${testScript}' - ${testDriver}/bin/nixos-test-driver ${vm.config.system.build.vm}/bin/run-*-vm + ${mkTestDriver qemu}/bin/nixos-test-driver --keep-vm-state ${vm.config.system.build.vm}/bin/run-*-vm ''; # */ in - lib.overrideDerivation drv (attrs: { - requiredSystemFeatures = [ "kvm" ]; - builder = "${bash}/bin/sh"; - args = ["-e" vmRunCommand]; - origArgs = attrs.args; - origBuilder = attrs.builder; - }); + lib.overrideDerivation drv (attrs: { + requiredSystemFeatures = [ "kvm" ]; + builder = "${bash}/bin/sh"; + args = [ "-e" vmRunCommand ]; + origArgs = attrs.args; + origBuilder = attrs.builder; + }); - runInMachineWithX = { require ? [], ... } @ args: + runInMachineWithX = { require ? [ ], ... } @ args: let client = { ... }: @@ -258,13 +284,13 @@ rec { services.xserver.windowManager.icewm.enable = true; }; in - runInMachine ({ - machine = client; - preBuild = - '' - client.wait_for_x() - ''; - } // args); + runInMachine ({ + machine = client; + preBuild = + '' + client.wait_for_x() + ''; + } // args); simpleTest = as: (makeTest as).test; diff --git a/nixos/maintainers/scripts/ec2/amazon-image.nix b/nixos/maintainers/scripts/ec2/amazon-image.nix index b09f4ca47a3fc..0ecf07669a119 100644 --- a/nixos/maintainers/scripts/ec2/amazon-image.nix +++ b/nixos/maintainers/scripts/ec2/amazon-image.nix @@ -57,7 +57,7 @@ in { inherit (cfg) contents format name; pkgs = import ../../../.. { inherit (pkgs) system; }; # ensure we use the regular qemu-kvm package partitionTableType = if config.ec2.efi then "efi" - else if config.ec2.hvm then "legacy" + else if config.ec2.hvm then "legacy+gpt" else "none"; diskSize = cfg.sizeMB; fsType = "ext4"; diff --git a/nixos/maintainers/scripts/ec2/create-amis.sh b/nixos/maintainers/scripts/ec2/create-amis.sh index 89e24f2ccfd95..ec2eb53667901 100755 --- a/nixos/maintainers/scripts/ec2/create-amis.sh +++ b/nixos/maintainers/scripts/ec2/create-amis.sh @@ -1,13 +1,15 @@ #!/usr/bin/env nix-shell #!nix-shell -p awscli -p jq -p qemu -i bash +# shellcheck shell=bash # Uploads and registers NixOS images built from the # <nixos/release.nix> amazonImage attribute. Images are uploaded and # registered via a home region, and then copied to other regions. -# The home region requires an s3 bucket, and a "vmimport" IAM role -# with access to the S3 bucket. Configuration of the vmimport role is -# documented in +# The home region requires an s3 bucket, and an IAM role named "vmimport" +# (by default) with access to the S3 bucket. The name can be +# configured with the "service_role_name" variable. Configuration of the +# vmimport role is documented in # https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html # set -x @@ -17,6 +19,7 @@ set -euo pipefail state_dir=$HOME/amis/ec2-images home_region=eu-west-1 bucket=nixos-amis +service_role_name=vmimport regions=(eu-west-1 eu-west-2 eu-west-3 eu-central-1 eu-north-1 us-east-1 us-east-2 us-west-1 us-west-2 @@ -64,7 +67,7 @@ image_logical_bytes=$(read_image_info .logical_bytes) # Derived attributes -image_logical_gigabytes=$((($image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB +image_logical_gigabytes=$(((image_logical_bytes-1)/1024/1024/1024+1)) # Round to the next GB case "$image_system" in aarch64-linux) @@ -100,7 +103,7 @@ write_state() { local type=$2 local val=$3 - mkdir -p $state_dir + mkdir -p "$state_dir" echo "$val" > "$state_dir/$state_key.$type" } @@ -110,8 +113,8 @@ wait_for_import() { local state snapshot_id log "Waiting for import task $task_id to be completed" while true; do - read state progress snapshot_id < <( - aws ec2 describe-import-snapshot-tasks --region $region --import-task-ids "$task_id" | \ + read -r state progress snapshot_id < <( + aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" | \ jq -r '.ImportSnapshotTasks[].SnapshotTaskDetail | "\(.Status) \(.Progress) \(.SnapshotId)"' ) log " ... state=$state progress=$progress snapshot_id=$snapshot_id" @@ -125,6 +128,8 @@ wait_for_import() { ;; *) log "Unexpected snapshot import state: '${state}'" + log "Full response: " + aws ec2 describe-import-snapshot-tasks --region "$region" --import-task-ids "$task_id" >&2 exit 1 ;; esac @@ -138,8 +143,8 @@ wait_for_image() { log "Waiting for image $ami_id to be available" while true; do - read state < <( - aws ec2 describe-images --image-ids "$ami_id" --region $region | \ + read -r state < <( + aws ec2 describe-images --image-ids "$ami_id" --region "$region" | \ jq -r ".Images[].State" ) log " ... state=$state" @@ -163,7 +168,7 @@ make_image_public() { local region=$1 local ami_id=$2 - wait_for_image $region "$ami_id" + wait_for_image "$region" "$ami_id" log "Making image $ami_id public" @@ -177,27 +182,30 @@ upload_image() { local aws_path=${image_file#/} local state_key="$region.$image_label.$image_system" - local task_id=$(read_state "$state_key" task_id) - local snapshot_id=$(read_state "$state_key" snapshot_id) - local ami_id=$(read_state "$state_key" ami_id) + local task_id + task_id=$(read_state "$state_key" task_id) + local snapshot_id + snapshot_id=$(read_state "$state_key" snapshot_id) + local ami_id + ami_id=$(read_state "$state_key" ami_id) if [ -z "$task_id" ]; then log "Checking for image on S3" if ! aws s3 ls --region "$region" "s3://${bucket}/${aws_path}" >&2; then log "Image missing from aws, uploading" - aws s3 cp --region $region "$image_file" "s3://${bucket}/${aws_path}" >&2 + aws s3 cp --region "$region" "$image_file" "s3://${bucket}/${aws_path}" >&2 fi log "Importing image from S3 path s3://$bucket/$aws_path" - task_id=$(aws ec2 import-snapshot --disk-container "{ + task_id=$(aws ec2 import-snapshot --role-name "$service_role_name" --disk-container "{ \"Description\": \"nixos-image-${image_label}-${image_system}\", \"Format\": \"vhd\", \"UserBucket\": { \"S3Bucket\": \"$bucket\", \"S3Key\": \"$aws_path\" } - }" --region $region | jq -r '.ImportTaskId') + }" --region "$region" | jq -r '.ImportTaskId') write_state "$state_key" task_id "$task_id" fi @@ -221,16 +229,16 @@ upload_image() { --virtualization-type hvm ) - block_device_mappings+=(DeviceName=/dev/sdb,VirtualName=ephemeral0) - block_device_mappings+=(DeviceName=/dev/sdc,VirtualName=ephemeral1) - block_device_mappings+=(DeviceName=/dev/sdd,VirtualName=ephemeral2) - block_device_mappings+=(DeviceName=/dev/sde,VirtualName=ephemeral3) + block_device_mappings+=("DeviceName=/dev/sdb,VirtualName=ephemeral0") + block_device_mappings+=("DeviceName=/dev/sdc,VirtualName=ephemeral1") + block_device_mappings+=("DeviceName=/dev/sdd,VirtualName=ephemeral2") + block_device_mappings+=("DeviceName=/dev/sde,VirtualName=ephemeral3") ami_id=$( aws ec2 register-image \ --name "$image_name" \ --description "$image_description" \ - --region $region \ + --region "$region" \ --architecture $amazon_arch \ --block-device-mappings "${block_device_mappings[@]}" \ "${extra_flags[@]}" \ @@ -240,7 +248,7 @@ upload_image() { write_state "$state_key" ami_id "$ami_id" fi - make_image_public $region "$ami_id" + make_image_public "$region" "$ami_id" echo "$ami_id" } @@ -268,7 +276,7 @@ copy_to_region() { write_state "$state_key" ami_id "$ami_id" fi - make_image_public $region "$ami_id" + make_image_public "$region" "$ami_id" echo "$ami_id" } diff --git a/nixos/modules/config/fonts/fontdir.nix b/nixos/modules/config/fonts/fontdir.nix index 264d73ebafa53..c4bd3a077d339 100644 --- a/nixos/modules/config/fonts/fontdir.nix +++ b/nixos/modules/config/fonts/fontdir.nix @@ -8,7 +8,7 @@ let x11Fonts = pkgs.runCommand "X11-fonts" { preferLocalBuild = true; } '' mkdir -p "$out/share/X11/fonts" - font_regexp='.*\.\(ttf\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?' + font_regexp='.*\.\(ttf\|ttc\|otf\|pcf\|pfa\|pfb\|bdf\)\(\.gz\)?' find ${toString config.fonts.fonts} -regex "$font_regexp" \ -exec ln -sf -t "$out/share/X11/fonts" '{}' \; cd "$out/share/X11/fonts" diff --git a/nixos/modules/config/pulseaudio.nix b/nixos/modules/config/pulseaudio.nix index 408d0a9c33f27..a77524d75d8dc 100644 --- a/nixos/modules/config/pulseaudio.nix +++ b/nixos/modules/config/pulseaudio.nix @@ -36,6 +36,8 @@ let ${addModuleIf cfg.zeroconf.discovery.enable "module-zeroconf-discover"} ${addModuleIf cfg.tcp.enable (concatStringsSep " " ([ "module-native-protocol-tcp" ] ++ allAnon ++ ipAnon))} + ${addModuleIf config.services.jack.jackd.enable "module-jack-sink"} + ${addModuleIf config.services.jack.jackd.enable "module-jack-source"} ${cfg.extraConfig} ''; }; @@ -144,7 +146,9 @@ in { package = mkOption { type = types.package; - default = pkgs.pulseaudio; + default = if config.services.jack.jackd.enable + then pkgs.pulseaudioFull + else pkgs.pulseaudio; defaultText = "pkgs.pulseaudio"; example = literalExample "pkgs.pulseaudioFull"; description = '' @@ -259,7 +263,7 @@ in { (drv: drv.override { pulseaudio = overriddenPackage; }) cfg.extraModules; modulePaths = builtins.map - (drv: "${drv}/lib/pulse-${overriddenPackage.version}/modules") + (drv: "${drv}/${overriddenPackage.pulseDir}/modules") # User-provided extra modules take precedence (overriddenModules ++ [ overriddenPackage ]); in lib.concatStringsSep ":" modulePaths; @@ -284,6 +288,8 @@ in { RestartSec = "500ms"; PassEnvironment = "DISPLAY"; }; + } // optionalAttrs config.services.jack.jackd.enable { + environment.JACK_PROMISCUOUS_SERVER = "jackaudio"; }; sockets.pulseaudio = { wantedBy = [ "sockets.target" ]; diff --git a/nixos/modules/config/system-path.nix b/nixos/modules/config/system-path.nix index c46937f800815..c65fa1a684f8e 100644 --- a/nixos/modules/config/system-path.nix +++ b/nixos/modules/config/system-path.nix @@ -33,6 +33,7 @@ let pkgs.ncurses pkgs.netcat config.programs.ssh.package + pkgs.mkpasswd pkgs.procps pkgs.su pkgs.time diff --git a/nixos/modules/config/users-groups.nix b/nixos/modules/config/users-groups.nix index 5264d5b56fa50..72285fe631dac 100644 --- a/nixos/modules/config/users-groups.nix +++ b/nixos/modules/config/users-groups.nix @@ -35,8 +35,7 @@ let ''; hashedPasswordDescription = '' - To generate a hashed password install the <literal>mkpasswd</literal> - package and run <literal>mkpasswd -m sha-512</literal>. + To generate a hashed password run <literal>mkpasswd -m sha-512</literal>. If set to an empty string (<literal>""</literal>), this user will be able to log in without being asked for a password (but not via remote diff --git a/nixos/modules/hardware/acpilight.nix b/nixos/modules/hardware/acpilight.nix index 34e8a22209653..2de448a265c79 100644 --- a/nixos/modules/hardware/acpilight.nix +++ b/nixos/modules/hardware/acpilight.nix @@ -19,6 +19,7 @@ in }; config = mkIf cfg.enable { + environment.systemPackages = with pkgs; [ acpilight ]; services.udev.packages = with pkgs; [ acpilight ]; }; } diff --git a/nixos/modules/hardware/rtl-sdr.nix b/nixos/modules/hardware/rtl-sdr.nix new file mode 100644 index 0000000000000..77c8cb59a3d58 --- /dev/null +++ b/nixos/modules/hardware/rtl-sdr.nix @@ -0,0 +1,20 @@ +{ config, lib, pkgs, ... }: + +let + cfg = config.hardware.rtl-sdr; + +in { + options.hardware.rtl-sdr = { + enable = lib.mkEnableOption '' + Enables rtl-sdr udev rules and ensures 'plugdev' group exists. + This is a prerequisite to using devices supported by rtl-sdr without + being root, since rtl-sdr USB descriptors will be owned by plugdev + through udev. + ''; + }; + + config = lib.mkIf cfg.enable { + services.udev.packages = [ pkgs.rtl-sdr ]; + users.groups.plugdev = {}; + }; +} diff --git a/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix b/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix index 8c98691116dc9..803bae4212ef7 100644 --- a/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix +++ b/nixos/modules/installer/cd-dvd/installation-cd-graphical-gnome.nix @@ -9,7 +9,14 @@ with lib; isoImage.edition = "gnome"; - services.xserver.desktopManager.gnome3.enable = true; + services.xserver.desktopManager.gnome3 = { + # Add firefox to favorite-apps + favoriteAppsOverride = '' + [org.gnome.shell] + favorite-apps=[ 'firefox.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ] + ''; + enable = true; + }; services.xserver.displayManager = { gdm = { diff --git a/nixos/modules/installer/tools/nix-fallback-paths.nix b/nixos/modules/installer/tools/nix-fallback-paths.nix index a15a2dbadb8c6..bd70bd20013b4 100644 --- a/nixos/modules/installer/tools/nix-fallback-paths.nix +++ b/nixos/modules/installer/tools/nix-fallback-paths.nix @@ -1,6 +1,6 @@ { - x86_64-linux = "/nix/store/4vz8sh9ngx34ivi0bw5hlycxdhvy5hvz-nix-2.3.7"; - i686-linux = "/nix/store/dzxkg9lpp60bjmzvagns42vqlz3yq5kx-nix-2.3.7"; - aarch64-linux = "/nix/store/cfvf8nl8mwyw817by5y8zd3s8pnf5m9f-nix-2.3.7"; - x86_64-darwin = "/nix/store/5ira7xgs92inqz1x8l0n1wci4r79hnd0-nix-2.3.7"; + x86_64-linux = "/nix/store/qxayqjmlpqnmwg5yfsjjayw220ls8i2r-nix-2.3.8"; + i686-linux = "/nix/store/5834psaay75048jp6d07liqh4j0v1swd-nix-2.3.8"; + aarch64-linux = "/nix/store/pic90a5fxvifz05jzkd0zak21f9mjin6-nix-2.3.8"; + x86_64-darwin = "/nix/store/cjx3f8z12wlayp5983kli2a52ipi8jz2-nix-2.3.8"; } diff --git a/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix b/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix index 0c9f8522cc12b..e49ceba242459 100644 --- a/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix +++ b/nixos/modules/installer/tools/nixos-build-vms/build-vms.nix @@ -15,4 +15,4 @@ with import ../../../../lib/testing-python.nix { pkgs = import ../../../../.. { inherit system config; }; }; -(makeTest { inherit nodes; testScript = ""; }).driver +(makeTest { inherit nodes; testScript = ""; }).driverInteractive diff --git a/nixos/modules/installer/tools/nixos-generate-config.pl b/nixos/modules/installer/tools/nixos-generate-config.pl index 54b84521bdffd..6e3ddb875e1b4 100644 --- a/nixos/modules/installer/tools/nixos-generate-config.pl +++ b/nixos/modules/installer/tools/nixos-generate-config.pl @@ -183,6 +183,11 @@ sub pciCheck { push @imports, "(modulesPath + \"/hardware/network/broadcom-43xx.nix\")"; } + # In case this is a virtio scsi device, we need to explicitly make this available. + if ($vendor eq "0x1af4" && $device eq "0x1004") { + push @initrdAvailableKernelModules, "virtio_scsi"; + } + # Can't rely on $module here, since the module may not be loaded # due to missing firmware. Ideally we would check modules.pcimap # here. @@ -625,6 +630,10 @@ EOF my $networkingDhcpConfig = generateNetworkingDhcpConfig(); + (my $desktopConfiguration = <<EOF)=~s/^/ /gm; +@desktopConfiguration@ +EOF + write_file($fn, <<EOF); @configuration@ EOF diff --git a/nixos/modules/installer/tools/nixos-install.sh b/nixos/modules/installer/tools/nixos-install.sh index 90555f1458036..9d49d4055e431 100644 --- a/nixos/modules/installer/tools/nixos-install.sh +++ b/nixos/modules/installer/tools/nixos-install.sh @@ -64,7 +64,7 @@ while [ "$#" -gt 0 ]; do --no-bootloader) noBootLoader=1 ;; - --show-trace) + --show-trace|--impure|--keep-going) extraBuildFlags+=("$i") ;; --help) diff --git a/nixos/modules/installer/tools/tools.nix b/nixos/modules/installer/tools/tools.nix index 666a1eff0e62c..0582812f92d24 100644 --- a/nixos/modules/installer/tools/tools.nix +++ b/nixos/modules/installer/tools/tools.nix @@ -45,7 +45,7 @@ let src = ./nixos-generate-config.pl; path = lib.optionals (lib.elem "btrfs" config.boot.supportedFilesystems) [ pkgs.btrfs-progs ]; perl = "${pkgs.perl}/bin/perl -I${pkgs.perlPackages.FileSlurp}/${pkgs.perl.libPrefix}"; - inherit (config.system.nixos-generate-config) configuration; + inherit (config.system.nixos-generate-config) configuration desktopConfiguration; }; nixos-option = @@ -78,24 +78,42 @@ in { - options.system.nixos-generate-config.configuration = mkOption { - internal = true; - type = types.str; - description = '' - The NixOS module that <literal>nixos-generate-config</literal> - saves to <literal>/etc/nixos/configuration.nix</literal>. - - This is an internal option. No backward compatibility is guaranteed. - Use at your own risk! + options.system.nixos-generate-config = { + configuration = mkOption { + internal = true; + type = types.str; + description = '' + The NixOS module that <literal>nixos-generate-config</literal> + saves to <literal>/etc/nixos/configuration.nix</literal>. + + This is an internal option. No backward compatibility is guaranteed. + Use at your own risk! + + Note that this string gets spliced into a Perl script. The perl + variable <literal>$bootLoaderConfig</literal> can be used to + splice in the boot loader configuration. + ''; + }; - Note that this string gets spliced into a Perl script. The perl - variable <literal>$bootLoaderConfig</literal> can be used to - splice in the boot loader configuration. - ''; + desktopConfiguration = mkOption { + internal = true; + type = types.str; + default = ""; + description = '' + Text to preseed the desktop configuration that <literal>nixos-generate-config</literal> + saves to <literal>/etc/nixos/configuration.nix</literal>. + + This is an internal option. No backward compatibility is guaranteed. + Use at your own risk! + + Note that this string gets spliced into a Perl script. The perl + variable <literal>$bootLoaderConfig</literal> can be used to + splice in the boot loader configuration. + ''; + }; }; config = { - system.nixos-generate-config.configuration = mkDefault '' # Edit this configuration file to define what should be installed on # your system. Help is available in the configuration.nix(5) man page @@ -113,6 +131,9 @@ in # networking.hostName = "nixos"; # Define your hostname. # networking.wireless.enable = true; # Enables wireless support via wpa_supplicant. + # Set your time zone. + # time.timeZone = "Europe/Amsterdam"; + $networkingDhcpConfig # Configure network proxy if necessary # networking.proxy.default = "http://user:password\@proxy:port/"; @@ -125,13 +146,32 @@ in # keyMap = "us"; # }; - # Set your time zone. - # time.timeZone = "Europe/Amsterdam"; + $desktopConfiguration + # Configure keymap in X11 + # services.xserver.layout = "us"; + # services.xserver.xkbOptions = "eurosign:e"; + + # Enable CUPS to print documents. + # services.printing.enable = true; + + # Enable sound. + # sound.enable = true; + # hardware.pulseaudio.enable = true; + + # Enable touchpad support (enabled default in most desktopManager). + # services.xserver.libinput.enable = true; + + # Define a user account. Don't forget to set a password with ‘passwd’. + # users.users.jane = { + # isNormalUser = true; + # extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user. + # }; # List packages installed in system profile. To search, run: # \$ nix search wget # environment.systemPackages = with pkgs; [ # wget vim + # firefox # ]; # Some programs need SUID wrappers, can be configured further or are @@ -140,7 +180,6 @@ in # programs.gnupg.agent = { # enable = true; # enableSSHSupport = true; - # pinentryFlavor = "gnome3"; # }; # List services that you want to enable: @@ -154,31 +193,6 @@ in # Or disable the firewall altogether. # networking.firewall.enable = false; - # Enable CUPS to print documents. - # services.printing.enable = true; - - # Enable sound. - # sound.enable = true; - # hardware.pulseaudio.enable = true; - - # Enable the X11 windowing system. - # services.xserver.enable = true; - # services.xserver.layout = "us"; - # services.xserver.xkbOptions = "eurosign:e"; - - # Enable touchpad support. - # services.xserver.libinput.enable = true; - - # Enable the KDE Desktop Environment. - # services.xserver.displayManager.sddm.enable = true; - # services.xserver.desktopManager.plasma5.enable = true; - - # Define a user account. Don't forget to set a password with ‘passwd’. - # users.users.jane = { - # isNormalUser = true; - # extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user. - # }; - # This value determines the NixOS release from which the default # settings for stateful data, like file locations and database versions # on your system were taken. It‘s perfectly fine and recommended to leave diff --git a/nixos/modules/misc/documentation.nix b/nixos/modules/misc/documentation.nix index 71a40b4f4d6e2..bc43cc33b5d43 100644 --- a/nixos/modules/misc/documentation.nix +++ b/nixos/modules/misc/documentation.nix @@ -40,9 +40,9 @@ let in scrubbedEval.options; }; - helpScript = pkgs.writeScriptBin "nixos-help" - '' - #! ${pkgs.runtimeShell} -e + + nixos-help = let + helpScript = pkgs.writeShellScriptBin "nixos-help" '' # Finds first executable browser in a colon-separated list. # (see how xdg-open defines BROWSER) browser="$( @@ -59,14 +59,22 @@ let exec "$browser" ${manual.manualHTMLIndex} ''; - desktopItem = pkgs.makeDesktopItem { - name = "nixos-manual"; - desktopName = "NixOS Manual"; - genericName = "View NixOS documentation in a web browser"; - icon = "nix-snowflake"; - exec = "${helpScript}/bin/nixos-help"; - categories = "System"; - }; + desktopItem = pkgs.makeDesktopItem { + name = "nixos-manual"; + desktopName = "NixOS Manual"; + genericName = "View NixOS documentation in a web browser"; + icon = "nix-snowflake"; + exec = "nixos-help"; + categories = "System"; + }; + + in pkgs.symlinkJoin { + name = "nixos-help"; + paths = [ + helpScript + desktopItem + ]; + }; in @@ -250,8 +258,8 @@ in environment.systemPackages = [] ++ optional cfg.man.enable manual.manpages - ++ optionals cfg.doc.enable ([ manual.manualHTML helpScript ] - ++ optionals config.services.xserver.enable [ desktopItem pkgs.nixos-icons ]); + ++ optionals cfg.doc.enable ([ manual.manualHTML nixos-help ] + ++ optionals config.services.xserver.enable [ pkgs.nixos-icons ]); services.mingetty.helpLine = mkIf cfg.doc.enable ( "\nRun 'nixos-help' for the NixOS manual." diff --git a/nixos/modules/misc/ids.nix b/nixos/modules/misc/ids.nix index cd21a1609c915..bafa222504009 100644 --- a/nixos/modules/misc/ids.nix +++ b/nixos/modules/misc/ids.nix @@ -290,8 +290,8 @@ in hound = 259; leaps = 260; ipfs = 261; - stanchion = 262; - riak-cs = 263; + # stanchion = 262; # unused, removed 2020-10-14 + # riak-cs = 263; # unused, removed 2020-10-14 infinoted = 264; sickbeard = 265; headphones = 266; @@ -468,7 +468,7 @@ in #minecraft = 114; # unused vault = 115; #ripped = 116; # unused - #murmur = 117; # unused + murmur = 117; foundationdb = 118; newrelic = 119; starbound = 120; @@ -593,8 +593,8 @@ in hound = 259; leaps = 260; ipfs = 261; - stanchion = 262; - riak-cs = 263; + # stanchion = 262; # unused, removed 2020-10-14 + # riak-cs = 263; # unused, removed 2020-10-14 infinoted = 264; sickbeard = 265; headphones = 266; diff --git a/nixos/modules/module-list.nix b/nixos/modules/module-list.nix index ed6201237b3dd..0a38fa317305d 100644 --- a/nixos/modules/module-list.nix +++ b/nixos/modules/module-list.nix @@ -59,6 +59,7 @@ ./hardware/pcmcia.nix ./hardware/printers.nix ./hardware/raid/hpsa.nix + ./hardware/rtl-sdr.nix ./hardware/steam-hardware.nix ./hardware/system-76.nix ./hardware/tuxedo-keyboard.nix @@ -172,6 +173,7 @@ ./programs/wavemon.nix ./programs/waybar.nix ./programs/wireshark.nix + ./programs/wshowkeys.nix ./programs/x2goserver.nix ./programs/xfs_quota.nix ./programs/xonsh.nix @@ -294,8 +296,6 @@ ./services/databases/postgresql.nix ./services/databases/redis.nix ./services/databases/riak.nix - ./services/databases/riak-cs.nix - ./services/databases/stanchion.nix ./services/databases/victoriametrics.nix ./services/databases/virtuoso.nix ./services/desktops/accountsservice.nix @@ -392,6 +392,7 @@ ./services/logging/logcheck.nix ./services/logging/logrotate.nix ./services/logging/logstash.nix + ./services/logging/promtail.nix ./services/logging/rsyslogd.nix ./services/logging/syslog-ng.nix ./services/logging/syslogd.nix @@ -401,7 +402,6 @@ ./services/mail/dovecot.nix ./services/mail/dspam.nix ./services/mail/exim.nix - ./services/mail/freepops.nix ./services/mail/mail.nix ./services/mail/mailcatcher.nix ./services/mail/mailhog.nix @@ -542,6 +542,7 @@ ./services/monitoring/kapacitor.nix ./services/monitoring/loki.nix ./services/monitoring/longview.nix + ./services/monitoring/mackerel-agent.nix ./services/monitoring/monit.nix ./services/monitoring/munin.nix ./services/monitoring/nagios.nix @@ -680,6 +681,7 @@ ./services/networking/murmur.nix ./services/networking/mxisd.nix ./services/networking/namecoind.nix + ./services/networking/nar-serve.nix ./services/networking/nat.nix ./services/networking/ndppd.nix ./services/networking/networkmanager.nix @@ -863,6 +865,7 @@ ./services/web-apps/ihatemoney ./services/web-apps/jirafeau.nix ./services/web-apps/jitsi-meet.nix + ./services/web-apps/keycloak.nix ./services/web-apps/limesurvey.nix ./services/web-apps/mattermost.nix ./services/web-apps/mediawiki.nix diff --git a/nixos/modules/programs/bandwhich.nix b/nixos/modules/programs/bandwhich.nix index 5413044f46141..1cffb5fa2765c 100644 --- a/nixos/modules/programs/bandwhich.nix +++ b/nixos/modules/programs/bandwhich.nix @@ -4,7 +4,7 @@ with lib; let cfg = config.programs.bandwhich; in { - meta.maintainers = with maintainers; [ filalex77 ]; + meta.maintainers = with maintainers; [ Br1ght0ne ]; options = { programs.bandwhich = { diff --git a/nixos/modules/programs/ssmtp.nix b/nixos/modules/programs/ssmtp.nix index 15d2750c193ff..1f49ddc91bb38 100644 --- a/nixos/modules/programs/ssmtp.nix +++ b/nixos/modules/programs/ssmtp.nix @@ -1,6 +1,6 @@ # Configuration for `ssmtp', a trivial mail transfer agent that can # replace sendmail/postfix on simple systems. It delivers email -# directly to an SMTP server defined in its configuration file, wihout +# directly to an SMTP server defined in its configuration file, without # queueing mail locally. { config, lib, pkgs, ... }: @@ -142,6 +142,13 @@ in config = mkIf cfg.enable { + assertions = [ + { + assertion = cfg.useSTARTTLS -> cfg.useTLS; + message = "services.ssmtp.useSTARTTLS has no effect without services.ssmtp.useTLS"; + } + ]; + services.ssmtp.settings = mkMerge [ ({ MailHub = cfg.hostName; diff --git a/nixos/modules/programs/wshowkeys.nix b/nixos/modules/programs/wshowkeys.nix new file mode 100644 index 0000000000000..09b008af1d5db --- /dev/null +++ b/nixos/modules/programs/wshowkeys.nix @@ -0,0 +1,22 @@ +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.programs.wshowkeys; +in { + meta.maintainers = with maintainers; [ primeos ]; + + options = { + programs.wshowkeys = { + enable = mkEnableOption '' + wshowkeys (displays keypresses on screen on supported Wayland + compositors). It requires root permissions to read input events, but + these permissions are dropped after startup''; + }; + }; + + config = mkIf cfg.enable { + security.wrappers.wshowkeys.source = "${pkgs.wshowkeys}/bin/wshowkeys"; + }; +} diff --git a/nixos/modules/security/acme.nix b/nixos/modules/security/acme.nix index 5732620f29083..47f6bead7c3ed 100644 --- a/nixos/modules/security/acme.nix +++ b/nixos/modules/security/acme.nix @@ -63,7 +63,7 @@ let script = with builtins; concatStringsSep "\n" (mapAttrsToList (cert: data: '' for fixpath in /var/lib/acme/${escapeShellArg cert} /var/lib/acme/.lego/${escapeShellArg cert}; do if [ -d "$fixpath" ]; then - chmod -R 750 "$fixpath" + chmod -R u=rwX,g=rX,o= "$fixpath" chown -R acme:${data.group} "$fixpath" fi done @@ -271,7 +271,7 @@ let mv domainhash.txt certificates/ chmod 640 certificates/* - chmod -R 700 accounts/* + chmod -R u=rwX,g=,o= accounts/* # Group might change between runs, re-apply it chown 'acme:${data.group}' certificates/* diff --git a/nixos/modules/security/pam.nix b/nixos/modules/security/pam.nix index a20d0a243a8ef..a517f9e51ce17 100644 --- a/nixos/modules/security/pam.nix +++ b/nixos/modules/security/pam.nix @@ -318,6 +318,42 @@ let ''; }; + gnupg = { + enable = mkOption { + type = types.bool; + default = false; + description = '' + If enabled, pam_gnupg will attempt to automatically unlock the + user's GPG keys with the login password via + <command>gpg-agent</command>. The keygrips of all keys to be + unlocked should be written to <filename>~/.pam-gnupg</filename>, + and can be queried with <command>gpg -K --with-keygrip</command>. + Presetting passphrases must be enabled by adding + <literal>allow-preset-passphrase</literal> in + <filename>~/.gnupg/gpg-agent.conf</filename>. + ''; + }; + + noAutostart = mkOption { + type = types.bool; + default = false; + description = '' + Don't start <command>gpg-agent</command> if it is not running. + Useful in conjunction with starting <command>gpg-agent</command> as + a systemd user service. + ''; + }; + + storeOnly = mkOption { + type = types.bool; + default = false; + description = '' + Don't send the password immediately after login, but store for PAM + <literal>session</literal>. + ''; + }; + }; + text = mkOption { type = types.nullOr types.lines; description = "Contents of the PAM service file."; @@ -386,6 +422,7 @@ let || cfg.enableKwallet || cfg.enableGnomeKeyring || cfg.googleAuthenticator.enable + || cfg.gnupg.enable || cfg.duoSecurity.enable)) '' auth required pam_unix.so ${optionalString cfg.allowNullPassword "nullok"} ${optionalString cfg.nodelay "nodelay"} likeauth ${optionalString config.security.pam.enableEcryptfs @@ -397,6 +434,10 @@ let " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")} ${optionalString cfg.enableGnomeKeyring "auth optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so"} + ${optionalString cfg.gnupg.enable + "auth optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so" + + optionalString cfg.gnupg.storeOnly " store-only" + } ${optionalString cfg.googleAuthenticator.enable "auth required ${pkgs.googleAuthenticator}/lib/security/pam_google_authenticator.so no_increment_hotp"} ${optionalString cfg.duoSecurity.enable @@ -472,6 +513,10 @@ let " kwalletd=${pkgs.kdeFrameworks.kwallet.bin}/bin/kwalletd5")} ${optionalString (cfg.enableGnomeKeyring) "session optional ${pkgs.gnome3.gnome-keyring}/lib/security/pam_gnome_keyring.so auto_start"} + ${optionalString cfg.gnupg.enable + "session optional ${pkgs.pam_gnupg}/lib/security/pam_gnupg.so" + + optionalString cfg.gnupg.noAutostart " no-autostart" + } ${optionalString (config.virtualisation.lxc.lxcfs.enable) "session optional ${pkgs.lxc}/lib/security/pam_cgfs.so -c all"} ''); diff --git a/nixos/modules/services/admin/salt/master.nix b/nixos/modules/services/admin/salt/master.nix index c6b1b0cc0bd8e..cb803d323bbd2 100644 --- a/nixos/modules/services/admin/salt/master.nix +++ b/nixos/modules/services/admin/salt/master.nix @@ -59,5 +59,5 @@ in }; }; - meta.maintainers = with lib.maintainers; [ aneeshusa ]; + meta.maintainers = with lib.maintainers; [ Flakebi ]; } diff --git a/nixos/modules/services/audio/jack.nix b/nixos/modules/services/audio/jack.nix index ceff366d0bbb2..bee97dbfc6b3d 100644 --- a/nixos/modules/services/audio/jack.nix +++ b/nixos/modules/services/audio/jack.nix @@ -246,6 +246,9 @@ in { description = "JACK Audio Connection Kit"; serviceConfig = { User = "jackaudio"; + SupplementaryGroups = lib.optional + (config.hardware.pulseaudio.enable + && !config.hardware.pulseaudio.systemWide) "users"; ExecStart = "${cfg.jackd.package}/bin/jackd ${lib.escapeShellArgs cfg.jackd.extraOptions}"; LimitRTPRIO = 99; LimitMEMLOCK = "infinity"; diff --git a/nixos/modules/services/backup/syncoid.nix b/nixos/modules/services/backup/syncoid.nix index fff119c2cf009..e72e3fa59cf92 100644 --- a/nixos/modules/services/backup/syncoid.nix +++ b/nixos/modules/services/backup/syncoid.nix @@ -4,6 +4,15 @@ with lib; let cfg = config.services.syncoid; + + # Extract pool names of local datasets (ones that don't contain "@") that + # have the specified type (either "source" or "target") + getPools = type: unique (map (d: head (builtins.match "([^/]+).*" d)) ( + # Filter local datasets + filter (d: !hasInfix "@" d) + # Get datasets of the specified type + (catAttrs type (attrValues cfg.commands)) + )); in { # Interface @@ -26,14 +35,25 @@ in { user = mkOption { type = types.str; - default = "root"; + default = "syncoid"; example = "backup"; description = '' - The user for the service. Sudo or ZFS privilege delegation must be - configured to use a user other than root. + The user for the service. ZFS privilege delegation will be + automatically configured for any local pools used by syncoid if this + option is set to a user other than root. The user will be given the + "hold" and "send" privileges on any pool that has datasets being sent + and the "create", "mount", "receive", and "rollback" privileges on + any pool that has datasets being received. ''; }; + group = mkOption { + type = types.str; + default = "syncoid"; + example = "backup"; + description = "The group for the service."; + }; + sshKey = mkOption { type = types.nullOr types.path; # Prevent key from being copied to store @@ -150,6 +170,18 @@ in { # Implementation config = mkIf cfg.enable { + users = { + users = mkIf (cfg.user == "syncoid") { + syncoid = { + group = cfg.group; + isSystemUser = true; + }; + }; + groups = mkIf (cfg.group == "syncoid") { + syncoid = {}; + }; + }; + systemd.services.syncoid = { description = "Syncoid ZFS synchronization service"; script = concatMapStringsSep "\n" (c: lib.escapeShellArgs @@ -160,10 +192,22 @@ in { ++ c.extraArgs ++ [ "--sendoptions" c.sendOptions "--recvoptions" c.recvOptions + "--no-privilege-elevation" c.source c.target ])) (attrValues cfg.commands); after = [ "zfs.target" ]; - serviceConfig.User = cfg.user; + serviceConfig = { + ExecStartPre = (map (pool: lib.escapeShellArgs [ + "+/run/booted-system/sw/bin/zfs" "allow" + cfg.user "hold,send" pool + ]) (getPools "source")) ++ + (map (pool: lib.escapeShellArgs [ + "+/run/booted-system/sw/bin/zfs" "allow" + cfg.user "create,mount,receive,rollback" pool + ]) (getPools "target")); + User = cfg.user; + Group = cfg.group; + }; startAt = cfg.interval; }; }; diff --git a/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix b/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix index d2e7e8e18f94e..79d1ce5805457 100644 --- a/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix +++ b/nixos/modules/services/continuous-integration/hercules-ci-agent/default.nix @@ -25,19 +25,18 @@ in ]; config = mkIf cfg.enable { - systemd.services.hercules-ci-agent = { wantedBy = [ "multi-user.target" ]; after = [ "network-online.target" ]; wants = [ "network-online.target" ]; path = [ config.nix.package ]; + startLimitBurst = 30 * 1000000; # practically infinite serviceConfig = { User = "hercules-ci-agent"; ExecStart = command; ExecStartPre = testCommand; Restart = "on-failure"; RestartSec = 120; - StartLimitBurst = 30 * 1000000; # practically infinite }; }; diff --git a/nixos/modules/services/continuous-integration/hydra/default.nix b/nixos/modules/services/continuous-integration/hydra/default.nix index 502a5898a5de5..252ca17006da8 100644 --- a/nixos/modules/services/continuous-integration/hydra/default.nix +++ b/nixos/modules/services/continuous-integration/hydra/default.nix @@ -37,8 +37,6 @@ let haveLocalDB = cfg.dbi == localDB; - inherit (config.system) stateVersion; - hydra-package = let makeWrapperArgs = concatStringsSep " " (mapAttrsToList (key: value: "--set \"${key}\" \"${value}\"") hydraEnv); @@ -96,7 +94,8 @@ in package = mkOption { type = types.package; - defaultText = "pkgs.hydra"; + default = pkgs.hydra-unstable; + defaultText = "pkgs.hydra-unstable"; description = "The Hydra package."; }; @@ -225,34 +224,6 @@ in config = mkIf cfg.enable { - warnings = optional (cfg.package.migration or false) '' - You're currently deploying an older version of Hydra which is needed to - make some required database changes[1]. As soon as this is done, it's recommended - to run `hydra-backfill-ids` and set `services.hydra.package` to `pkgs.hydra-unstable` - after that. - - [1] https://github.com/NixOS/hydra/pull/711 - ''; - - services.hydra.package = with pkgs; - mkDefault ( - if pkgs ? hydra - then throw '' - The Hydra package doesn't exist anymore in `nixpkgs`! It probably exists - due to an overlay. To upgrade Hydra, you need to take two steps as some - bigger changes in the database schema were implemented recently[1]. You first - need to deploy `pkgs.hydra-migration`, run `hydra-backfill-ids` on the server - and then deploy `pkgs.hydra-unstable`. - - If you want to use `pkgs.hydra` from your overlay, please set `services.hydra.package` - explicitly to `pkgs.hydra` and make sure you know what you're doing. - - [1] https://github.com/NixOS/hydra/pull/711 - '' - else if versionOlder stateVersion "20.03" then hydra-migration - else hydra-unstable - ); - users.groups.hydra = { gid = config.ids.gids.hydra; }; diff --git a/nixos/modules/services/databases/postgresql.nix b/nixos/modules/services/databases/postgresql.nix index 5056d50153f6d..f582b0592774f 100644 --- a/nixos/modules/services/databases/postgresql.nix +++ b/nixos/modules/services/databases/postgresql.nix @@ -69,11 +69,16 @@ in type = types.lines; default = ""; description = '' - Defines how users authenticate themselves to the server. By - default, "trust" access to local users will always be granted - along with any other custom options. If you do not want this, - set this option using "lib.mkForce" to override this - behaviour. + Defines how users authenticate themselves to the server. See the + <link xlink:href="https://www.postgresql.org/docs/current/auth-pg-hba-conf.html"> + PostgreSQL documentation for pg_hba.conf</link> + for details on the expected format of this option. By default, + peer based authentication will be used for users connecting + via the Unix socket, and md5 password authentication will be + used for users connecting via TCP. Any added rules will be + inserted above the default rules. If you'd like to replace the + default rules entirely, you can use <function>lib.mkForce</function> in your + module. ''; }; diff --git a/nixos/modules/services/databases/redis.nix b/nixos/modules/services/databases/redis.nix index f1777854e141c..6b8853ae390b9 100644 --- a/nixos/modules/services/databases/redis.nix +++ b/nixos/modules/services/databases/redis.nix @@ -87,9 +87,12 @@ in bind = mkOption { type = with types; nullOr str; - default = null; # All interfaces - description = "The IP interface to bind to."; - example = "127.0.0.1"; + default = "127.0.0.1"; + description = '' + The IP interface to bind to. + <literal>null</literal> means "all interfaces". + ''; + example = "192.0.2.1"; }; unixSocket = mkOption { diff --git a/nixos/modules/services/databases/riak-cs.nix b/nixos/modules/services/databases/riak-cs.nix deleted file mode 100644 index fa6ac88633185..0000000000000 --- a/nixos/modules/services/databases/riak-cs.nix +++ /dev/null @@ -1,202 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - - cfg = config.services.riak-cs; - -in - -{ - - ###### interface - - options = { - - services.riak-cs = { - - enable = mkEnableOption "riak-cs"; - - package = mkOption { - type = types.package; - default = pkgs.riak-cs; - defaultText = "pkgs.riak-cs"; - example = literalExample "pkgs.riak-cs"; - description = '' - Riak package to use. - ''; - }; - - nodeName = mkOption { - type = types.str; - default = "riak-cs@127.0.0.1"; - description = '' - Name of the Erlang node. - ''; - }; - - anonymousUserCreation = mkOption { - type = types.bool; - default = false; - description = '' - Anonymous user creation. - ''; - }; - - riakHost = mkOption { - type = types.str; - default = "127.0.0.1:8087"; - description = '' - Name of riak hosting service. - ''; - }; - - listener = mkOption { - type = types.str; - default = "127.0.0.1:8080"; - description = '' - Name of Riak CS listening service. - ''; - }; - - stanchionHost = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of stanchion hosting service. - ''; - }; - - stanchionSsl = mkOption { - type = types.bool; - default = true; - description = '' - Tell stanchion to use SSL. - ''; - }; - - distributedCookie = mkOption { - type = types.str; - default = "riak"; - description = '' - Cookie for distributed node communication. All nodes in the - same cluster should use the same cookie or they will not be able to - communicate. - ''; - }; - - dataDir = mkOption { - type = types.path; - default = "/var/db/riak-cs"; - description = '' - Data directory for Riak CS. - ''; - }; - - logDir = mkOption { - type = types.path; - default = "/var/log/riak-cs"; - description = '' - Log directory for Riak CS. - ''; - }; - - extraConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>riak-cs.conf</filename>. - ''; - }; - - extraAdvancedConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>advanced.config</filename>. - ''; - }; - }; - - }; - - ###### implementation - - config = mkIf cfg.enable { - - environment.systemPackages = [ cfg.package ]; - environment.etc."riak-cs/riak-cs.conf".text = '' - nodename = ${cfg.nodeName} - distributed_cookie = ${cfg.distributedCookie} - - platform_log_dir = ${cfg.logDir} - - riak_host = ${cfg.riakHost} - listener = ${cfg.listener} - stanchion_host = ${cfg.stanchionHost} - - anonymous_user_creation = ${if cfg.anonymousUserCreation then "on" else "off"} - - ${cfg.extraConfig} - ''; - - environment.etc."riak-cs/advanced.config".text = '' - ${cfg.extraAdvancedConfig} - ''; - - users.users.riak-cs = { - name = "riak-cs"; - uid = config.ids.uids.riak-cs; - group = "riak"; - description = "Riak CS server user"; - }; - - systemd.services.riak-cs = { - description = "Riak CS Server"; - - wantedBy = [ "multi-user.target" ]; - after = [ "network.target" ]; - - path = [ - pkgs.utillinux # for `logger` - pkgs.bash - ]; - - environment.HOME = "${cfg.dataDir}"; - environment.RIAK_CS_DATA_DIR = "${cfg.dataDir}"; - environment.RIAK_CS_LOG_DIR = "${cfg.logDir}"; - environment.RIAK_CS_ETC_DIR = "/etc/riak"; - - preStart = '' - if ! test -e ${cfg.logDir}; then - mkdir -m 0755 -p ${cfg.logDir} - chown -R riak-cs ${cfg.logDir} - fi - - if ! test -e ${cfg.dataDir}; then - mkdir -m 0700 -p ${cfg.dataDir} - chown -R riak-cs ${cfg.dataDir} - fi - ''; - - serviceConfig = { - ExecStart = "${cfg.package}/bin/riak-cs console"; - ExecStop = "${cfg.package}/bin/riak-cs stop"; - StandardInput = "tty"; - User = "riak-cs"; - Group = "riak-cs"; - PermissionsStartOnly = true; - # Give Riak a decent amount of time to clean up. - TimeoutStopSec = 120; - LimitNOFILE = 65536; - }; - - unitConfig.RequiresMountsFor = [ - "${cfg.dataDir}" - "${cfg.logDir}" - "/etc/riak" - ]; - }; - }; -} diff --git a/nixos/modules/services/databases/stanchion.nix b/nixos/modules/services/databases/stanchion.nix deleted file mode 100644 index 97e55bc70c470..0000000000000 --- a/nixos/modules/services/databases/stanchion.nix +++ /dev/null @@ -1,194 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - - cfg = config.services.stanchion; - -in - -{ - - ###### interface - - options = { - - services.stanchion = { - - enable = mkEnableOption "stanchion"; - - package = mkOption { - type = types.package; - default = pkgs.stanchion; - defaultText = "pkgs.stanchion"; - example = literalExample "pkgs.stanchion"; - description = '' - Stanchion package to use. - ''; - }; - - nodeName = mkOption { - type = types.str; - default = "stanchion@127.0.0.1"; - description = '' - Name of the Erlang node. - ''; - }; - - adminKey = mkOption { - type = types.str; - default = ""; - description = '' - Name of admin user. - ''; - }; - - adminSecret = mkOption { - type = types.str; - default = ""; - description = '' - Name of admin secret - ''; - }; - - riakHost = mkOption { - type = types.str; - default = "127.0.0.1:8087"; - description = '' - Name of riak hosting service. - ''; - }; - - listener = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of Riak CS listening service. - ''; - }; - - stanchionHost = mkOption { - type = types.str; - default = "127.0.0.1:8085"; - description = '' - Name of stanchion hosting service. - ''; - }; - - distributedCookie = mkOption { - type = types.str; - default = "riak"; - description = '' - Cookie for distributed node communication. All nodes in the - same cluster should use the same cookie or they will not be able to - communicate. - ''; - }; - - dataDir = mkOption { - type = types.path; - default = "/var/db/stanchion"; - description = '' - Data directory for Stanchion. - ''; - }; - - logDir = mkOption { - type = types.path; - default = "/var/log/stanchion"; - description = '' - Log directory for Stanchion. - ''; - }; - - extraConfig = mkOption { - type = types.lines; - default = ""; - description = '' - Additional text to be appended to <filename>stanchion.conf</filename>. - ''; - }; - }; - }; - - ###### implementation - - config = mkIf cfg.enable { - - environment.systemPackages = [ cfg.package ]; - - environment.etc."stanchion/advanced.config".text = '' - [{stanchion, []}]. - ''; - - environment.etc."stanchion/stanchion.conf".text = '' - listener = ${cfg.listener} - - riak_host = ${cfg.riakHost} - - ${optionalString (cfg.adminKey == "") "#"} admin.key=${optionalString (cfg.adminKey != "") cfg.adminKey} - ${optionalString (cfg.adminSecret == "") "#"} admin.secret=${optionalString (cfg.adminSecret != "") cfg.adminSecret} - - platform_bin_dir = ${pkgs.stanchion}/bin - platform_data_dir = ${cfg.dataDir} - platform_etc_dir = /etc/stanchion - platform_lib_dir = ${pkgs.stanchion}/lib - platform_log_dir = ${cfg.logDir} - - nodename = ${cfg.nodeName} - - distributed_cookie = ${cfg.distributedCookie} - - ${cfg.extraConfig} - ''; - - users.users.stanchion = { - name = "stanchion"; - uid = config.ids.uids.stanchion; - group = "stanchion"; - description = "Stanchion server user"; - }; - - users.groups.stanchion.gid = config.ids.gids.stanchion; - - systemd.tmpfiles.rules = [ - "d '${cfg.logDir}' - stanchion stanchion --" - "d '${cfg.dataDir}' 0700 stanchion stanchion --" - ]; - - systemd.services.stanchion = { - description = "Stanchion Server"; - - wantedBy = [ "multi-user.target" ]; - after = [ "network.target" ]; - - path = [ - pkgs.utillinux # for `logger` - pkgs.bash - ]; - - environment.HOME = "${cfg.dataDir}"; - environment.STANCHION_DATA_DIR = "${cfg.dataDir}"; - environment.STANCHION_LOG_DIR = "${cfg.logDir}"; - environment.STANCHION_ETC_DIR = "/etc/stanchion"; - - serviceConfig = { - ExecStart = "${cfg.package}/bin/stanchion console"; - ExecStop = "${cfg.package}/bin/stanchion stop"; - StandardInput = "tty"; - User = "stanchion"; - Group = "stanchion"; - # Give Stanchion a decent amount of time to clean up. - TimeoutStopSec = 120; - LimitNOFILE = 65536; - }; - - unitConfig.RequiresMountsFor = [ - "${cfg.dataDir}" - "${cfg.logDir}" - "/etc/stanchion" - ]; - }; - }; -} diff --git a/nixos/modules/services/databases/victoriametrics.nix b/nixos/modules/services/databases/victoriametrics.nix index 0af5d2adf3724..5b09115bb2fb0 100644 --- a/nixos/modules/services/databases/victoriametrics.nix +++ b/nixos/modules/services/databases/victoriametrics.nix @@ -40,10 +40,10 @@ let cfg = config.services.victoriametrics; in systemd.services.victoriametrics = { description = "VictoriaMetrics time series database"; after = [ "network.target" ]; + startLimitBurst = 5; serviceConfig = { Restart = "on-failure"; RestartSec = 1; - StartLimitBurst = 5; StateDirectory = "victoriametrics"; DynamicUser = true; ExecStart = '' diff --git a/nixos/modules/services/desktops/flatpak.nix b/nixos/modules/services/desktops/flatpak.nix index 7da92cc9f2649..d0f6b66328a4c 100644 --- a/nixos/modules/services/desktops/flatpak.nix +++ b/nixos/modules/services/desktops/flatpak.nix @@ -15,6 +15,18 @@ in { options = { services.flatpak = { enable = mkEnableOption "flatpak"; + + guiPackages = mkOption { + internal = true; + type = types.listOf types.package; + default = []; + example = literalExample "[ pkgs.gnome3.gnome-software ]"; + description = '' + Packages that provide an interface for flatpak + (like gnome-software) that will be automatically available + to all users when flatpak is enabled. + ''; + }; }; }; @@ -28,7 +40,7 @@ in { } ]; - environment.systemPackages = [ pkgs.flatpak ]; + environment.systemPackages = [ pkgs.flatpak ] ++ cfg.guiPackages; services.dbus.packages = [ pkgs.flatpak ]; diff --git a/nixos/modules/services/desktops/pipewire.nix b/nixos/modules/services/desktops/pipewire.nix index 5aee59cfdcce9..5179cbaf6bc22 100644 --- a/nixos/modules/services/desktops/pipewire.nix +++ b/nixos/modules/services/desktops/pipewire.nix @@ -5,8 +5,22 @@ with lib; let cfg = config.services.pipewire; - packages = with pkgs; [ pipewire ]; + enable32BitAlsaPlugins = cfg.alsa.support32Bit + && pkgs.stdenv.isx86_64 + && pkgs.pkgsi686Linux.pipewire != null; + # The package doesn't output to $out/lib/pipewire directly so that the + # overlays can use the outputs to replace the originals in FHS environments. + # + # This doesn't work in general because of missing development information. + jack-libs = pkgs.runCommand "jack-libs" {} '' + mkdir -p "$out/lib" + ln -s "${pkgs.pipewire.jack}/lib" "$out/lib/pipewire" + ''; + pulse-libs = pkgs.runCommand "pulse-libs" {} '' + mkdir -p "$out/lib" + ln -s "${pkgs.pipewire.pulse}/lib" "$out/lib/pipewire" + ''; in { meta = { @@ -25,17 +39,67 @@ in { Automatically run pipewire when connections are made to the pipewire socket. ''; }; + + alsa = { + enable = mkEnableOption "ALSA support"; + support32Bit = mkEnableOption "32-bit ALSA support on 64-bit systems"; + }; + + jack = { + enable = mkEnableOption "JACK audio emulation"; + }; + + pulse = { + enable = mkEnableOption "PulseAudio emulation"; + }; }; }; ###### implementation config = mkIf cfg.enable { - environment.systemPackages = packages; + assertions = [ + { + assertion = cfg.pulse.enable -> !config.hardware.pulseaudio.enable; + message = "PipeWire based PulseAudio emulation doesn't use the PulseAudio service"; + } + { + assertion = cfg.jack.enable -> !config.services.jack.jackd.enable; + message = "PIpeWire based JACK emulation doesn't use the JACK service"; + } + ]; + + environment.systemPackages = [ pkgs.pipewire ] + ++ lib.optional cfg.jack.enable jack-libs + ++ lib.optional cfg.pulse.enable pulse-libs; - systemd.packages = packages; + systemd.packages = [ pkgs.pipewire ]; + # PipeWire depends on DBUS but doesn't list it. Without this booting + # into a terminal results in the service crashing with an error. systemd.user.sockets.pipewire.wantedBy = lib.mkIf cfg.socketActivation [ "sockets.target" ]; - }; + systemd.user.services.pipewire.bindsTo = [ "dbus.service" ]; + services.udev.packages = [ pkgs.pipewire ]; + # If any paths are updated here they must also be updated in the package test. + sound.extraConfig = mkIf cfg.alsa.enable '' + pcm_type.pipewire { + libs.native = ${pkgs.pipewire.lib}/lib/alsa-lib/libasound_module_pcm_pipewire.so ; + ${optionalString enable32BitAlsaPlugins + "libs.32Bit = ${pkgs.pkgsi686Linux.pipewire.lib}/lib/alsa-lib/libasound_module_pcm_pipewire.so ;"} + } + pcm.!default { + @func getenv + vars [ PCM ] + default "plug:pipewire" + playback_mode "-1" + capture_mode "-1" + } + ''; + environment.etc."alsa/conf.d/50-pipewire.conf" = mkIf cfg.alsa.enable { + source = "${pkgs.pipewire}/share/alsa/alsa.conf.d/50-pipewire.conf"; + }; + environment.sessionVariables.LD_LIBRARY_PATH = + lib.optional (cfg.jack.enable || cfg.pulse.enable) "/run/current-system/sw/lib/pipewire"; + }; } diff --git a/nixos/modules/services/desktops/telepathy.nix b/nixos/modules/services/desktops/telepathy.nix index 34596bf781849..8c50d860e5bb2 100644 --- a/nixos/modules/services/desktops/telepathy.nix +++ b/nixos/modules/services/desktops/telepathy.nix @@ -38,6 +38,11 @@ with lib; services.dbus.packages = [ pkgs.telepathy-mission-control ]; + # Enable runtime optional telepathy in gnome-shell + services.xserver.desktopManager.gnome3.sessionPath = with pkgs; [ + telepathy-glib + telepathy-logger + ]; }; } diff --git a/nixos/modules/services/hardware/bluetooth.nix b/nixos/modules/services/hardware/bluetooth.nix index dfa39e7f6024c..6f5a6d3bf2886 100644 --- a/nixos/modules/services/hardware/bluetooth.nix +++ b/nixos/modules/services/hardware/bluetooth.nix @@ -15,6 +15,8 @@ in { hardware.bluetooth = { enable = mkEnableOption "support for Bluetooth"; + hsphfpd.enable = mkEnableOption "support for hsphfpd[-prototype] implementation"; + powerOnBoot = mkOption { type = types.bool; default = true; @@ -72,7 +74,8 @@ in { }; }; - environment.systemPackages = [ bluez-bluetooth ]; + environment.systemPackages = [ bluez-bluetooth ] + ++ optionals cfg.hsphfpd.enable [ pkgs.hsphfpd ]; environment.etc."bluetooth/main.conf"= { source = pkgs.writeText "main.conf" @@ -80,19 +83,42 @@ in { }; services.udev.packages = [ bluez-bluetooth ]; - services.dbus.packages = [ bluez-bluetooth ]; + services.dbus.packages = [ bluez-bluetooth ] + ++ optionals cfg.hsphfpd.enable [ pkgs.hsphfpd ]; systemd.packages = [ bluez-bluetooth ]; systemd.services = { bluetooth = { wantedBy = [ "bluetooth.target" ]; aliases = [ "dbus-org.bluez.service" ]; + # restarting can leave people without a mouse/keyboard + unitConfig.X-RestartIfChanged = false; }; - }; + } + // (optionalAttrs cfg.hsphfpd.enable { + hsphfpd = { + after = [ "bluetooth.service" ]; + requires = [ "bluetooth.service" ]; + wantedBy = [ "multi-user.target" ]; + + description = "A prototype implementation used for connecting HSP/HFP Bluetooth devices"; + serviceConfig.ExecStart = "${pkgs.hsphfpd}/bin/hsphfpd.pl"; + }; + }) + ; systemd.user.services = { obex.aliases = [ "dbus-org.bluez.obex.service" ]; - }; + } + // (optionalAttrs cfg.hsphfpd.enable { + telephony_client = { + wantedBy = [ "default.target"]; + + description = "telephony_client for hsphfpd"; + serviceConfig.ExecStart = "${pkgs.hsphfpd}/bin/telephony_client.pl"; + }; + }) + ; }; diff --git a/nixos/modules/services/hardware/fwupd.nix b/nixos/modules/services/hardware/fwupd.nix index 222ac8e487eb3..51eca19dca32b 100644 --- a/nixos/modules/services/hardware/fwupd.nix +++ b/nixos/modules/services/hardware/fwupd.nix @@ -11,8 +11,8 @@ let "fwupd/daemon.conf" = { source = pkgs.writeText "daemon.conf" '' [fwupd] - BlacklistDevices=${lib.concatStringsSep ";" cfg.blacklistDevices} - BlacklistPlugins=${lib.concatStringsSep ";" cfg.blacklistPlugins} + DisabledDevices=${lib.concatStringsSep ";" cfg.disabledDevices} + DisabledPlugins=${lib.concatStringsSep ";" cfg.disabledPlugins} ''; }; "fwupd/uefi.conf" = { @@ -59,21 +59,21 @@ in { ''; }; - blacklistDevices = mkOption { + disabledDevices = mkOption { type = types.listOf types.str; default = []; example = [ "2082b5e0-7a64-478a-b1b2-e3404fab6dad" ]; description = '' - Allow blacklisting specific devices by their GUID + Allow disabling specific devices by their GUID ''; }; - blacklistPlugins = mkOption { + disabledPlugins = mkOption { type = types.listOf types.str; default = []; example = [ "udev" ]; description = '' - Allow blacklisting specific plugins + Allow disabling specific plugins ''; }; @@ -105,11 +105,15 @@ in { }; }; + imports = [ + (mkRenamedOptionModule [ "services" "fwupd" "blacklistDevices"] [ "services" "fwupd" "disabledDevices" ]) + (mkRenamedOptionModule [ "services" "fwupd" "blacklistPlugins"] [ "services" "fwupd" "disabledPlugins" ]) + ]; ###### implementation config = mkIf cfg.enable { # Disable test related plug-ins implicitly so that users do not have to care about them. - services.fwupd.blacklistPlugins = cfg.package.defaultBlacklistedPlugins; + services.fwupd.disabledPlugins = cfg.package.defaultDisabledPlugins; environment.systemPackages = [ cfg.package ]; diff --git a/nixos/modules/services/hardware/lcd.nix b/nixos/modules/services/hardware/lcd.nix index d78d742cd3185..dc8595ea60cde 100644 --- a/nixos/modules/services/hardware/lcd.nix +++ b/nixos/modules/services/hardware/lcd.nix @@ -151,14 +151,13 @@ in with lib; { description = "LCDproc - client"; after = [ "lcdd.service" ]; wantedBy = [ "lcd.target" ]; + # Allow restarting for eternity + startLimitIntervalSec = lib.mkIf cfg.client.restartForever 0; serviceConfig = serviceCfg // { ExecStart = "${pkg}/bin/lcdproc -f -c ${clientCfg}"; # If the server is being restarted at the same time, the client will # fail as it cannot connect, so space it out a bit. RestartSec = "5"; - # Allow restarting for eternity - StartLimitIntervalSec = lib.mkIf cfg.client.restartForever "0"; - StartLimitBurst = lib.mkIf cfg.client.restartForever "0"; }; }; }; diff --git a/nixos/modules/services/hardware/tlp.nix b/nixos/modules/services/hardware/tlp.nix index 4230f2edd2793..eb53f565a67f0 100644 --- a/nixos/modules/services/hardware/tlp.nix +++ b/nixos/modules/services/hardware/tlp.nix @@ -39,7 +39,7 @@ in default = ""; description = '' Verbatim additional configuration variables for TLP. - DEPRECATED: use services.tlp.config instead. + DEPRECATED: use services.tlp.settings instead. ''; }; }; diff --git a/nixos/modules/services/hardware/undervolt.nix b/nixos/modules/services/hardware/undervolt.nix index 054ffa35050a9..9c2f78a755ddd 100644 --- a/nixos/modules/services/hardware/undervolt.nix +++ b/nixos/modules/services/hardware/undervolt.nix @@ -3,7 +3,12 @@ with lib; let cfg = config.services.undervolt; - cliArgs = lib.cli.toGNUCommandLineShell {} { + + mkPLimit = limit: window: + if (isNull limit && isNull window) then null + else assert asserts.assertMsg (!isNull limit && !isNull window) "Both power limit and window must be set"; + "${toString limit} ${toString window}"; + cliArgs = lib.cli.toGNUCommandLine {} { inherit (cfg) verbose temp @@ -21,6 +26,9 @@ let temp-bat = cfg.tempBat; temp-ac = cfg.tempAc; + + power-limit-long = mkPLimit cfg.p1.limit cfg.p1.window; + power-limit-short = mkPLimit cfg.p2.limit cfg.p2.window; }; in { @@ -104,6 +112,40 @@ in ''; }; + p1.limit = mkOption { + type = with types; nullOr int; + default = null; + description = '' + The P1 Power Limit in Watts. + Both limit and window must be set. + ''; + }; + p1.window = mkOption { + type = with types; nullOr (oneOf [ float int ]); + default = null; + description = '' + The P1 Time Window in seconds. + Both limit and window must be set. + ''; + }; + + p2.limit = mkOption { + type = with types; nullOr int; + default = null; + description = '' + The P2 Power Limit in Watts. + Both limit and window must be set. + ''; + }; + p2.window = mkOption { + type = with types; nullOr (oneOf [ float int ]); + default = null; + description = '' + The P2 Time Window in seconds. + Both limit and window must be set. + ''; + }; + useTimer = mkOption { type = types.bool; default = false; @@ -133,7 +175,7 @@ in serviceConfig = { Type = "oneshot"; Restart = "no"; - ExecStart = "${pkgs.undervolt}/bin/undervolt ${cliArgs}"; + ExecStart = "${pkgs.undervolt}/bin/undervolt ${toString cliArgs}"; }; }; diff --git a/nixos/modules/services/logging/promtail.nix b/nixos/modules/services/logging/promtail.nix new file mode 100644 index 0000000000000..19b12daa41528 --- /dev/null +++ b/nixos/modules/services/logging/promtail.nix @@ -0,0 +1,86 @@ +{ config, lib, pkgs, ... }: with lib; +let + cfg = config.services.promtail; + + prettyJSON = conf: pkgs.runCommandLocal "promtail-config.json" {} '' + echo '${builtins.toJSON conf}' | ${pkgs.buildPackages.jq}/bin/jq 'del(._module)' > $out + ''; + + allowSystemdJournal = cfg.configuration ? scrape_configs && lib.any (v: v ? journal) cfg.configuration.scrape_configs; +in { + options.services.promtail = with types; { + enable = mkEnableOption "the Promtail ingresser"; + + + configuration = mkOption { + type = (pkgs.formats.json {}).type; + description = '' + Specify the configuration for Promtail in Nix. + ''; + }; + + extraFlags = mkOption { + type = listOf str; + default = []; + example = [ "--server.http-listen-port=3101" ]; + description = '' + Specify a list of additional command line flags, + which get escaped and are then passed to Loki. + ''; + }; + }; + + config = mkIf cfg.enable { + services.promtail.configuration.positions.filename = mkDefault "/var/cache/promtail/positions.yaml"; + + systemd.services.promtail = { + description = "Promtail log ingress"; + wantedBy = [ "multi-user.target" ]; + stopIfChanged = false; + + serviceConfig = { + Restart = "on-failure"; + + ExecStart = "${pkgs.grafana-loki}/bin/promtail -config.file=${prettyJSON cfg.configuration} ${escapeShellArgs cfg.extraFlags}"; + + ProtectSystem = "strict"; + ProtectHome = true; + PrivateTmp = true; + PrivateDevices = true; + ProtectKernelTunables = true; + ProtectControlGroups = true; + RestrictSUIDSGID = true; + PrivateMounts = true; + CacheDirectory = "promtail"; + + User = "promtail"; + Group = "promtail"; + + CapabilityBoundingSet = ""; + NoNewPrivileges = true; + + ProtectKernelModules = true; + SystemCallArchitectures = "native"; + ProtectKernelLogs = true; + ProtectClock = true; + + LockPersonality = true; + ProtectHostname = true; + RestrictRealtime = true; + MemoryDenyWriteExecute = true; + PrivateUsers = true; + + SupplementaryGroups = lib.optional (allowSystemdJournal) "systemd-journal"; + } // (optionalAttrs (!pkgs.stdenv.isAarch64) { # FIXME: figure out why this breaks on aarch64 + SystemCallFilter = "@system-service"; + }); + }; + + users.groups.promtail = {}; + users.users.promtail = { + description = "Promtail service user"; + isSystemUser = true; + group = "promtail"; + }; + }; +} diff --git a/nixos/modules/services/mail/dovecot.nix b/nixos/modules/services/mail/dovecot.nix index f5c5f795dc1b6..03e7e40e388e1 100644 --- a/nixos/modules/services/mail/dovecot.nix +++ b/nixos/modules/services/mail/dovecot.nix @@ -427,12 +427,12 @@ in wantedBy = [ "multi-user.target" ]; restartTriggers = [ cfg.configFile modulesDir ]; + startLimitIntervalSec = 60; # 1 min serviceConfig = { ExecStart = "${dovecotPkg}/sbin/dovecot -F"; ExecReload = "${dovecotPkg}/sbin/doveadm reload"; Restart = "on-failure"; RestartSec = "1s"; - StartLimitInterval = "1min"; RuntimeDirectory = [ "dovecot2" ]; }; diff --git a/nixos/modules/services/mail/freepops.nix b/nixos/modules/services/mail/freepops.nix deleted file mode 100644 index 5b729ca50a5e4..0000000000000 --- a/nixos/modules/services/mail/freepops.nix +++ /dev/null @@ -1,89 +0,0 @@ -{ config, lib, pkgs, ... }: - -with lib; - -let - cfg = config.services.mail.freepopsd; -in - -{ - options = { - services.mail.freepopsd = { - enable = mkOption { - default = false; - type = with types; bool; - description = '' - Enables Freepops, a POP3 webmail wrapper. - ''; - }; - - port = mkOption { - default = 2000; - type = with types; uniq int; - description = '' - Port on which the pop server will listen. - ''; - }; - - threads = mkOption { - default = 5; - type = with types; uniq int; - description = '' - Max simultaneous connections. - ''; - }; - - bind = mkOption { - default = "0.0.0.0"; - type = types.str; - description = '' - Bind over an IPv4 address instead of any. - ''; - }; - - logFile = mkOption { - default = "/var/log/freepopsd"; - example = "syslog"; - type = types.str; - description = '' - Filename of the log file or syslog to rely on the logging daemon. - ''; - }; - - suid = { - user = mkOption { - default = "nobody"; - type = types.str; - description = '' - User name under which freepopsd will be after binding the port. - ''; - }; - - group = mkOption { - default = "nogroup"; - type = types.str; - description = '' - Group under which freepopsd will be after binding the port. - ''; - }; - }; - - }; - }; - - config = mkIf cfg.enable { - systemd.services.freepopsd = { - description = "Freepopsd (webmail over POP3)"; - after = [ "network.target" ]; - wantedBy = [ "multi-user.target" ]; - script = '' - ${pkgs.freepops}/bin/freepopsd \ - -p ${toString cfg.port} \ - -t ${toString cfg.threads} \ - -b ${cfg.bind} \ - -vv -l ${cfg.logFile} \ - -s ${cfg.suid.user}.${cfg.suid.group} - ''; - }; - }; -} diff --git a/nixos/modules/services/mail/roundcube.nix b/nixos/modules/services/mail/roundcube.nix index a0bbab64985b2..ee7aa7e22fb96 100644 --- a/nixos/modules/services/mail/roundcube.nix +++ b/nixos/modules/services/mail/roundcube.nix @@ -204,6 +204,11 @@ in }; systemd.services.phpfpm-roundcube.after = [ "roundcube-setup.service" ]; + # Restart on config changes. + systemd.services.phpfpm-roundcube.restartTriggers = [ + config.environment.etc."roundcube/config.inc.php".source + ]; + systemd.services.roundcube-setup = mkMerge [ (mkIf (cfg.database.host == "localhost") { requires = [ "postgresql.service" ]; diff --git a/nixos/modules/services/mail/rspamd.nix b/nixos/modules/services/mail/rspamd.nix index aacdbe2aeed25..86a3f52107d51 100644 --- a/nixos/modules/services/mail/rspamd.nix +++ b/nixos/modules/services/mail/rspamd.nix @@ -153,7 +153,7 @@ let ${concatStringsSep "\n" (mapAttrsToList (name: value: let includeName = if name == "rspamd_proxy" then "proxy" else name; - tryOverride = if value.extraConfig == "" then "true" else "false"; + tryOverride = boolToString (value.extraConfig == ""); in '' worker "${value.type}" { type = "${value.type}"; diff --git a/nixos/modules/services/misc/autorandr.nix b/nixos/modules/services/misc/autorandr.nix index cf7fb5f78d3d5..dfb418af6edeb 100644 --- a/nixos/modules/services/misc/autorandr.nix +++ b/nixos/modules/services/misc/autorandr.nix @@ -37,9 +37,9 @@ in { description = "Autorandr execution hook"; after = [ "sleep.target" ]; + startLimitIntervalSec = 5; + startLimitBurst = 1; serviceConfig = { - StartLimitInterval = 5; - StartLimitBurst = 1; ExecStart = "${pkgs.autorandr}/bin/autorandr --batch --change --default ${cfg.defaultTarget}"; Type = "oneshot"; RemainAfterExit = false; diff --git a/nixos/modules/services/misc/cfdyndns.nix b/nixos/modules/services/misc/cfdyndns.nix index dcf416022734f..15af1f50da1d6 100644 --- a/nixos/modules/services/misc/cfdyndns.nix +++ b/nixos/modules/services/misc/cfdyndns.nix @@ -6,6 +6,12 @@ let cfg = config.services.cfdyndns; in { + imports = [ + (mkRemovedOptionModule + [ "services" "cfdyndns" "apikey" ] + "Use services.cfdyndns.apikeyFile instead.") + ]; + options = { services.cfdyndns = { enable = mkEnableOption "Cloudflare Dynamic DNS Client"; @@ -17,10 +23,12 @@ in ''; }; - apikey = mkOption { - type = types.str; + apikeyFile = mkOption { + default = null; + type = types.nullOr types.str; description = '' - The API Key to use to authenticate to CloudFlare. + The path to a file containing the API Key + used to authenticate with CloudFlare. ''; }; @@ -45,13 +53,17 @@ in Type = "simple"; User = config.ids.uids.cfdyndns; Group = config.ids.gids.cfdyndns; - ExecStart = "/bin/sh -c '${pkgs.cfdyndns}/bin/cfdyndns'"; }; environment = { CLOUDFLARE_EMAIL="${cfg.email}"; - CLOUDFLARE_APIKEY="${cfg.apikey}"; CLOUDFLARE_RECORDS="${concatStringsSep "," cfg.records}"; }; + script = '' + ${optionalString (cfg.apikeyFile != null) '' + export CLOUDFLARE_APIKEY="$(cat ${escapeShellArg cfg.apikeyFile})" + ''} + ${pkgs.cfdyndns}/bin/cfdyndns + ''; }; users.users = { diff --git a/nixos/modules/services/misc/cgminer.nix b/nixos/modules/services/misc/cgminer.nix index 7635c2a0f4e95..fa9c8c54509ea 100644 --- a/nixos/modules/services/misc/cgminer.nix +++ b/nixos/modules/services/misc/cgminer.nix @@ -126,12 +126,12 @@ in GPU_USE_SYNC_OBJECTS = "1"; }; + startLimitIntervalSec = 60; # 1 min serviceConfig = { ExecStart = "${pkgs.cgminer}/bin/cgminer --syslog --text-only --config ${cgminerConfig}"; User = cfg.user; RestartSec = "30s"; Restart = "always"; - StartLimitInterval = "1m"; }; }; diff --git a/nixos/modules/services/misc/nix-daemon.nix b/nixos/modules/services/misc/nix-daemon.nix index 2680b1cc0d3b8..ed05882a6343a 100644 --- a/nixos/modules/services/misc/nix-daemon.nix +++ b/nixos/modules/services/misc/nix-daemon.nix @@ -45,7 +45,7 @@ let trusted-substituters = ${toString cfg.trustedBinaryCaches} trusted-public-keys = ${toString cfg.binaryCachePublicKeys} auto-optimise-store = ${boolToString cfg.autoOptimiseStore} - require-sigs = ${if cfg.requireSignedBinaryCaches then "true" else "false"} + require-sigs = ${boolToString cfg.requireSignedBinaryCaches} trusted-users = ${toString cfg.trustedUsers} allowed-users = ${toString cfg.allowedUsers} ${optionalString (!cfg.distributedBuilds) '' diff --git a/nixos/modules/services/misc/octoprint.nix b/nixos/modules/services/misc/octoprint.nix index e2fbd3b401cc5..a69e650730508 100644 --- a/nixos/modules/services/misc/octoprint.nix +++ b/nixos/modules/services/misc/octoprint.nix @@ -68,7 +68,7 @@ in plugins = mkOption { default = plugins: []; defaultText = "plugins: []"; - example = literalExample "plugins: with plugins; [ m33-fio stlviewer ]"; + example = literalExample "plugins: with plugins; [ themeify stlviewer ]"; description = "Additional plugins to be used. Available plugins are passed through the plugins input."; }; diff --git a/nixos/modules/services/misc/safeeyes.nix b/nixos/modules/services/misc/safeeyes.nix index 6ecb0d13187c1..1e748195e41aa 100644 --- a/nixos/modules/services/misc/safeeyes.nix +++ b/nixos/modules/services/misc/safeeyes.nix @@ -32,14 +32,14 @@ in wantedBy = [ "graphical-session.target" ]; partOf = [ "graphical-session.target" ]; + startLimitIntervalSec = 350; + startLimitBurst = 10; serviceConfig = { ExecStart = '' ${pkgs.safeeyes}/bin/safeeyes ''; Restart = "on-failure"; RestartSec = 3; - StartLimitInterval = 350; - StartLimitBurst = 10; }; }; diff --git a/nixos/modules/services/monitoring/loki.nix b/nixos/modules/services/monitoring/loki.nix index f4eec7e0d2841..51cabaa274a3b 100644 --- a/nixos/modules/services/monitoring/loki.nix +++ b/nixos/modules/services/monitoring/loki.nix @@ -39,7 +39,7 @@ in { }; configuration = mkOption { - type = types.attrs; + type = (pkgs.formats.json {}).type; default = {}; description = '' Specify the configuration for Loki in Nix. @@ -78,6 +78,8 @@ in { ''; }]; + environment.systemPackages = [ pkgs.grafana-loki ]; # logcli + users.groups.${cfg.group} = { }; users.users.${cfg.user} = { description = "Loki Service User"; diff --git a/nixos/modules/services/monitoring/mackerel-agent.nix b/nixos/modules/services/monitoring/mackerel-agent.nix new file mode 100644 index 0000000000000..7046de9d403cf --- /dev/null +++ b/nixos/modules/services/monitoring/mackerel-agent.nix @@ -0,0 +1,111 @@ +{ config, lib, pkgs, ... }: + +with lib; + +let + cfg = config.services.mackerel-agent; + settingsFmt = pkgs.formats.toml {}; +in { + options.services.mackerel-agent = { + enable = mkEnableOption "mackerel.io agent"; + + # the upstream package runs as root, but doesn't seem to be strictly + # necessary for basic functionality + runAsRoot = mkEnableOption "Whether to run as root."; + + autoRetirement = mkEnableOption '' + Whether to automatically retire the host upon OS shutdown. + ''; + + apiKeyFile = mkOption { + type = types.path; + default = ""; + example = "/run/keys/mackerel-api-key"; + description = '' + Path to file containing the Mackerel API key. The file should contain a + single line of the following form: + + <literallayout>apikey = "EXAMPLE_API_KEY"</literallayout> + ''; + }; + + settings = mkOption { + description = '' + Options for mackerel-agent.conf. + + Documentation: + <link xlink:href="https://mackerel.io/docs/entry/spec/agent"/> + ''; + + default = {}; + example = { + verbose = false; + silent = false; + }; + + type = types.submodule { + freeformType = settingsFmt.type; + + options.host_status = { + on_start = mkOption { + type = types.enum [ "working" "standby" "maintenance" "poweroff" ]; + description = "Host status after agent startup."; + default = "working"; + }; + on_stop = mkOption { + type = types.enum [ "working" "standby" "maintenance" "poweroff" ]; + description = "Host status after agent shutdown."; + default = "poweroff"; + }; + }; + + options.diagnostic = + mkEnableOption "Collect memory usage for the agent itself"; + }; + }; + }; + + config = mkIf cfg.enable { + environment.systemPackages = with pkgs; [ mackerel-agent ]; + + environment.etc = { + "mackerel-agent/mackerel-agent.conf".source = + settingsFmt.generate "mackerel-agent.conf" cfg.settings; + "mackerel-agent/conf.d/api-key.conf".source = cfg.apiKeyFile; + }; + + services.mackerel-agent.settings = { + root = mkDefault "/var/lib/mackerel-agent"; + pidfile = mkDefault "/run/mackerel-agent/mackerel-agent.pid"; + + # conf.d stores the symlink to cfg.apiKeyFile + include = mkDefault "/etc/mackerel-agent/conf.d/*.conf"; + }; + + # upstream service file in https://git.io/JUt4Q + systemd.services.mackerel-agent = { + description = "mackerel.io agent"; + after = [ "network-online.target" "nss-lookup.target" ]; + wantedBy = [ "multi-user.target" ]; + environment = { + MACKEREL_PLUGIN_WORKDIR = mkDefault "%C/mackerel-agent"; + }; + serviceConfig = { + DynamicUser = !cfg.runAsRoot; + PrivateTmp = mkDefault true; + CacheDirectory = "mackerel-agent"; + ConfigurationDirectory = "mackerel-agent"; + RuntimeDirectory = "mackerel-agent"; + StateDirectory = "mackerel-agent"; + ExecStart = "${pkgs.mackerel-agent}/bin/mackerel-agent supervise"; + ExecStopPost = mkIf cfg.autoRetirement "${pkg.mackerel-agent}/bin/mackerel-agent retire -force"; + ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + LimitNOFILE = mkDefault 65536; + LimitNPROC = mkDefault 65536; + }; + restartTriggers = [ + config.environment.etc."mackerel-agent/mackerel-agent.conf".source + ]; + }; + }; +} diff --git a/nixos/modules/services/monitoring/prometheus/default.nix b/nixos/modules/services/monitoring/prometheus/default.nix index 98aaa9c0f030a..72428957109c6 100644 --- a/nixos/modules/services/monitoring/prometheus/default.nix +++ b/nixos/modules/services/monitoring/prometheus/default.nix @@ -45,7 +45,7 @@ let cmdlineArgs = cfg.extraFlags ++ [ "--storage.tsdb.path=${workingDir}/data/" - "--config.file=${prometheusYml}" + "--config.file=/run/prometheus/prometheus-substituted.yaml" "--web.listen-address=${cfg.listenAddress}:${builtins.toString cfg.port}" "--alertmanager.notification-queue-capacity=${toString cfg.alertmanagerNotificationQueueCapacity}" "--alertmanager.timeout=${toString cfg.alertmanagerTimeout}s" @@ -522,6 +522,45 @@ in { ''; }; + environmentFile = mkOption { + type = types.nullOr types.path; + default = null; + example = "/root/prometheus.env"; + description = '' + Environment file as defined in <citerefentry> + <refentrytitle>systemd.exec</refentrytitle><manvolnum>5</manvolnum> + </citerefentry>. + + Secrets may be passed to the service without adding them to the + world-readable Nix store, by specifying placeholder variables as + the option value in Nix and setting these variables accordingly in the + environment file. + + Environment variables from this file will be interpolated into the + config file using envsubst with this syntax: + <literal>$ENVIRONMENT ''${VARIABLE}</literal> + + <programlisting> + # Example scrape config entry handling an OAuth bearer token + { + job_name = "home_assistant"; + metrics_path = "/api/prometheus"; + scheme = "https"; + bearer_token = "\''${HOME_ASSISTANT_BEARER_TOKEN}"; + [...] + } + </programlisting> + + <programlisting> + # Content of the environment file + HOME_ASSISTANT_BEARER_TOKEN=someoauthbearertoken + </programlisting> + + Note that this file needs to be available on the host on which + <literal>Prometheus</literal> is running. + ''; + }; + configText = mkOption { type = types.nullOr types.lines; default = null; @@ -662,12 +701,19 @@ in { systemd.services.prometheus = { wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; + preStart = '' + ${lib.getBin pkgs.envsubst}/bin/envsubst -o "/run/prometheus/prometheus-substituted.yaml" \ + -i "${prometheusYml}" + ''; serviceConfig = { ExecStart = "${cfg.package}/bin/prometheus" + optionalString (length cmdlineArgs != 0) (" \\\n " + concatStringsSep " \\\n " cmdlineArgs); User = "prometheus"; Restart = "always"; + EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ]; + RuntimeDirectory = "prometheus"; + RuntimeDirectoryMode = "0700"; WorkingDirectory = workingDir; StateDirectory = cfg.stateDir; }; diff --git a/nixos/modules/services/monitoring/prometheus/exporters.nix b/nixos/modules/services/monitoring/prometheus/exporters.nix index 1233e5cdd1a9c..995afca96ff81 100644 --- a/nixos/modules/services/monitoring/prometheus/exporters.nix +++ b/nixos/modules/services/monitoring/prometheus/exporters.nix @@ -43,7 +43,9 @@ let "postgres" "redis" "rspamd" + "rtl_433" "snmp" + "sql" "surfboard" "tor" "unifi" @@ -217,6 +219,14 @@ in Please specify either 'services.prometheus.exporters.mail.configuration' or 'services.prometheus.exporters.mail.configFile'. ''; + } { + assertion = cfg.sql.enable -> ( + (cfg.sql.configFile == null) != (cfg.sql.configuration == null) + ); + message = '' + Please specify either 'services.prometheus.exporters.sql.configuration' or + 'services.prometheus.exporters.sql.configFile' + ''; } ]; }] ++ [(mkIf config.services.minio.enable { services.prometheus.exporters.minio.minioAddress = mkDefault "http://localhost:9000"; @@ -224,6 +234,8 @@ in services.prometheus.exporters.minio.minioAccessSecret = mkDefault config.services.minio.secretKey; })] ++ [(mkIf config.services.rspamd.enable { services.prometheus.exporters.rspamd.url = mkDefault "http://localhost:11334/stat"; + })] ++ [(mkIf config.services.prometheus.exporters.rtl_433.enable { + hardware.rtl-sdr.enable = mkDefault true; })] ++ [(mkIf config.services.nginx.enable { systemd.services.prometheus-nginx-exporter.after = [ "nginx.service" ]; systemd.services.prometheus-nginx-exporter.requires = [ "nginx.service" ]; diff --git a/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix b/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix new file mode 100644 index 0000000000000..01e420db38978 --- /dev/null +++ b/nixos/modules/services/monitoring/prometheus/exporters/rtl_433.nix @@ -0,0 +1,78 @@ +{ config, lib, pkgs, options }: + +let + cfg = config.services.prometheus.exporters.rtl_433; +in +{ + port = 9550; + + extraOpts = let + mkMatcherOptionType = field: description: with lib.types; + listOf (submodule { + options = { + name = lib.mkOption { + type = str; + description = "Name to match."; + }; + "${field}" = lib.mkOption { + type = int; + inherit description; + }; + location = lib.mkOption { + type = str; + description = "Location to match."; + }; + }; + }); + in + { + rtl433Flags = lib.mkOption { + type = lib.types.str; + default = "-C si"; + example = "-C si -R 19"; + description = '' + Flags passed verbatim to rtl_433 binary. + Having <literal>-C si</literal> (the default) is recommended since only Celsius temperatures are parsed. + ''; + }; + channels = lib.mkOption { + type = mkMatcherOptionType "channel" "Channel to match."; + default = []; + example = [ + { name = "Acurite"; channel = 6543; location = "Kitchen"; } + ]; + description = '' + List of channel matchers to export. + ''; + }; + ids = lib.mkOption { + type = mkMatcherOptionType "id" "ID to match."; + default = []; + example = [ + { name = "Nexus"; id = 1; location = "Bedroom"; } + ]; + description = '' + List of ID matchers to export. + ''; + }; + }; + + serviceOpts = { + serviceConfig = { + # rtl-sdr udev rules make supported USB devices +rw by plugdev. + SupplementaryGroups = "plugdev"; + ExecStart = let + matchers = (map (m: + "--channel_matcher '${m.name},${toString m.channel},${m.location}'" + ) cfg.channels) ++ (map (m: + "--id_matcher '${m.name},${toString m.id},${m.location}'" + ) cfg.ids); in '' + ${pkgs.prometheus-rtl_433-exporter}/bin/rtl_433_prometheus \ + -listen ${cfg.listenAddress}:${toString cfg.port} \ + -subprocess "${pkgs.rtl_433}/bin/rtl_433 -F json ${cfg.rtl433Flags}" \ + ${lib.concatStringsSep " \\\n " matchers} \ + ${lib.concatStringsSep " \\\n " cfg.extraFlags} + ''; + }; + }; +} diff --git a/nixos/modules/services/monitoring/prometheus/exporters/sql.nix b/nixos/modules/services/monitoring/prometheus/exporters/sql.nix new file mode 100644 index 0000000000000..d9be724ebc036 --- /dev/null +++ b/nixos/modules/services/monitoring/prometheus/exporters/sql.nix @@ -0,0 +1,104 @@ +{ config, lib, pkgs, options }: +with lib; +let + cfg = config.services.prometheus.exporters.sql; + cfgOptions = { + options = with types; { + jobs = mkOption { + type = attrsOf (submodule jobOptions); + default = { }; + description = "An attrset of metrics scraping jobs to run."; + }; + }; + }; + jobOptions = { + options = with types; { + interval = mkOption { + type = str; + description = '' + How often to run this job, specified in + <link xlink:href="https://golang.org/pkg/time/#ParseDuration">Go duration</link> format. + ''; + }; + connections = mkOption { + type = listOf str; + description = "A list of connection strings of the SQL servers to scrape metrics from"; + }; + startupSql = mkOption { + type = listOf str; + default = []; + description = "A list of SQL statements to execute once after making a connection."; + }; + queries = mkOption { + type = attrsOf (submodule queryOptions); + description = "SQL queries to run."; + }; + }; + }; + queryOptions = { + options = with types; { + help = mkOption { + type = nullOr str; + default = null; + description = "A human-readable description of this metric."; + }; + labels = mkOption { + type = listOf str; + default = [ ]; + description = "A set of columns that will be used as Prometheus labels."; + }; + query = mkOption { + type = str; + description = "The SQL query to run."; + }; + values = mkOption { + type = listOf str; + description = "A set of columns that will be used as values of this metric."; + }; + }; + }; + + configFile = + if cfg.configFile != null + then cfg.configFile + else + let + nameInline = mapAttrsToList (k: v: v // { name = k; }); + renameStartupSql = j: removeAttrs (j // { startup_sql = j.startupSql; }) [ "startupSql" ]; + configuration = { + jobs = map renameStartupSql + (nameInline (mapAttrs (k: v: (v // { queries = nameInline v.queries; })) cfg.configuration.jobs)); + }; + in + builtins.toFile "config.yaml" (builtins.toJSON configuration); +in +{ + extraOpts = { + configFile = mkOption { + type = with types; nullOr path; + default = null; + description = '' + Path to configuration file. + ''; + }; + configuration = mkOption { + type = with types; nullOr (submodule cfgOptions); + default = null; + description = '' + Exporter configuration as nix attribute set. Mutually exclusive with 'configFile' option. + ''; + }; + }; + + port = 9237; + serviceOpts = { + serviceConfig = { + ExecStart = '' + ${pkgs.prometheus-sql-exporter}/bin/sql_exporter \ + -web.listen-address ${cfg.listenAddress}:${toString cfg.port} \ + -config.file ${configFile} \ + ${concatStringsSep " \\\n " cfg.extraFlags} + ''; + }; + }; +} diff --git a/nixos/modules/services/monitoring/teamviewer.nix b/nixos/modules/services/monitoring/teamviewer.nix index 8d781d82d0864..ce9e57a187cdc 100644 --- a/nixos/modules/services/monitoring/teamviewer.nix +++ b/nixos/modules/services/monitoring/teamviewer.nix @@ -31,14 +31,14 @@ in after = [ "NetworkManager-wait-online.service" "network.target" ]; preStart = "mkdir -pv /var/lib/teamviewer /var/log/teamviewer"; + startLimitIntervalSec = 60; + startLimitBurst = 10; serviceConfig = { Type = "forking"; ExecStart = "${pkgs.teamviewer}/bin/teamviewerd -d"; PIDFile = "/run/teamviewerd.pid"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; Restart = "on-abort"; - StartLimitInterval = "60"; - StartLimitBurst = "10"; }; }; }; diff --git a/nixos/modules/services/monitoring/telegraf.nix b/nixos/modules/services/monitoring/telegraf.nix index 5d131557e8be9..b341a9005c2a4 100644 --- a/nixos/modules/services/monitoring/telegraf.nix +++ b/nixos/modules/services/monitoring/telegraf.nix @@ -5,14 +5,8 @@ with lib; let cfg = config.services.telegraf; - configFile = pkgs.runCommand "config.toml" { - buildInputs = [ pkgs.remarshal ]; - preferLocalBuild = true; - } '' - remarshal -if json -of toml \ - < ${pkgs.writeText "config.json" (builtins.toJSON cfg.extraConfig)} \ - > $out - ''; + settingsFormat = pkgs.formats.toml {}; + configFile = settingsFormat.generate "config.toml" cfg.extraConfig; in { ###### interface options = { @@ -26,22 +20,31 @@ in { type = types.package; }; + environmentFiles = mkOption { + type = types.listOf types.path; + default = []; + example = "/run/keys/telegraf.env"; + description = '' + File to load as environment file. Environment variables + from this file will be interpolated into the config file + using envsubst with this syntax: + <literal>$ENVIRONMENT ''${VARIABLE}</literal> + This is useful to avoid putting secrets into the nix store. + ''; + }; + extraConfig = mkOption { default = {}; description = "Extra configuration options for telegraf"; - type = types.attrs; + type = settingsFormat.type; example = { - outputs = { - influxdb = { - urls = ["http://localhost:8086"]; - database = "telegraf"; - }; + outputs.influxdb = { + urls = ["http://localhost:8086"]; + database = "telegraf"; }; - inputs = { - statsd = { - service_address = ":8125"; - delete_timings = true; - }; + inputs.statsd = { + service_address = ":8125"; + delete_timings = true; }; }; }; @@ -51,15 +54,28 @@ in { ###### implementation config = mkIf config.services.telegraf.enable { - systemd.services.telegraf = { + systemd.services.telegraf = let + finalConfigFile = if config.services.telegraf.environmentFiles == [] + then configFile + else "/var/run/telegraf/config.toml"; + in { description = "Telegraf Agent"; wantedBy = [ "multi-user.target" ]; after = [ "network-online.target" ]; serviceConfig = { - ExecStart=''${cfg.package}/bin/telegraf -config "${configFile}"''; + EnvironmentFile = config.services.telegraf.environmentFiles; + ExecStartPre = lib.optional (config.services.telegraf.environmentFiles != []) + (pkgs.writeShellScript "pre-start" '' + umask 077 + ${pkgs.envsubst}/bin/envsubst -i "${configFile}" > /var/run/telegraf/config.toml + ''); + ExecStart=''${cfg.package}/bin/telegraf -config ${finalConfigFile}''; ExecReload="${pkgs.coreutils}/bin/kill -HUP $MAINPID"; + RuntimeDirectory = "telegraf"; User = "telegraf"; Restart = "on-failure"; + # for ping probes + AmbientCapabilities = [ "CAP_NET_RAW" ]; }; }; diff --git a/nixos/modules/services/network-filesystems/ceph.nix b/nixos/modules/services/network-filesystems/ceph.nix index d17959a6a3059..f2dc740fd88e9 100644 --- a/nixos/modules/services/network-filesystems/ceph.nix +++ b/nixos/modules/services/network-filesystems/ceph.nix @@ -28,6 +28,9 @@ let # Don't start services that are not yet initialized unitConfig.ConditionPathExists = "/var/lib/${stateDirectory}/keyring"; + startLimitBurst = + if daemonType == "osd" then 30 else if lib.elem daemonType ["mgr" "mds"] then 3 else 5; + startLimitIntervalSec = 60 * 30; # 30 mins serviceConfig = { LimitNOFILE = 1048576; @@ -39,8 +42,6 @@ let ProtectHome = "true"; ProtectSystem = "full"; Restart = "on-failure"; - StartLimitBurst = "5"; - StartLimitInterval = "30min"; StateDirectory = stateDirectory; User = "ceph"; Group = if daemonType == "osd" then "disk" else "ceph"; @@ -48,13 +49,10 @@ let -f --cluster ${clusterName} --id ${daemonId}''; } // optionalAttrs (daemonType == "osd") { ExecStartPre = ''${ceph.lib}/libexec/ceph/ceph-osd-prestart.sh --id ${daemonId} --cluster ${clusterName}''; - StartLimitBurst = "30"; RestartSec = "20s"; PrivateDevices = "no"; # osd needs disk access } // optionalAttrs ( daemonType == "mon") { RestartSec = "10"; - } // optionalAttrs (lib.elem daemonType ["mgr" "mds"]) { - StartLimitBurst = "3"; }; }); diff --git a/nixos/modules/services/network-filesystems/ipfs.nix b/nixos/modules/services/network-filesystems/ipfs.nix index f298f831fa7b2..2082d513161e3 100644 --- a/nixos/modules/services/network-filesystems/ipfs.nix +++ b/nixos/modules/services/network-filesystems/ipfs.nix @@ -44,6 +44,13 @@ in { enable = mkEnableOption "Interplanetary File System (WARNING: may cause severe network degredation)"; + package = mkOption { + type = types.package; + default = pkgs.ipfs; + defaultText = "pkgs.ipfs"; + description = "Which IPFS package to use."; + }; + user = mkOption { type = types.str; default = "ipfs"; @@ -176,7 +183,7 @@ in { ###### implementation config = mkIf cfg.enable { - environment.systemPackages = [ pkgs.ipfs ]; + environment.systemPackages = [ cfg.package ]; environment.variables.IPFS_PATH = cfg.dataDir; programs.fuse = mkIf cfg.autoMount { @@ -207,14 +214,14 @@ in { "d '${cfg.ipnsMountDir}' - ${cfg.user} ${cfg.group} - -" ]; - systemd.packages = [ pkgs.ipfs ]; + systemd.packages = [ cfg.package ]; systemd.services.ipfs-init = { description = "IPFS Initializer"; environment.IPFS_PATH = cfg.dataDir; - path = [ pkgs.ipfs ]; + path = [ cfg.package ]; script = '' if [[ ! -f ${cfg.dataDir}/config ]]; then @@ -239,7 +246,7 @@ in { }; systemd.services.ipfs = { - path = [ "/run/wrappers" pkgs.ipfs ]; + path = [ "/run/wrappers" cfg.package ]; environment.IPFS_PATH = cfg.dataDir; wants = [ "ipfs-init.service" ]; @@ -267,7 +274,7 @@ in { cfg.extraConfig)) ); serviceConfig = { - ExecStart = ["" "${pkgs.ipfs}/bin/ipfs daemon ${ipfsFlags}"]; + ExecStart = ["" "${cfg.package}/bin/ipfs daemon ${ipfsFlags}"]; User = cfg.user; Group = cfg.group; } // optionalAttrs (cfg.serviceFdlimit != null) { LimitNOFILE = cfg.serviceFdlimit; }; diff --git a/nixos/modules/services/networking/avahi-daemon.nix b/nixos/modules/services/networking/avahi-daemon.nix index 2900c37f990f4..0b7d5575c11fc 100644 --- a/nixos/modules/services/networking/avahi-daemon.nix +++ b/nixos/modules/services/networking/avahi-daemon.nix @@ -86,7 +86,8 @@ in ipv6 = mkOption { type = types.bool; - default = false; + default = config.networking.enableIPv6; + defaultText = "config.networking.enableIPv6"; description = "Whether to use IPv6."; }; diff --git a/nixos/modules/services/networking/babeld.nix b/nixos/modules/services/networking/babeld.nix index e62c74d0069df..90395dbd3c54c 100644 --- a/nixos/modules/services/networking/babeld.nix +++ b/nixos/modules/services/networking/babeld.nix @@ -87,9 +87,37 @@ in description = "Babel routing daemon"; after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; - serviceConfig.ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile}"; + serviceConfig = { + ExecStart = "${pkgs.babeld}/bin/babeld -c ${configFile} -I /run/babeld/babeld.pid -S /var/lib/babeld/state"; + CapabilityBoundingSet = [ "CAP_NET_ADMIN" ]; + IPAddressAllow = [ "fe80::/64" "ff00::/8" "::1/128" "127.0.0.0/8" ]; + IPAddressDeny = "any"; + LockPersonality = true; + NoNewPrivileges = true; + MemoryDenyWriteExecute = true; + ProtectSystem = "strict"; + ProtectClock = true; + ProtectKernelTunables = false; # Couldn't write sysctl: Read-only file system + ProtectKernelModules = true; + ProtectKernelLogs = true; + ProtectControlGroups = true; + RestrictAddressFamilies = [ "AF_NETLINK" "AF_INET6" ]; + RestrictNamespaces = true; + RestrictRealtime = true; + RestrictSUIDSGID = true; + RemoveIPC = true; + ProtectHome = true; + ProtectHostname = true; + PrivateMounts = true; + PrivateTmp = true; + PrivateDevices = true; + PrivateUsers = false; # kernel_route(ADD): Operation not permitted + SystemCallArchitectures = "native"; + SystemCallFilter = [ "@system-service" ]; + UMask = "0177"; + RuntimeDirectory = "babeld"; + StateDirectory = "babeld"; + }; }; - }; - } diff --git a/nixos/modules/services/networking/blockbook-frontend.nix b/nixos/modules/services/networking/blockbook-frontend.nix index dde24522756af..ca323e495ec1d 100644 --- a/nixos/modules/services/networking/blockbook-frontend.nix +++ b/nixos/modules/services/networking/blockbook-frontend.nix @@ -158,15 +158,21 @@ let type = types.attrs; default = {}; example = literalExample '' { - alternative_estimate_fee = "whatthefee-disabled"; - alternative_estimate_fee_params = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}"; - fiat_rates = "coingecko"; - fiat_rates_params = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}"; - coin_shortcut = "BTC"; - coin_label = "Bitcoin"; - xpub_magic = 76067358; - xpub_magic_segwit_p2sh = 77429938; - xpub_magic_segwit_native = 78792518; + "alternative_estimate_fee" = "whatthefee-disabled"; + "alternative_estimate_fee_params" = "{\"url\": \"https://whatthefee.io/data.json\", \"periodSeconds\": 60}"; + "fiat_rates" = "coingecko"; + "fiat_rates_params" = "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"bitcoin\", \"periodSeconds\": 60}"; + "coin_shortcut" = "BTC"; + "coin_label" = "Bitcoin"; + "parse" = true; + "subversion" = ""; + "address_format" = ""; + "xpub_magic" = 76067358; + "xpub_magic_segwit_p2sh" = 77429938; + "xpub_magic_segwit_native" = 78792518; + "mempool_workers" = 8; + "mempool_sub_workers" = 2; + "block_addresses_to_keep" = 300; }''; description = '' Additional configurations to be appended to <filename>coin.conf</filename>. diff --git a/nixos/modules/services/networking/cjdns.nix b/nixos/modules/services/networking/cjdns.nix index 5f8ac96b2292c..f116d6392ea7d 100644 --- a/nixos/modules/services/networking/cjdns.nix +++ b/nixos/modules/services/networking/cjdns.nix @@ -264,10 +264,10 @@ in '' ); + startLimitIntervalSec = 0; serviceConfig = { Type = "forking"; Restart = "always"; - StartLimitInterval = 0; RestartSec = 1; CapabilityBoundingSet = "CAP_NET_ADMIN CAP_NET_RAW CAP_SETUID"; ProtectSystem = true; diff --git a/nixos/modules/services/networking/dhcpcd.nix b/nixos/modules/services/networking/dhcpcd.nix index 0507b739d4999..d10bffd914743 100644 --- a/nixos/modules/services/networking/dhcpcd.nix +++ b/nixos/modules/services/networking/dhcpcd.nix @@ -69,6 +69,11 @@ let if-carrier-up = ""; }.${cfg.wait}} + ${optionalString (config.networking.enableIPv6 == false) '' + # Don't solicit or accept IPv6 Router Advertisements and DHCPv6 if disabled IPv6 + noipv6 + ''} + ${cfg.extraConfig} ''; diff --git a/nixos/modules/services/networking/dnscrypt-wrapper.nix b/nixos/modules/services/networking/dnscrypt-wrapper.nix index b9333cd19a2a5..ee7e9b0454ded 100644 --- a/nixos/modules/services/networking/dnscrypt-wrapper.nix +++ b/nixos/modules/services/networking/dnscrypt-wrapper.nix @@ -55,7 +55,10 @@ let rotateKeys = '' # check if keys are not expired keyValid() { - fingerprint=$(dnscrypt-wrapper --show-provider-publickey | awk '{print $(NF)}') + fingerprint=$(dnscrypt-wrapper \ + --show-provider-publickey \ + --provider-publickey-file=${publicKey} \ + | awk '{print $(NF)}') dnscrypt-proxy --test=${toString (cfg.keys.checkInterval + 1)} \ --resolver-address=127.0.0.1:${toString cfg.port} \ --provider-name=${cfg.providerName} \ diff --git a/nixos/modules/services/networking/dnsdist.nix b/nixos/modules/services/networking/dnsdist.nix index c48835e73612b..05c2bdef83e70 100644 --- a/nixos/modules/services/networking/dnsdist.nix +++ b/nixos/modules/services/networking/dnsdist.nix @@ -41,6 +41,7 @@ in { systemd.services.dnsdist = { wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 0; serviceConfig = { DynamicUser = true; diff --git a/nixos/modules/services/networking/kresd.nix b/nixos/modules/services/networking/kresd.nix index ccb34163d5f36..6f1c4c48b430a 100644 --- a/nixos/modules/services/networking/kresd.nix +++ b/nixos/modules/services/networking/kresd.nix @@ -23,18 +23,14 @@ let ''; configFile = pkgs.writeText "kresd.conf" ( - optionalString (cfg.listenDoH != []) '' - modules.load('http') - '' + "" + concatMapStrings (mkListen "dns") cfg.listenPlain + concatMapStrings (mkListen "tls") cfg.listenTLS - + concatMapStrings (mkListen "doh") cfg.listenDoH + + concatMapStrings (mkListen "doh2") cfg.listenDoH + cfg.extraConfig ); - package = if cfg.listenDoH == [] - then pkgs.knot-resolver # never force `extraFeatures = false` - else pkgs.knot-resolver.override { extraFeatures = true; }; + package = pkgs.knot-resolver; in { meta.maintainers = [ maintainers.vcunat /* upstream developer */ ]; @@ -92,7 +88,7 @@ in { default = []; example = [ "198.51.100.1:443" "[2001:db8::1]:443" "443" ]; description = '' - Addresses and ports on which kresd should provide DNS over HTTPS (see RFC 8484). + Addresses and ports on which kresd should provide DNS over HTTPS/2 (see RFC 8484). For detailed syntax see ListenStream in man systemd.socket. ''; }; diff --git a/nixos/modules/services/networking/mosquitto.nix b/nixos/modules/services/networking/mosquitto.nix index d2feb93e2b723..4a85b3956daeb 100644 --- a/nixos/modules/services/networking/mosquitto.nix +++ b/nixos/modules/services/networking/mosquitto.nix @@ -123,12 +123,33 @@ in ''; }; + passwordFile = mkOption { + type = with types; uniq (nullOr str); + example = "/path/to/file"; + default = null; + description = '' + Specifies the path to a file containing the + clear text password for the MQTT user. + ''; + }; + hashedPassword = mkOption { type = with types; uniq (nullOr str); default = null; description = '' Specifies the hashed password for the MQTT User. - <option>hashedPassword</option> overrides <option>password</option>. + To generate hashed password install <literal>mosquitto</literal> + package and use <literal>mosquitto_passwd</literal>. + ''; + }; + + hashedPasswordFile = mkOption { + type = with types; uniq (nullOr str); + example = "/path/to/file"; + default = null; + description = '' + Specifies the path to a file containing the + hashed password for the MQTT user. To generate hashed password install <literal>mosquitto</literal> package and use <literal>mosquitto_passwd</literal>. ''; @@ -190,6 +211,13 @@ in config = mkIf cfg.enable { + assertions = mapAttrsToList (name: cfg: { + assertion = length (filter (s: s != null) (with cfg; [ + password passwordFile hashedPassword hashedPasswordFile + ])) <= 1; + message = "Cannot set more than one password option"; + }) cfg.users; + systemd.services.mosquitto = { description = "Mosquitto MQTT Broker Daemon"; wantedBy = [ "multi-user.target" ]; @@ -210,7 +238,11 @@ in touch ${cfg.dataDir}/passwd '' + concatStringsSep "\n" ( mapAttrsToList (n: c: - if c.hashedPassword != null then + if c.hashedPasswordFile != null then + "echo '${n}:'$(cat '${c.hashedPasswordFile}') >> ${cfg.dataDir}/passwd" + else if c.passwordFile != null then + "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} $(cat '${c.passwordFile}')" + else if c.hashedPassword != null then "echo '${n}:${c.hashedPassword}' >> ${cfg.dataDir}/passwd" else optionalString (c.password != null) "${pkgs.mosquitto}/bin/mosquitto_passwd -b ${cfg.dataDir}/passwd ${n} '${c.password}'" diff --git a/nixos/modules/services/networking/mullvad-vpn.nix b/nixos/modules/services/networking/mullvad-vpn.nix index cc98414257ca7..d9e6e57c92cc6 100644 --- a/nixos/modules/services/networking/mullvad-vpn.nix +++ b/nixos/modules/services/networking/mullvad-vpn.nix @@ -29,9 +29,9 @@ with lib; # Needed for ping "/run/wrappers" ]; + startLimitBurst = 5; + startLimitIntervalSec = 20; serviceConfig = { - StartLimitBurst = 5; - StartLimitIntervalSec = 20; ExecStart = "${pkgs.mullvad-vpn}/bin/mullvad-daemon -v --disable-stdout-timestamps"; Restart = "always"; RestartSec = 1; diff --git a/nixos/modules/services/networking/murmur.nix b/nixos/modules/services/networking/murmur.nix index b384f436861db..c6e5649ec4798 100644 --- a/nixos/modules/services/networking/murmur.nix +++ b/nixos/modules/services/networking/murmur.nix @@ -278,6 +278,10 @@ in home = "/var/lib/murmur"; createHome = true; uid = config.ids.uids.murmur; + group = "murmur"; + }; + users.groups.murmur = { + gid = config.ids.gids.murmur; }; systemd.services.murmur = { @@ -300,6 +304,7 @@ in RuntimeDirectory = "murmur"; RuntimeDirectoryMode = "0700"; User = "murmur"; + Group = "murmur"; }; }; }; diff --git a/nixos/modules/services/networking/namecoind.nix b/nixos/modules/services/networking/namecoind.nix index 16f85df2e77c8..4966ed2cac8dc 100644 --- a/nixos/modules/services/networking/namecoind.nix +++ b/nixos/modules/services/networking/namecoind.nix @@ -165,6 +165,8 @@ in after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 120; + startLimitBurst = 5; serviceConfig = { User = "namecoin"; Group = "namecoin"; @@ -176,8 +178,6 @@ in TimeoutStopSec = "60s"; TimeoutStartSec = "2s"; Restart = "always"; - StartLimitInterval = "120s"; - StartLimitBurst = "5"; }; preStart = optionalString (cfg.wallet != "${dataDir}/wallet.dat") '' diff --git a/nixos/modules/services/networking/nar-serve.nix b/nixos/modules/services/networking/nar-serve.nix new file mode 100644 index 0000000000000..ddd42fa010737 --- /dev/null +++ b/nixos/modules/services/networking/nar-serve.nix @@ -0,0 +1,55 @@ +{ config, pkgs, lib, ... }: + +with lib; +let + cfg = config.services.nar-serve; +in +{ + meta = { + maintainers = [ maintainers.rizary ]; + }; + options = { + services.nar-serve = { + enable = mkEnableOption "Serve NAR file contents via HTTP"; + + port = mkOption { + type = types.int; + default = 8383; + description = '' + Port number where nar-serve will listen on. + ''; + }; + + cacheURL = mkOption { + type = types.str; + default = "https://cache.nixos.org/"; + description = '' + Binary cache URL to connect to. + + The URL format is compatible with the nix remote url style, such as: + - http://, https:// for binary caches via HTTP or HTTPS + - s3:// for binary caches stored in Amazon S3 + - gs:// for binary caches stored in Google Cloud Storage + ''; + }; + }; + }; + + config = mkIf cfg.enable { + systemd.services.nar-serve = { + description = "NAR server"; + after = [ "network.target" ]; + wantedBy = [ "multi-user.target" ]; + + environment.PORT = toString cfg.port; + environment.NAR_CACHE_URL = cfg.cacheURL; + + serviceConfig = { + Restart = "always"; + RestartSec = "5s"; + ExecStart = "${pkgs.nar-serve}/bin/nar-serve"; + DynamicUser = true; + }; + }; + }; +} diff --git a/nixos/modules/services/networking/nextdns.nix b/nixos/modules/services/networking/nextdns.nix index a633bff62ec7e..b070eeec894fe 100644 --- a/nixos/modules/services/networking/nextdns.nix +++ b/nixos/modules/services/networking/nextdns.nix @@ -28,9 +28,9 @@ in { environment = { SERVICE_RUN_MODE = "1"; }; + startLimitIntervalSec = 5; + startLimitBurst = 10; serviceConfig = { - StartLimitInterval = 5; - StartLimitBurst = 10; ExecStart = "${pkgs.nextdns}/bin/nextdns run ${escapeShellArgs config.services.nextdns.arguments}"; RestartSec = 120; LimitMEMLOCK = "infinity"; diff --git a/nixos/modules/services/networking/nix-store-gcs-proxy.nix b/nixos/modules/services/networking/nix-store-gcs-proxy.nix index 3f2ce5bca4da4..0012302db2e3c 100644 --- a/nixos/modules/services/networking/nix-store-gcs-proxy.nix +++ b/nixos/modules/services/networking/nix-store-gcs-proxy.nix @@ -42,9 +42,9 @@ in description = "A HTTP nix store that proxies requests to Google Storage"; wantedBy = ["multi-user.target"]; + startLimitIntervalSec = 10; serviceConfig = { RestartSec = 5; - StartLimitInterval = 10; ExecStart = '' ${pkgs.nix-store-gcs-proxy}/bin/nix-store-gcs-proxy \ --bucket-name ${cfg.bucketName} \ diff --git a/nixos/modules/services/networking/nsd.nix b/nixos/modules/services/networking/nsd.nix index 3ecbd06ee416b..f33c350a257a9 100644 --- a/nixos/modules/services/networking/nsd.nix +++ b/nixos/modules/services/networking/nsd.nix @@ -916,14 +916,14 @@ in after = [ "network.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitBurst = 4; + startLimitIntervalSec = 5 * 60; # 5 mins serviceConfig = { ExecStart = "${nsdPkg}/sbin/nsd -d -c ${nsdEnv}/nsd.conf"; StandardError = "null"; PIDFile = pidFile; Restart = "always"; RestartSec = "4s"; - StartLimitBurst = 4; - StartLimitInterval = "5min"; }; preStart = '' diff --git a/nixos/modules/services/networking/ntp/chrony.nix b/nixos/modules/services/networking/ntp/chrony.nix index 78de50583f348..e6fa48daf46cd 100644 --- a/nixos/modules/services/networking/ntp/chrony.nix +++ b/nixos/modules/services/networking/ntp/chrony.nix @@ -6,6 +6,7 @@ let cfg = config.services.chrony; stateDir = "/var/lib/chrony"; + driftFile = "${stateDir}/chrony.drift"; keyFile = "${stateDir}/chrony.keys"; configFile = pkgs.writeText "chrony.conf" '' @@ -16,7 +17,7 @@ let "initstepslew ${toString cfg.initstepslew.threshold} ${concatStringsSep " " cfg.servers}" } - driftfile ${stateDir}/chrony.drift + driftfile ${driftFile} keyfile ${keyFile} ${optionalString (!config.time.hardwareClockInLocalTime) "rtconutc"} @@ -95,6 +96,7 @@ in systemd.tmpfiles.rules = [ "d ${stateDir} 0755 chrony chrony - -" + "f ${driftFile} 0640 chrony chrony -" "f ${keyFile} 0640 chrony chrony -" ]; diff --git a/nixos/modules/services/networking/powerdns.nix b/nixos/modules/services/networking/powerdns.nix index ba05e15389f66..8cae61b835431 100644 --- a/nixos/modules/services/networking/powerdns.nix +++ b/nixos/modules/services/networking/powerdns.nix @@ -8,42 +8,40 @@ let in { options = { services.powerdns = { - enable = mkEnableOption "Powerdns domain name server"; + enable = mkEnableOption "PowerDNS domain name server"; extraConfig = mkOption { type = types.lines; default = "launch=bind"; description = '' - Extra lines to be added verbatim to pdns.conf. - Powerdns will chroot to /var/lib/powerdns. - So any file, powerdns is supposed to be read, - should be in /var/lib/powerdns and needs to specified - relative to the chroot. + PowerDNS configuration. Refer to + <link xlink:href="https://doc.powerdns.com/authoritative/settings.html"/> + for details on supported values. ''; }; }; }; - config = mkIf config.services.powerdns.enable { + config = mkIf cfg.enable { + + systemd.packages = [ pkgs.powerdns ]; + systemd.services.pdns = { - unitConfig.Documentation = "man:pdns_server(1) man:pdns_control(1)"; - description = "Powerdns name server"; wantedBy = [ "multi-user.target" ]; - after = ["network.target" "mysql.service" "postgresql.service" "openldap.service"]; + after = [ "network.target" "mysql.service" "postgresql.service" "openldap.service" ]; serviceConfig = { - Restart="on-failure"; - RestartSec="1"; - StartLimitInterval="0"; - PrivateDevices=true; - CapabilityBoundingSet="CAP_CHOWN CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID CAP_SYS_CHROOT"; - NoNewPrivileges=true; - ExecStartPre = "${pkgs.coreutils}/bin/mkdir -p /var/lib/powerdns"; - ExecStart = "${pkgs.powerdns}/bin/pdns_server --setuid=nobody --setgid=nogroup --chroot=/var/lib/powerdns --socket-dir=/ --daemon=no --guardian=no --disable-syslog --write-pid=no --config-dir=${configDir}"; - ProtectSystem="full"; - ProtectHome=true; - RestrictAddressFamilies="AF_UNIX AF_INET AF_INET6"; + ExecStart = [ "" "${pkgs.powerdns}/bin/pdns_server --config-dir=${configDir} --guardian=no --daemon=no --disable-syslog --log-timestamp=no --write-pid=no" ]; }; }; + + users.users.pdns = { + isSystemUser = true; + group = "pdns"; + description = "PowerDNS"; + }; + + users.groups.pdns = {}; + }; } diff --git a/nixos/modules/services/networking/prosody.nix b/nixos/modules/services/networking/prosody.nix index a6c1cb0f4797a..e7a7aa700be6e 100644 --- a/nixos/modules/services/networking/prosody.nix +++ b/nixos/modules/services/networking/prosody.nix @@ -261,7 +261,7 @@ let toLua = x: if builtins.isString x then ''"${x}"'' - else if builtins.isBool x then (if x == true then "true" else "false") + else if builtins.isBool x then boolToString x else if builtins.isInt x then toString x else if builtins.isList x then ''{ ${lib.concatStringsSep ", " (map (n: toLua n) x) } }'' else throw "Invalid Lua value"; diff --git a/nixos/modules/services/networking/ssh/sshd.nix b/nixos/modules/services/networking/ssh/sshd.nix index 5365b8b9b1072..1b745931c4fa8 100644 --- a/nixos/modules/services/networking/ssh/sshd.nix +++ b/nixos/modules/services/networking/ssh/sshd.nix @@ -269,6 +269,7 @@ in kexAlgorithms = mkOption { type = types.listOf types.str; default = [ + "curve25519-sha256" "curve25519-sha256@libssh.org" "diffie-hellman-group-exchange-sha256" ]; @@ -279,7 +280,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; @@ -300,7 +301,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; @@ -321,7 +322,7 @@ in Defaults to recommended settings from both <link xlink:href="https://stribika.github.io/2015/01/04/secure-secure-shell.html" /> and - <link xlink:href="https://wiki.mozilla.org/Security/Guidelines/OpenSSH#Modern_.28OpenSSH_6.7.2B.29" /> + <link xlink:href="https://infosec.mozilla.org/guidelines/openssh#modern-openssh-67" /> ''; }; diff --git a/nixos/modules/services/networking/sslh.nix b/nixos/modules/services/networking/sslh.nix index 0921febba668a..4c2740d201927 100644 --- a/nixos/modules/services/networking/sslh.nix +++ b/nixos/modules/services/networking/sslh.nix @@ -31,7 +31,7 @@ let { name: "openvpn"; host: "localhost"; port: "1194"; probe: "builtin"; }, { name: "xmpp"; host: "localhost"; port: "5222"; probe: "builtin"; }, { name: "http"; host: "localhost"; port: "80"; probe: "builtin"; }, - { name: "ssl"; host: "localhost"; port: "443"; probe: "builtin"; }, + { name: "tls"; host: "localhost"; port: "443"; probe: "builtin"; }, { name: "anyprot"; host: "localhost"; port: "443"; probe: "builtin"; } ); ''; diff --git a/nixos/modules/services/networking/stunnel.nix b/nixos/modules/services/networking/stunnel.nix index ab51bba2f6acf..fe1616f411f0a 100644 --- a/nixos/modules/services/networking/stunnel.nix +++ b/nixos/modules/services/networking/stunnel.nix @@ -16,8 +16,12 @@ let serverConfig = { options = { accept = mkOption { - type = types.int; - description = "On which port stunnel should listen for incoming TLS connections."; + type = types.either types.str types.int; + description = '' + On which [host:]port stunnel should listen for incoming TLS connections. + Note that unlike other softwares stunnel ipv6 address need no brackets, + so to listen on all IPv6 addresses on port 1234 one would use ':::1234'. + ''; }; connect = mkOption { @@ -129,7 +133,6 @@ in type = with types; attrsOf (submodule serverConfig); example = { fancyWebserver = { - enable = true; accept = 443; connect = 8080; cert = "/path/to/pem/file"; diff --git a/nixos/modules/services/networking/supybot.nix b/nixos/modules/services/networking/supybot.nix index dc9fb31ffd0bf..7a62e04ec7c4b 100644 --- a/nixos/modules/services/networking/supybot.nix +++ b/nixos/modules/services/networking/supybot.nix @@ -103,6 +103,8 @@ in rm -f '${cfg.stateDir}/supybot.cfg.bak' ''; + startLimitIntervalSec = 5 * 60; # 5 min + startLimitBurst = 1; serviceConfig = { ExecStart = "${pyEnv}/bin/supybot ${cfg.stateDir}/supybot.cfg"; PIDFile = "/run/supybot.pid"; @@ -110,8 +112,6 @@ in Group = "supybot"; UMask = "0007"; Restart = "on-abort"; - StartLimitInterval = "5m"; - StartLimitBurst = "1"; NoNewPrivileges = true; PrivateDevices = true; diff --git a/nixos/modules/services/networking/tailscale.nix b/nixos/modules/services/networking/tailscale.nix index 4d6aeb75ebd15..d6684d69e615d 100644 --- a/nixos/modules/services/networking/tailscale.nix +++ b/nixos/modules/services/networking/tailscale.nix @@ -17,33 +17,11 @@ in { }; config = mkIf cfg.enable { - systemd.services.tailscale = { - description = "Tailscale client daemon"; - - after = [ "network-pre.target" ]; - wants = [ "network-pre.target" ]; + environment.systemPackages = [ pkgs.tailscale ]; # for the CLI + systemd.packages = [ pkgs.tailscale ]; + systemd.services.tailscaled = { wantedBy = [ "multi-user.target" ]; - - unitConfig = { - StartLimitIntervalSec = 0; - StartLimitBurst = 0; - }; - - serviceConfig = { - ExecStart = - "${pkgs.tailscale}/bin/tailscaled --port ${toString cfg.port}"; - - RuntimeDirectory = "tailscale"; - RuntimeDirectoryMode = 755; - - StateDirectory = "tailscale"; - StateDirectoryMode = 750; - - CacheDirectory = "tailscale"; - CacheDirectoryMode = 750; - - Restart = "on-failure"; - }; + serviceConfig.Environment = "PORT=${toString cfg.port}"; }; }; } diff --git a/nixos/modules/services/networking/unbound.nix b/nixos/modules/services/networking/unbound.nix index baed83591e1ed..9a46fa3075fa1 100644 --- a/nixos/modules/services/networking/unbound.nix +++ b/nixos/modules/services/networking/unbound.nix @@ -1,9 +1,7 @@ { config, lib, pkgs, ... }: with lib; - let - cfg = config.services.unbound; stateDir = "/var/lib/unbound"; @@ -17,12 +15,12 @@ let forward = optionalString (any isLocalAddress cfg.forwardAddresses) '' do-not-query-localhost: no - '' + - optionalString (cfg.forwardAddresses != []) '' + '' + + optionalString (cfg.forwardAddresses != []) '' forward-zone: name: . - '' + - concatMapStringsSep "\n" (x: " forward-addr: ${x}") cfg.forwardAddresses; + '' + + concatMapStringsSep "\n" (x: " forward-addr: ${x}") cfg.forwardAddresses; rootTrustAnchorFile = "${stateDir}/root.key"; @@ -31,19 +29,25 @@ let confFile = pkgs.writeText "unbound.conf" '' server: + ip-freebind: yes directory: "${stateDir}" username: unbound - chroot: "${stateDir}" + chroot: "" pidfile: "" + # when running under systemd there is no need to daemonize + do-daemonize: no ${interfaces} ${access} ${trustAnchor} + ${lib.optionalString (cfg.localControlSocketPath != null) '' + remote-control: + control-enable: yes + control-interface: ${cfg.localControlSocketPath} + ''} ${cfg.extraConfig} ${forward} ''; - in - { ###### interface @@ -55,8 +59,8 @@ in package = mkOption { type = types.package; - default = pkgs.unbound; - defaultText = "pkgs.unbound"; + default = pkgs.unbound-with-systemd; + defaultText = "pkgs.unbound-with-systemd"; description = "The unbound package to use"; }; @@ -69,11 +73,14 @@ in interfaces = mkOption { default = [ "127.0.0.1" ] ++ optional config.networking.enableIPv6 "::1"; type = types.listOf types.str; - description = "What addresses the server should listen on."; + description = '' + What addresses the server should listen on. This supports the interface syntax documented in + <citerefentry><refentrytitle>unbound.conf</refentrytitle><manvolnum>8</manvolnum></citerefentry>. + ''; }; forwardAddresses = mkOption { - default = [ ]; + default = []; type = types.listOf types.str; description = "What servers to forward queries to."; }; @@ -84,6 +91,28 @@ in description = "Use and update root trust anchor for DNSSEC validation."; }; + localControlSocketPath = mkOption { + default = null; + # FIXME: What is the proper type here so users can specify strings, + # paths and null? + # My guess would be `types.nullOr (types.either types.str types.path)` + # but I haven't verified yet. + type = types.nullOr types.str; + example = "/run/unbound/unbound.ctl"; + description = '' + When not set to <literal>null</literal> this option defines the path + at which the unbound remote control socket should be created at. The + socket will be owned by the unbound user (<literal>unbound</literal>) + and group will be <literal>nogroup</literal>. + + Users that should be permitted to access the socket must be in the + <literal>unbound</literal> group. + + If this option is <literal>null</literal> remote control will not be + configured at all. Unbounds default values apply. + ''; + }; + extraConfig = mkOption { default = ""; type = types.lines; @@ -106,43 +135,85 @@ in users.users.unbound = { description = "unbound daemon user"; isSystemUser = true; + group = lib.mkIf (cfg.localControlSocketPath != null) (lib.mkDefault "unbound"); + }; + + # We need a group so that we can give users access to the configured + # control socket. Unbound allows access to the socket only to the unbound + # user and the primary group. + users.groups = lib.mkIf (cfg.localControlSocketPath != null) { + unbound = {}; }; networking.resolvconf.useLocalResolver = mkDefault true; + + environment.etc."unbound/unbound.conf".source = confFile; + systemd.services.unbound = { description = "Unbound recursive Domain Name Server"; after = [ "network.target" ]; before = [ "nss-lookup.target" ]; - wants = [ "nss-lookup.target" ]; - wantedBy = [ "multi-user.target" ]; - - preStart = '' - mkdir -m 0755 -p ${stateDir}/dev/ - cp ${confFile} ${stateDir}/unbound.conf - ${optionalString cfg.enableRootTrustAnchor '' - ${cfg.package}/bin/unbound-anchor -a ${rootTrustAnchorFile} || echo "Root anchor updated!" - chown unbound ${stateDir} ${rootTrustAnchorFile} - ''} - touch ${stateDir}/dev/random - ${pkgs.utillinux}/bin/mount --bind -n /dev/urandom ${stateDir}/dev/random + wantedBy = [ "multi-user.target" "nss-lookup.target" ]; + + preStart = lib.mkIf cfg.enableRootTrustAnchor '' + ${cfg.package}/bin/unbound-anchor -a ${rootTrustAnchorFile} || echo "Root anchor updated!" ''; - serviceConfig = { - ExecStart = "${cfg.package}/bin/unbound -d -c ${stateDir}/unbound.conf"; - ExecStopPost="${pkgs.utillinux}/bin/umount ${stateDir}/dev/random"; + restartTriggers = [ + confFile + ]; - ProtectSystem = true; - ProtectHome = true; + serviceConfig = { + ExecStart = "${cfg.package}/bin/unbound -p -d -c /etc/unbound/unbound.conf"; + ExecReload = "+/run/current-system/sw/bin/kill -HUP $MAINPID"; + + NotifyAccess = "main"; + Type = "notify"; + + # FIXME: Which of these do we actualy need, can we drop the chroot flag? + AmbientCapabilities = [ + "CAP_NET_BIND_SERVICE" + "CAP_NET_RAW" + "CAP_SETGID" + "CAP_SETUID" + "CAP_SYS_CHROOT" + "CAP_SYS_RESOURCE" + ]; + + User = "unbound"; + Group = lib.mkIf (cfg.localControlSocketPath != null) (lib.mkDefault "unbound"); + + MemoryDenyWriteExecute = true; + NoNewPrivileges = true; PrivateDevices = true; - Restart = "always"; - RestartSec = "5s"; + PrivateTmp = true; + ProtectHome = true; + ProtectControlGroups = true; + ProtectKernelModules = true; + ProtectSystem = "strict"; + RuntimeDirectory = "unbound"; + ConfigurationDirectory = "unbound"; + StateDirectory = "unbound"; + RestrictAddressFamilies = [ "AF_INET" "AF_INET6" "AF_UNIX" ]; + RestrictRealtime = true; + SystemCallArchitectures = "native"; + SystemCallFilter = [ + "~@clock" + "@cpu-emulation" + "@debug" + "@keyring" + "@module" + "mount" + "@obsolete" + "@resources" + ]; + RestrictNamespaces = true; + LockPersonality = true; + RestrictSUIDSGID = true; }; }; - # If networkmanager is enabled, ask it to interface with unbound. networking.networkmanager.dns = "unbound"; - }; - } diff --git a/nixos/modules/services/security/fail2ban.nix b/nixos/modules/services/security/fail2ban.nix index 3f84f9c2560cc..cf0d72d5c5319 100644 --- a/nixos/modules/services/security/fail2ban.nix +++ b/nixos/modules/services/security/fail2ban.nix @@ -282,12 +282,12 @@ in services.fail2ban.jails.DEFAULT = '' ${optionalString cfg.bantime-increment.enable '' # Bantime incremental - bantime.increment = ${if cfg.bantime-increment.enable then "true" else "false"} + bantime.increment = ${boolToString cfg.bantime-increment.enable} bantime.maxtime = ${cfg.bantime-increment.maxtime} bantime.factor = ${cfg.bantime-increment.factor} bantime.formula = ${cfg.bantime-increment.formula} bantime.multipliers = ${cfg.bantime-increment.multipliers} - bantime.overalljails = ${if cfg.bantime-increment.overalljails then "true" else "false"} + bantime.overalljails = ${boolToString cfg.bantime-increment.overalljails} ''} # Miscellaneous options ignoreip = 127.0.0.1/8 ${optionalString config.networking.enableIPv6 "::1"} ${concatStringsSep " " cfg.ignoreIP} diff --git a/nixos/modules/services/security/usbguard.nix b/nixos/modules/services/security/usbguard.nix index 16a90da52314e..71fd71a2cab2f 100644 --- a/nixos/modules/services/security/usbguard.nix +++ b/nixos/modules/services/security/usbguard.nix @@ -19,13 +19,13 @@ let PresentDevicePolicy=${cfg.presentDevicePolicy} PresentControllerPolicy=${cfg.presentControllerPolicy} InsertedDevicePolicy=${cfg.insertedDevicePolicy} - RestoreControllerDeviceState=${if cfg.restoreControllerDeviceState then "true" else "false"} + RestoreControllerDeviceState=${boolToString cfg.restoreControllerDeviceState} # this does not seem useful for endusers to change DeviceManagerBackend=uevent IPCAllowedUsers=${concatStringsSep " " cfg.IPCAllowedUsers} IPCAllowedGroups=${concatStringsSep " " cfg.IPCAllowedGroups} IPCAccessControlFiles=/var/lib/usbguard/IPCAccessControl.d/ - DeviceRulesWithPort=${if cfg.deviceRulesWithPort then "true" else "false"} + DeviceRulesWithPort=${boolToString cfg.deviceRulesWithPort} # HACK: that way audit logs still land in the journal AuditFilePath=/dev/null ''; diff --git a/nixos/modules/services/security/vault.nix b/nixos/modules/services/security/vault.nix index 6a8a3a93327eb..64622454b9dee 100644 --- a/nixos/modules/services/security/vault.nix +++ b/nixos/modules/services/security/vault.nix @@ -131,6 +131,8 @@ in restartIfChanged = false; # do not restart on "nixos-rebuild switch". It would seal the storage and disrupt the clients. + startLimitIntervalSec = 60; + startLimitBurst = 3; serviceConfig = { User = "vault"; Group = "vault"; @@ -145,8 +147,6 @@ in KillSignal = "SIGINT"; TimeoutStopSec = "30s"; Restart = "on-failure"; - StartLimitInterval = "60s"; - StartLimitBurst = 3; }; unitConfig.RequiresMountsFor = optional (cfg.storagePath != null) cfg.storagePath; diff --git a/nixos/modules/services/system/dbus.nix b/nixos/modules/services/system/dbus.nix index d9dd26f0f1892..f8d909a4a3cca 100644 --- a/nixos/modules/services/system/dbus.nix +++ b/nixos/modules/services/system/dbus.nix @@ -1,6 +1,6 @@ # D-Bus configuration and system bus daemon. -{ config, lib, pkgs, ... }: +{ config, lib, options, pkgs, ... }: with lib; @@ -18,13 +18,6 @@ let in { - - imports = [ - (mkRemovedOptionModule - [ "services" "dbus" "socketActivated" ] - "The user D-Bus session is now always socket activated and this option can safely be removed.") - ]; - ###### interface options = { @@ -57,12 +50,29 @@ in <filename><replaceable>pkg</replaceable>/share/dbus-1/services</filename> ''; }; + + socketActivated = mkOption { + type = types.nullOr types.bool; + default = null; + visible = false; + description = '' + Removed option, do not use. + ''; + }; }; }; ###### implementation config = mkIf cfg.enable { + warnings = optional (cfg.socketActivated != null) ( + let + files = showFiles options.services.dbus.socketActivated.files; + in + "The option 'services.dbus.socketActivated' in ${files} no longer has" + + " any effect and can be safely removed: the user D-Bus session is" + + " now always socket activated." + ); environment.systemPackages = [ pkgs.dbus.daemon pkgs.dbus ]; diff --git a/nixos/modules/services/torrent/transmission.nix b/nixos/modules/services/torrent/transmission.nix index 014a22bb5a8d6..717c18d367f01 100644 --- a/nixos/modules/services/torrent/transmission.nix +++ b/nixos/modules/services/torrent/transmission.nix @@ -197,7 +197,7 @@ in install -D -m 600 -o '${cfg.user}' -g '${cfg.group}' /dev/stdin \ '${cfg.home}/${settingsDir}/settings.json' '')]; - ExecStart="${pkgs.transmission}/bin/transmission-daemon -f"; + ExecStart="${pkgs.transmission}/bin/transmission-daemon -f -g ${cfg.home}/${settingsDir}"; ExecReload = "${pkgs.coreutils}/bin/kill -HUP $MAINPID"; User = cfg.user; Group = cfg.group; @@ -236,6 +236,7 @@ in # an AppArmor profile is provided to get a confinement based upon paths and rights. builtins.storeDir "/etc" + "/run" ] ++ optional (cfg.settings.script-torrent-done-enabled && cfg.settings.script-torrent-done-filename != "") @@ -408,6 +409,7 @@ in #r @{PROC}/@{pid}/environ, r @{PROC}/@{pid}/mounts, rwk /tmp/tr_session_id_*, + r /run/systemd/resolve/stub-resolv.conf, r ${pkgs.openssl.out}/etc/**, r ${config.systemd.services.transmission.environment.CURL_CA_BUNDLE}, diff --git a/nixos/modules/services/video/epgstation/generate b/nixos/modules/services/video/epgstation/generate deleted file mode 100755 index 2940768b6d2c8..0000000000000 --- a/nixos/modules/services/video/epgstation/generate +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env -S nix-build --no-out-link - -# Script to generate default streaming configurations for EPGStation. There's -# no need to run this script directly since generate.sh in the EPGStation -# package directory would run this script for you. -# -# Usage: ./generate | xargs cat > streaming.json - -{ pkgs ? (import ../../../../.. {}) }: - -let - sampleConfigPath = "${pkgs.epgstation.src}/config/config.sample.json"; - sampleConfig = builtins.fromJSON (builtins.readFile sampleConfigPath); - streamingConfig = { - inherit (sampleConfig) - mpegTsStreaming - mpegTsViewer - liveHLS - liveMP4 - liveWebM - recordedDownloader - recordedStreaming - recordedViewer - recordedHLS; - }; -in -pkgs.runCommand "streaming.json" { nativeBuildInputs = [ pkgs.jq ]; } '' - jq . <<<'${builtins.toJSON streamingConfig}' > $out -'' - -# vim:set ft=nix: diff --git a/nixos/modules/services/video/epgstation/streaming.json b/nixos/modules/services/video/epgstation/streaming.json index 37957f6cb6a22..8eb99cf85584b 100644 --- a/nixos/modules/services/video/epgstation/streaming.json +++ b/nixos/modules/services/video/epgstation/streaming.json @@ -1,119 +1,119 @@ { "liveHLS": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 48k -ac 2 -c:v libx264 -vf yadif,scale=-2:180 -b:v 100k -preset veryfast -maxrate 110k -bufsize 1000k -flags +loop-global_header %OUTPUT%", - "name": "180p" + "name": "180p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 17 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 48k -ac 2 -c:v libx264 -vf yadif,scale=-2:180 -b:v 100k -preset veryfast -maxrate 110k -bufsize 1000k -flags +loop-global_header %OUTPUT%" } ], "liveMP4": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1" } ], "liveWebM": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -b:a 192k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 -b:v 3000k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -b:a 192k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 -b:v 3000k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -b:a 128k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:480 -b:v 1500k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -b:a 128k -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:480 -b:v 1500k -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1" } ], "mpegTsStreaming": [ { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -y -f mpegts pipe:1", - "name": "720p" + "name": "720p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -y -f mpegts pipe:1" }, { - "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -y -f mpegts pipe:1", - "name": "480p" + "name": "480p", + "cmd": "%FFMPEG% -re -dual_mono_mode main -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -y -f mpegts pipe:1" }, { "name": "Original" } ], "mpegTsViewer": { - "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end", - "ios": "vlc-x-callback://x-callback-url/stream?url=http://ADDRESS" + "ios": "vlc-x-callback://x-callback-url/stream?url=http://ADDRESS", + "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end" }, "recordedDownloader": { - "android": "intent://ADDRESS#Intent;package=com.dv.adm;type=video;scheme=http;end", - "ios": "vlc-x-callback://x-callback-url/download?url=http://ADDRESS&filename=FILENAME" + "ios": "vlc-x-callback://x-callback-url/download?url=http://ADDRESS&filename=FILENAME", + "android": "intent://ADDRESS#Intent;package=com.dv.adm;type=video;scheme=http;end" }, - "recordedHLS": [ - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "720p" - }, - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%", - "name": "480p" - }, - { - "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_type fmp4 -hls_fmp4_init_filename stream%streamNum%-init.mp4 -hls_segment_filename stream%streamNum%-%09d.m4s -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx265 -vf yadif,scale=-2:480 -b:v 350k -preset veryfast -tag:v hvc1 %OUTPUT%", - "name": "480p(h265)" - } - ], "recordedStreaming": { - "mp4": [ + "webm": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", "name": "720p", - "vb": "3000k" + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", "name": "360p", - "vb": "1500k" + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", + "vb": "1500k", + "ab": "128k" } ], - "mpegTs": [ + "mp4": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", - "name": "720p (H.264)", - "vb": "3000k" + "name": "720p", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", - "name": "360p (H.264)", - "vb": "1500k" + "name": "360p", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -movflags frag_keyframe+empty_moov+faststart+default_base_moof -y -f mp4 pipe:1", + "vb": "1500k", + "ab": "128k" } ], - "webm": [ + "mpegTs": [ { - "ab": "192k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 3 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "720p", - "vb": "3000k" + "name": "720p (H.264)", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:720 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", + "vb": "3000k", + "ab": "192k" }, { - "ab": "128k", - "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 2 -c:a libvorbis -ar 48000 -ac 2 -c:v libvpx-vp9 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -deadline realtime -speed 4 -cpu-used -8 -y -f webm pipe:1", - "name": "360p", - "vb": "1500k" + "name": "360p (H.264)", + "cmd": "%FFMPEG% -dual_mono_mode main %RE% -i pipe:0 -sn -threads 0 -c:a aac -ar 48000 -ac 2 -c:v libx264 -vf yadif,scale=-2:360 %VB% %VBUFFER% %AB% %ABUFFER% -profile:v baseline -preset veryfast -tune fastdecode,zerolatency -y -f mpegts pipe:1", + "vb": "1500k", + "ab": "128k" } ] }, + "recordedHLS": [ + { + "name": "720p", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 192k -ac 2 -c:v libx264 -vf yadif,scale=-2:720 -b:v 3000k -preset veryfast -flags +loop-global_header %OUTPUT%" + }, + { + "name": "480p", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -threads 0 -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_filename %streamFileDir%/stream%streamNum%-%09d.ts -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx264 -vf yadif,scale=-2:480 -b:v 1500k -preset veryfast -flags +loop-global_header %OUTPUT%" + }, + { + "name": "480p(h265)", + "cmd": "%FFMPEG% -dual_mono_mode main -i %INPUT% -sn -map 0 -ignore_unknown -max_muxing_queue_size 1024 -f hls -hls_time 3 -hls_list_size 0 -hls_allow_cache 1 -hls_segment_type fmp4 -hls_fmp4_init_filename stream%streamNum%-init.mp4 -hls_segment_filename stream%streamNum%-%09d.m4s -c:a aac -ar 48000 -b:a 128k -ac 2 -c:v libx265 -vf yadif,scale=-2:480 -b:v 350k -preset veryfast -tag:v hvc1 %OUTPUT%" + } + ], "recordedViewer": { - "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end", - "ios": "infuse://x-callback-url/play?url=http://ADDRESS" + "ios": "infuse://x-callback-url/play?url=http://ADDRESS", + "android": "intent://ADDRESS#Intent;package=com.mxtech.videoplayer.ad;type=video;scheme=http;end" } } diff --git a/nixos/modules/services/web-apps/codimd.nix b/nixos/modules/services/web-apps/codimd.nix index c787c36b877c8..0fbc9ee820e66 100644 --- a/nixos/modules/services/web-apps/codimd.nix +++ b/nixos/modules/services/web-apps/codimd.nix @@ -6,8 +6,10 @@ let cfg = config.services.codimd; prettyJSON = conf: - pkgs.runCommand "codimd-config.json" { preferLocalBuild = true; } '' - echo '${builtins.toJSON conf}' | ${pkgs.jq}/bin/jq \ + pkgs.runCommandLocal "codimd-config.json" { + nativeBuildInputs = [ pkgs.jq ]; + } '' + echo '${builtins.toJSON conf}' | jq \ '{production:del(.[]|nulls)|del(.[][]?|nulls)}' > $out ''; in @@ -878,7 +880,6 @@ in }; }; - environmentFile = mkOption { type = with types; nullOr path; default = null; @@ -908,6 +909,14 @@ in <literal>CodiMD</literal> is running. ''; }; + + package = mkOption { + type = types.package; + default = pkgs.codimd; + description = '' + Package that provides CodiMD. + ''; + }; }; config = mkIf cfg.enable { @@ -938,7 +947,7 @@ in ''; serviceConfig = { WorkingDirectory = cfg.workDir; - ExecStart = "${pkgs.codimd}/bin/codimd"; + ExecStart = "${cfg.package}/bin/codimd"; EnvironmentFile = mkIf (cfg.environmentFile != null) [ cfg.environmentFile ]; Environment = [ "CMD_CONFIG_FILE=${cfg.workDir}/config.json" diff --git a/nixos/modules/services/web-apps/engelsystem.nix b/nixos/modules/services/web-apps/engelsystem.nix index 899582a203040..2e755ae9d5233 100644 --- a/nixos/modules/services/web-apps/engelsystem.nix +++ b/nixos/modules/services/web-apps/engelsystem.nix @@ -10,7 +10,7 @@ in { default = false; example = true; description = '' - Whether to enable engelsystem, an online tool for coordinating helpers + Whether to enable engelsystem, an online tool for coordinating volunteers and shifts on large events. ''; type = lib.types.bool; diff --git a/nixos/modules/services/web-apps/keycloak.nix b/nixos/modules/services/web-apps/keycloak.nix new file mode 100644 index 0000000000000..bbb0c8d048313 --- /dev/null +++ b/nixos/modules/services/web-apps/keycloak.nix @@ -0,0 +1,692 @@ +{ config, pkgs, lib, ... }: + +let + cfg = config.services.keycloak; +in +{ + options.services.keycloak = { + + enable = lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = '' + Whether to enable the Keycloak identity and access management + server. + ''; + }; + + bindAddress = lib.mkOption { + type = lib.types.str; + default = "\${jboss.bind.address:0.0.0.0}"; + example = "127.0.0.1"; + description = '' + On which address Keycloak should accept new connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + httpPort = lib.mkOption { + type = lib.types.str; + default = "\${jboss.http.port:80}"; + example = "8080"; + description = '' + On which port Keycloak should listen for new HTTP connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + httpsPort = lib.mkOption { + type = lib.types.str; + default = "\${jboss.https.port:443}"; + example = "8443"; + description = '' + On which port Keycloak should listen for new HTTPS connections. + + A special syntax can be used to allow command line Java system + properties to override the value: ''${property.name:value} + ''; + }; + + frontendUrl = lib.mkOption { + type = lib.types.str; + example = "keycloak.example.com/auth"; + description = '' + The public URL used as base for all frontend requests. Should + normally include a trailing <literal>/auth</literal>. + + See <link xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">the + Hostname section of the Keycloak server installation + manual</link> for more information. + ''; + }; + + forceBackendUrlToFrontendUrl = lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = '' + Whether Keycloak should force all requests to go through the + frontend URL configured in <xref + linkend="opt-services.keycloak.frontendUrl" />. By default, + Keycloak allows backend requests to instead use its local + hostname or IP address and may also advertise it to clients + through its OpenID Connect Discovery endpoint. + + See <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">the + Hostname section of the Keycloak server installation + manual</link> for more information. + ''; + }; + + certificatePrivateKeyBundle = lib.mkOption { + type = lib.types.nullOr lib.types.path; + default = null; + example = "/run/keys/ssl_cert"; + description = '' + The path to a PEM formatted bundle of the private key and + certificate to use for TLS connections. + + This should be a string, not a Nix path, since Nix paths are + copied into the world-readable Nix store. + ''; + }; + + databaseType = lib.mkOption { + type = lib.types.enum [ "mysql" "postgresql" ]; + default = "postgresql"; + example = "mysql"; + description = '' + The type of database Keycloak should connect to. + ''; + }; + + databaseHost = lib.mkOption { + type = lib.types.str; + default = "localhost"; + description = '' + Hostname of the database to connect to. + ''; + }; + + databasePort = + let + dbPorts = { + postgresql = 5432; + mysql = 3306; + }; + in + lib.mkOption { + type = lib.types.port; + default = dbPorts.${cfg.databaseType}; + description = '' + Port of the database to connect to. + ''; + }; + + databaseUseSSL = lib.mkOption { + type = lib.types.bool; + default = cfg.databaseHost != "localhost"; + description = '' + Whether the database connection should be secured by SSL / + TLS. + ''; + }; + + databaseCaCert = lib.mkOption { + type = lib.types.nullOr lib.types.path; + default = null; + description = '' + The SSL / TLS CA certificate that verifies the identity of the + database server. + + Required when PostgreSQL is used and SSL is turned on. + + For MySQL, if left at <literal>null</literal>, the default + Java keystore is used, which should suffice if the server + certificate is issued by an official CA. + ''; + }; + + databaseCreateLocally = lib.mkOption { + type = lib.types.bool; + default = true; + description = '' + Whether a database should be automatically created on the + local host. Set this to false if you plan on provisioning a + local database yourself. This has no effect if + services.keycloak.databaseHost is customized. + ''; + }; + + databaseUsername = lib.mkOption { + type = lib.types.str; + default = "keycloak"; + description = '' + Username to use when connecting to an external or manually + provisioned database; has no effect when a local database is + automatically provisioned. + ''; + }; + + databasePasswordFile = lib.mkOption { + type = lib.types.path; + example = "/run/keys/db_password"; + description = '' + File containing the database password. + + This should be a string, not a Nix path, since Nix paths are + copied into the world-readable Nix store. + ''; + }; + + package = lib.mkOption { + type = lib.types.package; + default = pkgs.keycloak; + description = '' + Keycloak package to use. + ''; + }; + + initialAdminPassword = lib.mkOption { + type = lib.types.str; + default = "changeme"; + description = '' + Initial password set for the <literal>admin</literal> + user. The password is not stored safely and should be changed + immediately in the admin panel. + ''; + }; + + extraConfig = lib.mkOption { + type = lib.types.attrs; + default = { }; + example = lib.literalExample '' + { + "subsystem=keycloak-server" = { + "spi=hostname" = { + "provider=default" = null; + "provider=fixed" = { + enabled = true; + properties.hostname = "keycloak.example.com"; + }; + default-provider = "fixed"; + }; + }; + } + ''; + description = '' + Additional Keycloak configuration options to set in + <literal>standalone.xml</literal>. + + Options are expressed as a Nix attribute set which matches the + structure of the jboss-cli configuration. The configuration is + effectively overlayed on top of the default configuration + shipped with Keycloak. To remove existing nodes and undefine + attributes from the default configuration, set them to + <literal>null</literal>. + + The example configuration does the equivalent of the following + script, which removes the hostname provider + <literal>default</literal>, adds the deprecated hostname + provider <literal>fixed</literal> and defines it the default: + + <programlisting> + /subsystem=keycloak-server/spi=hostname/provider=default:remove() + /subsystem=keycloak-server/spi=hostname/provider=fixed:add(enabled = true, properties = { hostname = "keycloak.example.com" }) + /subsystem=keycloak-server/spi=hostname:write-attribute(name=default-provider, value="fixed") + </programlisting> + + You can discover available options by using the <link + xlink:href="http://docs.wildfly.org/21/Admin_Guide.html#Command_Line_Interface">jboss-cli.sh</link> + program and by referring to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/index.html">Keycloak + Server Installation and Configuration Guide</link>. + ''; + }; + + }; + + config = + let + # We only want to create a database if we're actually going to connect to it. + databaseActuallyCreateLocally = cfg.databaseCreateLocally && cfg.databaseHost == "localhost"; + createLocalPostgreSQL = databaseActuallyCreateLocally && cfg.databaseType == "postgresql"; + createLocalMySQL = databaseActuallyCreateLocally && cfg.databaseType == "mysql"; + + mySqlCaKeystore = pkgs.runCommandNoCC "mysql-ca-keystore" {} '' + ${pkgs.jre}/bin/keytool -importcert -trustcacerts -alias MySQLCACert -file ${cfg.databaseCaCert} -keystore $out -storepass notsosecretpassword -noprompt + ''; + + keycloakConfig' = builtins.foldl' lib.recursiveUpdate { + "interface=public".inet-address = cfg.bindAddress; + "socket-binding-group=standard-sockets"."socket-binding=http".port = cfg.httpPort; + "subsystem=keycloak-server"."spi=hostname" = { + "provider=default" = { + enabled = true; + properties = { + inherit (cfg) frontendUrl forceBackendUrlToFrontendUrl; + }; + }; + }; + "subsystem=datasources"."data-source=KeycloakDS" = { + max-pool-size = "20"; + user-name = if databaseActuallyCreateLocally then "keycloak" else cfg.databaseUsername; + password = "@db-password@"; + }; + } [ + (lib.optionalAttrs (cfg.databaseType == "postgresql") { + "subsystem=datasources" = { + "jdbc-driver=postgresql" = { + driver-module-name = "org.postgresql"; + driver-name = "postgresql"; + driver-xa-datasource-class-name = "org.postgresql.xa.PGXADataSource"; + }; + "data-source=KeycloakDS" = { + connection-url = "jdbc:postgresql://${cfg.databaseHost}:${builtins.toString cfg.databasePort}/keycloak"; + driver-name = "postgresql"; + "connection-properties=ssl".value = lib.boolToString cfg.databaseUseSSL; + } // (lib.optionalAttrs (cfg.databaseCaCert != null) { + "connection-properties=sslrootcert".value = cfg.databaseCaCert; + "connection-properties=sslmode".value = "verify-ca"; + }); + }; + }) + (lib.optionalAttrs (cfg.databaseType == "mysql") { + "subsystem=datasources" = { + "jdbc-driver=mysql" = { + driver-module-name = "com.mysql"; + driver-name = "mysql"; + driver-class-name = "com.mysql.jdbc.Driver"; + }; + "data-source=KeycloakDS" = { + connection-url = "jdbc:mysql://${cfg.databaseHost}:${builtins.toString cfg.databasePort}/keycloak"; + driver-name = "mysql"; + "connection-properties=useSSL".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=requireSSL".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=verifyServerCertificate".value = lib.boolToString cfg.databaseUseSSL; + "connection-properties=characterEncoding".value = "UTF-8"; + valid-connection-checker-class-name = "org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"; + validate-on-match = true; + exception-sorter-class-name = "org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"; + } // (lib.optionalAttrs (cfg.databaseCaCert != null) { + "connection-properties=trustCertificateKeyStoreUrl".value = "file:${mySqlCaKeystore}"; + "connection-properties=trustCertificateKeyStorePassword".value = "notsosecretpassword"; + }); + }; + }) + (lib.optionalAttrs (cfg.certificatePrivateKeyBundle != null) { + "socket-binding-group=standard-sockets"."socket-binding=https".port = cfg.httpsPort; + "core-service=management"."security-realm=UndertowRealm"."server-identity=ssl" = { + keystore-path = "/run/keycloak/ssl/certificate_private_key_bundle.p12"; + keystore-password = "notsosecretpassword"; + }; + "subsystem=undertow"."server=default-server"."https-listener=https".security-realm = "UndertowRealm"; + }) + cfg.extraConfig + ]; + + + /* Produces a JBoss CLI script that creates paths and sets + attributes matching those described by `attrs`. When the + script is run, the existing settings are effectively overlayed + by those from `attrs`. Existing attributes can be unset by + defining them `null`. + + JBoss paths and attributes / maps are distinguished by their + name, where paths follow a `key=value` scheme. + + Example: + mkJbossScript { + "subsystem=keycloak-server"."spi=hostname" = { + "provider=fixed" = null; + "provider=default" = { + enabled = true; + properties = { + inherit frontendUrl; + forceBackendUrlToFrontendUrl = false; + }; + }; + }; + } + => '' + if (outcome != success) of /:read-resource() + /:add() + end-if + if (outcome != success) of /subsystem=keycloak-server:read-resource() + /subsystem=keycloak-server:add() + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname:read-resource() + /subsystem=keycloak-server/spi=hostname:add() + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname/provider=default:read-resource() + /subsystem=keycloak-server/spi=hostname/provider=default:add(enabled = true, properties = { forceBackendUrlToFrontendUrl = false, frontendUrl = "https://keycloak.example.com/auth" }) + end-if + if (result != true) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="enabled") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=enabled, value=true) + end-if + if (result != false) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.forceBackendUrlToFrontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.forceBackendUrlToFrontendUrl, value=false) + end-if + if (result != "https://keycloak.example.com/auth") of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.frontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.frontendUrl, value="https://keycloak.example.com/auth") + end-if + if (outcome != success) of /subsystem=keycloak-server/spi=hostname/provider=fixed:read-resource() + /subsystem=keycloak-server/spi=hostname/provider=fixed:remove() + end-if + '' + */ + mkJbossScript = attrs: + let + /* From a JBoss path and an attrset, produces a JBoss CLI + snippet that writes the corresponding attributes starting + at `path`. Recurses down into subattrsets as necessary, + producing the variable name from its full path in the + attrset. + + Example: + writeAttributes "/subsystem=keycloak-server/spi=hostname/provider=default" { + enabled = true; + properties = { + forceBackendUrlToFrontendUrl = false; + frontendUrl = "https://keycloak.example.com/auth"; + }; + } + => '' + if (result != true) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="enabled") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=enabled, value=true) + end-if + if (result != false) of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.forceBackendUrlToFrontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.forceBackendUrlToFrontendUrl, value=false) + end-if + if (result != "https://keycloak.example.com/auth") of /subsystem=keycloak-server/spi=hostname/provider=default:read-attribute(name="properties.frontendUrl") + /subsystem=keycloak-server/spi=hostname/provider=default:write-attribute(name=properties.frontendUrl, value="https://keycloak.example.com/auth") + end-if + '' + */ + writeAttributes = path: set: + let + # JBoss expressions like `${var}` need to be prefixed + # with `expression` to evaluate. + prefixExpression = string: + let + match = (builtins.match ''"\$\{.*}"'' string); + in + if match != null then + "expression " + string + else + string; + + writeAttribute = attribute: value: + let + type = builtins.typeOf value; + in + if type == "set" then + let + names = builtins.attrNames value; + in + builtins.foldl' (text: name: text + (writeAttribute "${attribute}.${name}" value.${name})) "" names + else if value == null then '' + if (outcome == success) of ${path}:read-attribute(name="${attribute}") + ${path}:undefine-attribute(name="${attribute}") + end-if + '' + else if builtins.elem type [ "string" "path" "bool" ] then + let + value' = if type == "bool" then lib.boolToString value else ''"${value}"''; + in '' + if (result != ${prefixExpression value'}) of ${path}:read-attribute(name="${attribute}") + ${path}:write-attribute(name=${attribute}, value=${value'}) + end-if + '' + else throw "Unsupported type '${type}' for path '${path}'!"; + in + lib.concatStrings + (lib.mapAttrsToList + (attribute: value: (writeAttribute attribute value)) + set); + + + /* Produces an argument list for the JBoss `add()` function, + which adds a JBoss path and takes as its arguments the + required subpaths and attributes. + + Example: + makeArgList { + enabled = true; + properties = { + forceBackendUrlToFrontendUrl = false; + frontendUrl = "https://keycloak.example.com/auth"; + }; + } + => '' + enabled = true, properties = { forceBackendUrlToFrontendUrl = false, frontendUrl = "https://keycloak.example.com/auth" } + '' + */ + makeArgList = set: + let + makeArg = attribute: value: + let + type = builtins.typeOf value; + in + if type == "set" then + "${attribute} = { " + (makeArgList value) + " }" + else if builtins.elem type [ "string" "path" "bool" ] then + "${attribute} = ${if type == "bool" then lib.boolToString value else ''"${value}"''}" + else if value == null then + "" + else + throw "Unsupported type '${type}' for attribute '${attribute}'!"; + in + lib.concatStringsSep ", " (lib.mapAttrsToList makeArg set); + + + /* Recurses into the `attrs` attrset, beginning at the path + resolved from `state.path ++ node`; if `node` is `null`, + starts from `state.path`. Only subattrsets that are JBoss + paths, i.e. follows the `key=value` format, are recursed + into - the rest are considered JBoss attributes / maps. + */ + recurse = state: node: + let + path = state.path ++ (lib.optional (node != null) node); + isPath = name: + let + value = lib.getAttrFromPath (path ++ [ name ]) attrs; + in + if (builtins.match ".*([=]).*" name) == [ "=" ] then + if builtins.isAttrs value || value == null then + true + else + throw "Parsing path '${lib.concatStringsSep "." (path ++ [ name ])}' failed: JBoss attributes cannot contain '='!" + else + false; + jbossPath = "/" + (lib.concatStringsSep "/" path); + nodeValue = lib.getAttrFromPath path attrs; + children = if !builtins.isAttrs nodeValue then {} else nodeValue; + subPaths = builtins.filter isPath (builtins.attrNames children); + jbossAttrs = lib.filterAttrs (name: _: !(isPath name)) children; + in + state // { + text = state.text + ( + if nodeValue != null then '' + if (outcome != success) of ${jbossPath}:read-resource() + ${jbossPath}:add(${makeArgList jbossAttrs}) + end-if + '' + (writeAttributes jbossPath jbossAttrs) + else '' + if (outcome == success) of ${jbossPath}:read-resource() + ${jbossPath}:remove() + end-if + '') + (builtins.foldl' recurse { text = ""; inherit path; } subPaths).text; + }; + in + (recurse { text = ""; path = []; } null).text; + + + jbossCliScript = pkgs.writeText "jboss-cli-script" (mkJbossScript keycloakConfig'); + + keycloakConfig = pkgs.runCommandNoCC "keycloak-config" {} '' + export JBOSS_BASE_DIR="$(pwd -P)"; + export JBOSS_MODULEPATH="${cfg.package}/modules"; + export JBOSS_LOG_DIR="$JBOSS_BASE_DIR/log"; + + cp -r ${cfg.package}/standalone/configuration . + chmod -R u+rwX ./configuration + + mkdir -p {deployments,ssl} + + "${cfg.package}/bin/standalone.sh"& + + attempt=1 + max_attempts=30 + while ! ${cfg.package}/bin/jboss-cli.sh --connect ':read-attribute(name=server-state)'; do + if [[ "$attempt" == "$max_attempts" ]]; then + echo "ERROR: Could not connect to Keycloak after $attempt attempts! Failing.." >&2 + exit 1 + fi + echo "Keycloak not fully started yet, retrying.. ($attempt/$max_attempts)" + sleep 1 + (( attempt++ )) + done + + ${cfg.package}/bin/jboss-cli.sh --connect --file=${jbossCliScript} --echo-command + + cp configuration/standalone.xml $out + ''; + in + lib.mkIf cfg.enable { + + assertions = [ + { + assertion = (cfg.databaseUseSSL && cfg.databaseType == "postgresql") -> (cfg.databaseCaCert != null); + message = ''A CA certificate must be specified (in 'services.keycloak.databaseCaCert') when PostgreSQL is used with SSL''; + } + ]; + + environment.systemPackages = [ cfg.package ]; + + systemd.services.keycloakPostgreSQLInit = lib.mkIf createLocalPostgreSQL { + after = [ "postgresql.service" ]; + before = [ "keycloak.service" ]; + bindsTo = [ "postgresql.service" ]; + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + User = "postgres"; + Group = "postgres"; + }; + script = '' + set -eu + + PSQL=${config.services.postgresql.package}/bin/psql + + db_password="$(<'${cfg.databasePasswordFile}')" + $PSQL -tAc "SELECT 1 FROM pg_roles WHERE rolname='keycloak'" | grep -q 1 || $PSQL -tAc "CREATE ROLE keycloak WITH LOGIN PASSWORD '$db_password' CREATEDB" + $PSQL -tAc "SELECT 1 FROM pg_database WHERE datname = 'keycloak'" | grep -q 1 || $PSQL -tAc 'CREATE DATABASE "keycloak" OWNER "keycloak"' + ''; + }; + + systemd.services.keycloakMySQLInit = lib.mkIf createLocalMySQL { + after = [ "mysql.service" ]; + before = [ "keycloak.service" ]; + bindsTo = [ "mysql.service" ]; + serviceConfig = { + Type = "oneshot"; + RemainAfterExit = true; + User = config.services.mysql.user; + Group = config.services.mysql.group; + }; + script = '' + set -eu + + db_password="$(<'${cfg.databasePasswordFile}')" + ( echo "CREATE USER IF NOT EXISTS 'keycloak'@'localhost' IDENTIFIED BY '$db_password';" + echo "CREATE DATABASE keycloak CHARACTER SET utf8 COLLATE utf8_unicode_ci;" + echo "GRANT ALL PRIVILEGES ON keycloak.* TO 'keycloak'@'localhost';" + ) | ${config.services.mysql.package}/bin/mysql -N + ''; + }; + + systemd.services.keycloak = + let + databaseServices = + if createLocalPostgreSQL then [ + "keycloakPostgreSQLInit.service" "postgresql.service" + ] + else if createLocalMySQL then [ + "keycloakMySQLInit.service" "mysql.service" + ] + else [ ]; + in { + after = databaseServices; + bindsTo = databaseServices; + wantedBy = [ "multi-user.target" ]; + environment = { + JBOSS_LOG_DIR = "/var/log/keycloak"; + JBOSS_BASE_DIR = "/run/keycloak"; + JBOSS_MODULEPATH = "${cfg.package}/modules"; + }; + serviceConfig = { + ExecStartPre = let + startPreFullPrivileges = '' + set -eu + + install -T -m 0400 -o keycloak -g keycloak '${cfg.databasePasswordFile}' /run/keycloak/secrets/db_password + '' + lib.optionalString (cfg.certificatePrivateKeyBundle != null) '' + install -T -m 0400 -o keycloak -g keycloak '${cfg.certificatePrivateKeyBundle}' /run/keycloak/secrets/ssl_cert_pk_bundle + ''; + startPre = '' + set -eu + + install -m 0600 ${cfg.package}/standalone/configuration/*.properties /run/keycloak/configuration + install -T -m 0600 ${keycloakConfig} /run/keycloak/configuration/standalone.xml + + db_password="$(</run/keycloak/secrets/db_password)" + ${pkgs.replace}/bin/replace-literal -fe '@db-password@' "$db_password" /run/keycloak/configuration/standalone.xml + + export JAVA_OPTS=-Djboss.server.config.user.dir=/run/keycloak/configuration + ${cfg.package}/bin/add-user-keycloak.sh -u admin -p '${cfg.initialAdminPassword}' + '' + lib.optionalString (cfg.certificatePrivateKeyBundle != null) '' + pushd /run/keycloak/ssl/ + cat /run/keycloak/secrets/ssl_cert_pk_bundle <(echo) /etc/ssl/certs/ca-certificates.crt > allcerts.pem + ${pkgs.openssl}/bin/openssl pkcs12 -export -in /run/keycloak/secrets/ssl_cert_pk_bundle -chain \ + -name "${cfg.frontendUrl}" -out certificate_private_key_bundle.p12 \ + -CAfile allcerts.pem -passout pass:notsosecretpassword + popd + ''; + in [ + "+${pkgs.writeShellScript "keycloak-start-pre-full-privileges" startPreFullPrivileges}" + "${pkgs.writeShellScript "keycloak-start-pre" startPre}" + ]; + ExecStart = "${cfg.package}/bin/standalone.sh"; + User = "keycloak"; + Group = "keycloak"; + DynamicUser = true; + RuntimeDirectory = map (p: "keycloak/" + p) [ + "secrets" + "configuration" + "deployments" + "data" + "ssl" + "log" + "tmp" + ]; + RuntimeDirectoryMode = 0700; + LogsDirectory = "keycloak"; + AmbientCapabilities = "CAP_NET_BIND_SERVICE"; + }; + }; + + services.postgresql.enable = lib.mkDefault createLocalPostgreSQL; + services.mysql.enable = lib.mkDefault createLocalMySQL; + services.mysql.package = lib.mkIf createLocalMySQL pkgs.mysql; + }; + + meta.doc = ./keycloak.xml; +} diff --git a/nixos/modules/services/web-apps/keycloak.xml b/nixos/modules/services/web-apps/keycloak.xml new file mode 100644 index 0000000000000..ca5e223eee467 --- /dev/null +++ b/nixos/modules/services/web-apps/keycloak.xml @@ -0,0 +1,205 @@ +<chapter xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + version="5.0" + xml:id="module-services-keycloak"> + <title>Keycloak</title> + <para> + <link xlink:href="https://www.keycloak.org/">Keycloak</link> is an + open source identity and access management server with support for + <link xlink:href="https://openid.net/connect/">OpenID + Connect</link>, <link xlink:href="https://oauth.net/2/">OAUTH + 2.0</link> and <link + xlink:href="https://en.wikipedia.org/wiki/SAML_2.0">SAML + 2.0</link>. + </para> + <section xml:id="module-services-keycloak-admin"> + <title>Administration</title> + <para> + An administrative user with the username + <literal>admin</literal> is automatically created in the + <literal>master</literal> realm. Its initial password can be + configured by setting <xref linkend="opt-services.keycloak.initialAdminPassword" /> + and defaults to <literal>changeme</literal>. The password is + not stored safely and should be changed immediately in the + admin panel. + </para> + + <para> + Refer to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_admin/index.html#admin-console">Admin + Console section of the Keycloak Server Administration Guide</link> for + information on how to administer your + <productname>Keycloak</productname> instance. + </para> + </section> + + <section xml:id="module-services-keycloak-database"> + <title>Database access</title> + <para> + <productname>Keycloak</productname> can be used with either + <productname>PostgreSQL</productname> or + <productname>MySQL</productname>. Which one is used can be + configured in <xref + linkend="opt-services.keycloak.databaseType" />. The selected + database will automatically be enabled and a database and role + created unless <xref + linkend="opt-services.keycloak.databaseHost" /> is changed from + its default of <literal>localhost</literal> or <xref + linkend="opt-services.keycloak.databaseCreateLocally" /> is set + to <literal>false</literal>. + </para> + + <para> + External database access can also be configured by setting + <xref linkend="opt-services.keycloak.databaseHost" />, <xref + linkend="opt-services.keycloak.databaseUsername" />, <xref + linkend="opt-services.keycloak.databaseUseSSL" /> and <xref + linkend="opt-services.keycloak.databaseCaCert" /> as + appropriate. Note that you need to manually create a database + called <literal>keycloak</literal> and allow the configured + database user full access to it. + </para> + + <para> + <xref linkend="opt-services.keycloak.databasePasswordFile" /> + must be set to the path to a file containing the password used + to log in to the database. If <xref linkend="opt-services.keycloak.databaseHost" /> + and <xref linkend="opt-services.keycloak.databaseCreateLocally" /> + are kept at their defaults, the database role + <literal>keycloak</literal> with that password is provisioned + on the local database instance. + </para> + + <warning> + <para> + The path should be provided as a string, not a Nix path, since Nix + paths are copied into the world readable Nix store. + </para> + </warning> + </section> + + <section xml:id="module-services-keycloak-frontendurl"> + <title>Frontend URL</title> + <para> + The frontend URL is used as base for all frontend requests and + must be configured through <xref linkend="opt-services.keycloak.frontendUrl" />. + It should normally include a trailing <literal>/auth</literal> + (the default web context). + </para> + + <para> + <xref linkend="opt-services.keycloak.forceBackendUrlToFrontendUrl" /> + determines whether Keycloak should force all requests to go + through the frontend URL. By default, + <productname>Keycloak</productname> allows backend requests to + instead use its local hostname or IP address and may also + advertise it to clients through its OpenID Connect Discovery + endpoint. + </para> + + <para> + See the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/#_hostname">Hostname + section of the Keycloak Server Installation and Configuration + Guide</link> for more information. + </para> + </section> + + <section xml:id="module-services-keycloak-tls"> + <title>Setting up TLS/SSL</title> + <para> + By default, <productname>Keycloak</productname> won't accept + unsecured HTTP connections originating from outside its local + network. + </para> + + <para> + For HTTPS support, a TLS certificate and private key is + required. They should be <link + xlink:href="https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail">PEM + formatted</link> and concatenated into a single file. The path + to this file should be configured in + <xref linkend="opt-services.keycloak.certificatePrivateKeyBundle" />. + </para> + + <warning> + <para> + The path should be provided as a string, not a Nix path, + since Nix paths are copied into the world readable Nix store. + </para> + </warning> + </section> + + <section xml:id="module-services-keycloak-extra-config"> + <title>Additional configuration</title> + <para> + Additional Keycloak configuration options, for which no + explicit <productname>NixOS</productname> options are provided, + can be set in <xref linkend="opt-services.keycloak.extraConfig" />. + </para> + + <para> + Options are expressed as a Nix attribute set which matches the + structure of the jboss-cli configuration. The configuration is + effectively overlayed on top of the default configuration + shipped with Keycloak. To remove existing nodes and undefine + attributes from the default configuration, set them to + <literal>null</literal>. + </para> + <para> + For example, the following script, which removes the hostname + provider <literal>default</literal>, adds the deprecated + hostname provider <literal>fixed</literal> and defines it the + default: + +<programlisting> +/subsystem=keycloak-server/spi=hostname/provider=default:remove() +/subsystem=keycloak-server/spi=hostname/provider=fixed:add(enabled = true, properties = { hostname = "keycloak.example.com" }) +/subsystem=keycloak-server/spi=hostname:write-attribute(name=default-provider, value="fixed") +</programlisting> + + would be expressed as + +<programlisting> +services.keycloak.extraConfig = { + "subsystem=keycloak-server" = { + "spi=hostname" = { + "provider=default" = null; + "provider=fixed" = { + enabled = true; + properties.hostname = "keycloak.example.com"; + }; + default-provider = "fixed"; + }; + }; +}; +</programlisting> + </para> + <para> + You can discover available options by using the <link + xlink:href="http://docs.wildfly.org/21/Admin_Guide.html#Command_Line_Interface">jboss-cli.sh</link> + program and by referring to the <link + xlink:href="https://www.keycloak.org/docs/latest/server_installation/index.html">Keycloak + Server Installation and Configuration Guide</link>. + </para> + </section> + + <section xml:id="module-services-keycloak-example-config"> + <title>Example configuration</title> + <para> + A basic configuration with some custom settings could look like this: +<programlisting> +services.keycloak = { + <link linkend="opt-services.keycloak.enable">enable</link> = true; + <link linkend="opt-services.keycloak.initialAdminPassword">initialAdminPassword</link> = "e6Wcm0RrtegMEHl"; # change on first login + <link linkend="opt-services.keycloak.frontendUrl">frontendUrl</link> = "https://keycloak.example.com/auth"; + <link linkend="opt-services.keycloak.forceBackendUrlToFrontendUrl">forceBackendUrlToFrontendUrl</link> = true; + <link linkend="opt-services.keycloak.certificatePrivateKeyBundle">certificatePrivateKeyBundle</link> = "/run/keys/ssl_cert"; + <link linkend="opt-services.keycloak.databasePasswordFile">databasePasswordFile</link> = "/run/keys/db_password"; +}; +</programlisting> + </para> + + </section> + </chapter> diff --git a/nixos/modules/services/web-apps/moinmoin.nix b/nixos/modules/services/web-apps/moinmoin.nix index dc7abce2a5cb5..3a876f75f4a48 100644 --- a/nixos/modules/services/web-apps/moinmoin.nix +++ b/nixos/modules/services/web-apps/moinmoin.nix @@ -224,6 +224,8 @@ in chmod -R u+w ${dataDir}/${wikiIdent}/underlay ''; + startLimitIntervalSec = 30; + serviceConfig = { User = user; Group = group; @@ -237,7 +239,6 @@ in Restart = "on-failure"; RestartSec = "2s"; - StartLimitIntervalSec = "30s"; StateDirectory = "moin/${wikiIdent}"; StateDirectoryMode = "0750"; diff --git a/nixos/modules/services/web-apps/nextcloud.nix b/nixos/modules/services/web-apps/nextcloud.nix index bad1bd9c767dd..53c2ab76fdfa2 100644 --- a/nixos/modules/services/web-apps/nextcloud.nix +++ b/nixos/modules/services/web-apps/nextcloud.nix @@ -549,9 +549,7 @@ in { }; "/" = { priority = 900; - extraConfig = if major < 20 - then "rewrite ^ /index.php;" - else "try_files $uri $uri/ /index.php$request_uri;"; + extraConfig = "rewrite ^ /index.php;"; }; "~ ^/store-apps" = { priority = 201; @@ -575,7 +573,7 @@ in { "~ ^/(?:\\.|autotest|occ|issue|indie|db_|console)".extraConfig = '' return 404; ''; - ${if major < 20 then "~ ^\\/(?:index|remote|public|cron|core\\/ajax\\/update|status|ocs\\/v[12]|updater\\/.+|oc[ms]-provider\\/.+|.+\\/richdocumentscode\\/proxy)\\.php(?:$|\\/)" else "~ \\.php(?:$|/)"} = { + "~ ^\\/(?:index|remote|public|cron|core\\/ajax\\/update|status|ocs\\/v[12]|updater\\/.+|oc[ms]-provider\\/.+|.+\\/richdocumentscode\\/proxy)\\.php(?:$|\\/)" = { priority = 500; extraConfig = '' include ${config.services.nginx.package}/conf/fastcgi.conf; diff --git a/nixos/modules/services/web-apps/shiori.nix b/nixos/modules/services/web-apps/shiori.nix index 1817a2039352d..9083ddfa2206b 100644 --- a/nixos/modules/services/web-apps/shiori.nix +++ b/nixos/modules/services/web-apps/shiori.nix @@ -37,11 +37,60 @@ in { description = "Shiori simple bookmarks manager"; wantedBy = [ "multi-user.target" ]; + environment.SHIORI_DIR = "/var/lib/shiori"; + serviceConfig = { ExecStart = "${package}/bin/shiori serve --address '${address}' --port '${toString port}'"; + DynamicUser = true; - Environment = "SHIORI_DIR=/var/lib/shiori"; StateDirectory = "shiori"; + # As the RootDirectory + RuntimeDirectory = "shiori"; + + # Security options + + BindReadOnlyPaths = [ + "/nix/store" + + # For SSL certificates, and the resolv.conf + "/etc" + ]; + + CapabilityBoundingSet = ""; + + DeviceAllow = ""; + + LockPersonality = true; + + MemoryDenyWriteExecute = true; + + PrivateDevices = true; + PrivateUsers = true; + + ProtectClock = true; + ProtectControlGroups = true; + ProtectHome = true; + ProtectHostname = true; + ProtectKernelLogs = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + + RestrictNamespaces = true; + RestrictAddressFamilies = [ "AF_INET" "AF_INET6" ]; + RestrictRealtime = true; + RestrictSUIDSGID = true; + + RootDirectory = "/run/shiori"; + + SystemCallArchitectures = "native"; + SystemCallErrorNumber = "EPERM"; + SystemCallFilter = [ + "@system-service" + + "~@chown" "~@cpu-emulation" "~@debug" "~@ipc" "~@keyring" "~@memlock" + "~@module" "~@obsolete" "~@privileged" "~@process" "~@raw-io" + "~@resources" "~@setuid" + ]; }; }; }; diff --git a/nixos/modules/services/web-servers/caddy.nix b/nixos/modules/services/web-servers/caddy.nix index 72bf9a9a1f448..297b732733927 100644 --- a/nixos/modules/services/web-servers/caddy.nix +++ b/nixos/modules/services/web-servers/caddy.nix @@ -24,6 +24,10 @@ let ${pkgs.jq}/bin/jq -s '.[0] * .[1]' ${adaptedConfig} ${tlsJSON} > $out ''; in { + imports = [ + (mkRemovedOptionModule [ "services" "caddy" "agree" ] "this option is no longer necessary for Caddy 2") + ]; + options.services.caddy = { enable = mkEnableOption "Caddy web server"; @@ -66,12 +70,6 @@ in { description = "Email address (for Let's Encrypt certificate)"; }; - agree = mkOption { - default = false; - type = types.bool; - description = "Agree to Let's Encrypt Subscriber Agreement"; - }; - dataDir = mkOption { default = "/var/lib/caddy"; type = types.path; @@ -103,6 +101,8 @@ in { after = [ "network-online.target" ]; wants = [ "network-online.target" ]; # systemd-networkd-wait-online.service wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 14400; + startLimitBurst = 10; serviceConfig = { ExecStart = "${cfg.package}/bin/caddy run --config ${configJSON}"; ExecReload = "${cfg.package}/bin/caddy reload --config ${configJSON}"; @@ -110,8 +110,6 @@ in { User = "caddy"; Group = "caddy"; Restart = "on-abnormal"; - StartLimitIntervalSec = 14400; - StartLimitBurst = 10; AmbientCapabilities = "cap_net_bind_service"; CapabilityBoundingSet = "cap_net_bind_service"; NoNewPrivileges = true; diff --git a/nixos/modules/services/web-servers/nginx/default.nix b/nixos/modules/services/web-servers/nginx/default.nix index 6d2ddea927e04..e9630d379f36c 100644 --- a/nixos/modules/services/web-servers/nginx/default.nix +++ b/nixos/modules/services/web-servers/nginx/default.nix @@ -86,7 +86,7 @@ let ''} ssl_protocols ${cfg.sslProtocols}; - ssl_ciphers ${cfg.sslCiphers}; + ${optionalString (cfg.sslCiphers != null) "ssl_ciphers ${cfg.sslCiphers};"} ${optionalString (cfg.sslDhparam != null) "ssl_dhparam ${cfg.sslDhparam};"} ${optionalString (cfg.recommendedTlsSettings) '' @@ -261,10 +261,7 @@ let ssl_trusted_certificate ${vhost.sslTrustedCertificate}; ''} - ${optionalString (vhost.basicAuthFile != null || vhost.basicAuth != {}) '' - auth_basic secured; - auth_basic_user_file ${if vhost.basicAuthFile != null then vhost.basicAuthFile else mkHtpasswd vhostName vhost.basicAuth}; - ''} + ${mkBasicAuth vhostName vhost} ${mkLocations vhost.locations} @@ -293,9 +290,19 @@ let ${optionalString (config.return != null) "return ${config.return};"} ${config.extraConfig} ${optionalString (config.proxyPass != null && cfg.recommendedProxySettings) "include ${recommendedProxyConfig};"} + ${mkBasicAuth "sublocation" config} } '') (sortProperties (mapAttrsToList (k: v: v // { location = k; }) locations))); - mkHtpasswd = vhostName: authDef: pkgs.writeText "${vhostName}.htpasswd" ( + + mkBasicAuth = name: zone: optionalString (zone.basicAuthFile != null || zone.basicAuth != {}) (let + auth_file = if zone.basicAuthFile != null + then zone.basicAuthFile + else mkHtpasswd name zone.basicAuth; + in '' + auth_basic secured; + auth_basic_user_file ${auth_file}; + ''); + mkHtpasswd = name: authDef: pkgs.writeText "${name}.htpasswd" ( concatStringsSep "\n" (mapAttrsToList (user: password: '' ${user}:{PLAIN}${password} '') authDef) @@ -487,7 +494,7 @@ in }; sslCiphers = mkOption { - type = types.str; + type = types.nullOr types.str; # Keep in sync with https://ssl-config.mozilla.org/#server=nginx&config=intermediate default = "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384"; description = "Ciphers to choose from when negotiating TLS handshakes."; @@ -693,6 +700,8 @@ in ${cfg.preStart} ${execCommand} -t ''; + + startLimitIntervalSec = 60; serviceConfig = { ExecStart = execCommand; ExecReload = [ @@ -701,7 +710,6 @@ in ]; Restart = "always"; RestartSec = "10s"; - StartLimitInterval = "1min"; # User and group User = cfg.user; Group = cfg.group; diff --git a/nixos/modules/services/web-servers/nginx/location-options.nix b/nixos/modules/services/web-servers/nginx/location-options.nix index 3d9e391ecf20f..f2fc072557256 100644 --- a/nixos/modules/services/web-servers/nginx/location-options.nix +++ b/nixos/modules/services/web-servers/nginx/location-options.nix @@ -9,6 +9,34 @@ with lib; { options = { + basicAuth = mkOption { + type = types.attrsOf types.str; + default = {}; + example = literalExample '' + { + user = "password"; + }; + ''; + description = '' + Basic Auth protection for a vhost. + + WARNING: This is implemented to store the password in plain text in the + Nix store. + ''; + }; + + basicAuthFile = mkOption { + type = types.nullOr types.path; + default = null; + description = '' + Basic Auth password file for a vhost. + Can be created via: <command>htpasswd -c <filename> <username></command>. + + WARNING: The generate file contains the users' passwords in a + non-cryptographically-securely hashed way. + ''; + }; + proxyPass = mkOption { type = types.nullOr types.str; default = null; diff --git a/nixos/modules/services/web-servers/nginx/vhost-options.nix b/nixos/modules/services/web-servers/nginx/vhost-options.nix index 455854e2a9655..cf211ea9a71b6 100644 --- a/nixos/modules/services/web-servers/nginx/vhost-options.nix +++ b/nixos/modules/services/web-servers/nginx/vhost-options.nix @@ -198,7 +198,7 @@ with lib; Basic Auth protection for a vhost. WARNING: This is implemented to store the password in plain text in the - nix store. + Nix store. ''; }; @@ -207,7 +207,10 @@ with lib; default = null; description = '' Basic Auth password file for a vhost. - Can be created via: <command>htpasswd -c <filename> <username></command> + Can be created via: <command>htpasswd -c <filename> <username></command>. + + WARNING: The generate file contains the users' passwords in a + non-cryptographically-securely hashed way. ''; }; diff --git a/nixos/modules/services/web-servers/traefik.nix b/nixos/modules/services/web-servers/traefik.nix index 4ab7307c3b671..3d29199dd4549 100644 --- a/nixos/modules/services/web-servers/traefik.nix +++ b/nixos/modules/services/web-servers/traefik.nix @@ -136,6 +136,8 @@ in { description = "Traefik web server"; after = [ "network-online.target" ]; wantedBy = [ "multi-user.target" ]; + startLimitIntervalSec = 86400; + startLimitBurst = 5; serviceConfig = { ExecStart = "${cfg.package}/bin/traefik --configfile=${staticConfigFile}"; @@ -143,8 +145,6 @@ in { User = "traefik"; Group = cfg.group; Restart = "on-failure"; - StartLimitInterval = 86400; - StartLimitBurst = 5; AmbientCapabilities = "cap_net_bind_service"; CapabilityBoundingSet = "cap_net_bind_service"; NoNewPrivileges = true; diff --git a/nixos/modules/services/x11/desktop-managers/gnome3.nix b/nixos/modules/services/x11/desktop-managers/gnome3.nix index acccbdb9950a3..68a65d77d62f0 100644 --- a/nixos/modules/services/x11/desktop-managers/gnome3.nix +++ b/nixos/modules/services/x11/desktop-managers/gnome3.nix @@ -17,6 +17,11 @@ let ''; }; + defaultFavoriteAppsOverride = '' + [org.gnome.shell] + favorite-apps=[ 'org.gnome.Geary.desktop', 'org.gnome.Calendar.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop' ] + ''; + nixos-gsettings-desktop-schemas = let defaultPackages = with pkgs; [ gsettings-desktop-schemas gnome3.gnome-shell ]; in @@ -42,8 +47,7 @@ let [org.gnome.desktop.screensaver] picture-uri='file://${pkgs.nixos-artwork.wallpapers.simple-dark-gray-bottom.gnomeFilePath}' - [org.gnome.shell] - favorite-apps=[ 'org.gnome.Epiphany.desktop', 'org.gnome.Geary.desktop', 'org.gnome.Music.desktop', 'org.gnome.Photos.desktop', 'org.gnome.Nautilus.desktop', 'org.gnome.Software.desktop' ] + ${cfg.favoriteAppsOverride} ${cfg.extraGSettingsOverrides} EOF @@ -69,6 +73,7 @@ in core-os-services.enable = mkEnableOption "essential services for GNOME3"; core-shell.enable = mkEnableOption "GNOME Shell services"; core-utilities.enable = mkEnableOption "GNOME core utilities"; + core-developer-tools.enable = mkEnableOption "GNOME core developer tools"; games.enable = mkEnableOption "GNOME games"; experimental-features = { @@ -123,6 +128,17 @@ in apply = list: list ++ [ pkgs.gnome3.gnome-shell pkgs.gnome3.gnome-shell-extensions ]; }; + favoriteAppsOverride = mkOption { + internal = true; # this is messy + default = defaultFavoriteAppsOverride; + type = types.lines; + example = literalExample '' + [org.gnome.shell] + favorite-apps=[ 'firefox.desktop', 'org.gnome.Calendar.desktop' ] + ''; + description = "List of desktop files to put as favorite apps into gnome-shell. These need to be installed somehow globally."; + }; + extraGSettingsOverrides = mkOption { default = ""; type = types.lines; @@ -179,6 +195,14 @@ in config = mkMerge [ (mkIf (cfg.enable || flashbackEnabled) { + # Seed our configuration into nixos-generate-config + system.nixos-generate-config.desktopConfiguration = '' + # Enable the GNOME 3 Desktop Environment. + services.xserver.enable = true; + services.xserver.displayManager.gdm.enable = true; + services.xserver.desktopManager.gnome3.enable = true; + ''; + services.gnome3.core-os-services.enable = true; services.gnome3.core-shell.enable = true; services.gnome3.core-utilities.enable = mkDefault true; @@ -207,6 +231,11 @@ in # If gnome3 is installed, build vim for gtk3 too. nixpkgs.config.vim.gui = "gtk3"; + + # Install gnome-software if flatpak is enabled + services.flatpak.guiPackages = [ + pkgs.gnome3.gnome-software + ]; }) (mkIf flashbackEnabled { @@ -294,6 +323,12 @@ in gnome-shell ]; + services.udev.packages = with pkgs.gnome3; [ + # Force enable KMS modifiers for devices that require them. + # https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1443 + mutter + ]; + services.avahi.enable = mkDefault true; xdg.portal.extraPortals = [ @@ -323,7 +358,7 @@ in source-sans-pro ]; - # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-36/elements/core/meta-gnome-core-shell.bst + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-38/elements/core/meta-gnome-core-shell.bst environment.systemPackages = with pkgs.gnome3; [ adwaita-icon-theme gnome-backgrounds @@ -368,7 +403,7 @@ in }; }) - # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-36/elements/core/meta-gnome-core-utilities.bst + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/blob/gnome-3-38/elements/core/meta-gnome-core-utilities.bst (mkIf serviceCfg.core-utilities.enable { environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ baobab @@ -387,17 +422,15 @@ in gnome-logs gnome-maps gnome-music - gnome-photos + pkgs.gnome-photos gnome-screenshot - gnome-software gnome-system-monitor gnome-weather nautilus + pkgs.gnome-connections simple-scan totem yelp - # Unsure if sensible for NixOS - /* gnome-boxes */ ] config.environment.gnome3.excludePackages); # Enable default program modules @@ -426,12 +459,43 @@ in (mkIf serviceCfg.games.enable { environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ - aisleriot atomix five-or-more four-in-a-row gnome-chess gnome-klotski - gnome-mahjongg gnome-mines gnome-nibbles gnome-robots gnome-sudoku - gnome-taquin gnome-tetravex hitori iagno lightsoff quadrapassel - swell-foop tali + aisleriot + atomix + five-or-more + four-in-a-row + gnome-chess + gnome-klotski + gnome-mahjongg + gnome-mines + gnome-nibbles + gnome-robots + gnome-sudoku + gnome-taquin + gnome-tetravex + hitori + iagno + lightsoff + quadrapassel + swell-foop + tali ] config.environment.gnome3.excludePackages); }) + + # Adapt from https://gitlab.gnome.org/GNOME/gnome-build-meta/-/blob/3.38.0/elements/core/meta-gnome-core-developer-tools.bst + (mkIf serviceCfg.core-developer-tools.enable { + environment.systemPackages = (with pkgs.gnome3; removePackagesByName [ + dconf-editor + devhelp + pkgs.gnome-builder + # boxes would make sense in this option, however + # it doesn't function well enough to be included + # in default configurations. + # https://github.com/NixOS/nixpkgs/issues/60908 + /* gnome-boxes */ + ] config.environment.gnome3.excludePackages); + + services.sysprof.enable = true; + }) ]; } diff --git a/nixos/modules/services/x11/desktop-managers/pantheon.nix b/nixos/modules/services/x11/desktop-managers/pantheon.nix index e67e216f90d90..cf02a71248b17 100644 --- a/nixos/modules/services/x11/desktop-managers/pantheon.nix +++ b/nixos/modules/services/x11/desktop-managers/pantheon.nix @@ -180,7 +180,6 @@ in gtk3.out hicolor-icon-theme lightlocker - nixos-artwork.wallpapers.simple-dark-gray onboard qgnomeplatform shared-mime-info diff --git a/nixos/modules/services/x11/desktop-managers/plasma5.nix b/nixos/modules/services/x11/desktop-managers/plasma5.nix index e48b5f23b58fe..8cc579af2ca09 100644 --- a/nixos/modules/services/x11/desktop-managers/plasma5.nix +++ b/nixos/modules/services/x11/desktop-managers/plasma5.nix @@ -184,6 +184,14 @@ in config = mkMerge [ (mkIf cfg.enable { + # Seed our configuration into nixos-generate-config + system.nixos-generate-config.desktopConfiguration = '' + # Enable the Plasma 5 Desktop Environment. + services.xserver.enable = true; + services.xserver.displayManager.sddm.enable = true; + services.xserver.desktopManager.plasma5.enable = true; + ''; + services.xserver.desktopManager.session = singleton { name = "plasma5"; bgSupport = true; @@ -192,7 +200,7 @@ in security.wrappers = { kcheckpass.source = "${lib.getBin plasma5.kscreenlocker}/libexec/kcheckpass"; - start_kdeinit.source = "${lib.getBin pkgs.kinit}/libexec/kf5/start_kdeinit"; + start_kdeinit.source = "${lib.getBin pkgs.kdeFrameworks.kinit}/libexec/kf5/start_kdeinit"; kwin_wayland = { source = "${lib.getBin plasma5.kwin}/bin/kwin_wayland"; capabilities = "cap_sys_nice+ep"; @@ -359,7 +367,7 @@ in security.pam.services.sddm.enableKwallet = true; xdg.portal.enable = true; - xdg.portal.extraPortals = [ pkgs.xdg-desktop-portal-kde ]; + xdg.portal.extraPortals = [ plasma5.xdg-desktop-portal-kde ]; # Update the start menu for each user that is currently logged in system.userActivationScripts.plasmaSetup = activationScript; diff --git a/nixos/modules/services/x11/display-managers/default.nix b/nixos/modules/services/x11/display-managers/default.nix index 568aeaceef75f..6945a241f92fc 100644 --- a/nixos/modules/services/x11/display-managers/default.nix +++ b/nixos/modules/services/x11/display-managers/default.nix @@ -474,6 +474,12 @@ in ) [dms wms] ); + + # Make xsessions and wayland sessions available in XDG_DATA_DIRS + # as some programs have behavior that depends on them being present + environment.sessionVariables.XDG_DATA_DIRS = [ + "${cfg.displayManager.sessionData.desktops}/share" + ]; }; imports = [ diff --git a/nixos/modules/services/x11/display-managers/gdm.nix b/nixos/modules/services/x11/display-managers/gdm.nix index eae70a57c781d..e3c5adb9737fc 100644 --- a/nixos/modules/services/x11/display-managers/gdm.nix +++ b/nixos/modules/services/x11/display-managers/gdm.nix @@ -160,7 +160,7 @@ in ]; # Otherwise GDM will not be able to start correctly and display Wayland sessions - systemd.packages = with pkgs.gnome3; [ gnome-session gnome-shell ]; + systemd.packages = with pkgs.gnome3; [ gdm gnome-session gnome-shell ]; environment.systemPackages = [ pkgs.gnome3.adwaita-icon-theme ]; systemd.services.display-manager.wants = [ @@ -264,7 +264,7 @@ in # presented and there's a little delay. environment.etc."gdm/custom.conf".text = '' [daemon] - WaylandEnable=${if cfg.gdm.wayland then "true" else "false"} + WaylandEnable=${boolToString cfg.gdm.wayland} ${optionalString cfg.autoLogin.enable ( if cfg.gdm.autoLogin.delay > 0 then '' TimedLoginEnable=true diff --git a/nixos/modules/services/x11/display-managers/lightdm.nix b/nixos/modules/services/x11/display-managers/lightdm.nix index 143785db0b4fc..2dafee9e36e3d 100644 --- a/nixos/modules/services/x11/display-managers/lightdm.nix +++ b/nixos/modules/services/x11/display-managers/lightdm.nix @@ -308,6 +308,7 @@ in home = "/var/lib/lightdm"; group = "lightdm"; uid = config.ids.uids.lightdm; + shell = pkgs.bash; }; systemd.tmpfiles.rules = [ diff --git a/nixos/modules/services/x11/display-managers/sddm.nix b/nixos/modules/services/x11/display-managers/sddm.nix index e63bb2e445396..a39bb55b38c40 100644 --- a/nixos/modules/services/x11/display-managers/sddm.nix +++ b/nixos/modules/services/x11/display-managers/sddm.nix @@ -9,7 +9,12 @@ let cfg = dmcfg.sddm; xEnv = config.systemd.services.display-manager.environment; - inherit (pkgs) sddm; + sddm = if config.services.xserver.desktopManager.lxqt.enable then + # TODO: Move lxqt to libsForQt515 + pkgs.libsForQt514.sddm + else + pkgs.libsForQt5.sddm + ; xserverWrapper = pkgs.writeScript "xserver-wrapper" '' #!/bin/sh @@ -55,10 +60,10 @@ let XauthPath=${pkgs.xorg.xauth}/bin/xauth DisplayCommand=${Xsetup} DisplayStopCommand=${Xstop} - EnableHidpi=${if cfg.enableHidpi then "true" else "false"} + EnableHidpi=${boolToString cfg.enableHidpi} [Wayland] - EnableHidpi=${if cfg.enableHidpi then "true" else "false"} + EnableHidpi=${boolToString cfg.enableHidpi} SessionDir=${dmcfg.sessionData.desktops}/share/wayland-sessions ${optionalString dmcfg.autoLogin.enable '' diff --git a/nixos/modules/services/x11/redshift.nix b/nixos/modules/services/x11/redshift.nix index 21b0b33553acc..60d80a28762b0 100644 --- a/nixos/modules/services/x11/redshift.nix +++ b/nixos/modules/services/x11/redshift.nix @@ -82,6 +82,15 @@ in { ''; }; + executable = mkOption { + type = types.str; + default = "/bin/redshift"; + example = "/bin/redshift-gtk"; + description = '' + Redshift executable to use within the package. + ''; + }; + extraOptions = mkOption { type = types.listOf types.str; default = []; @@ -114,7 +123,7 @@ in { partOf = [ "graphical-session.target" ]; serviceConfig = { ExecStart = '' - ${cfg.package}/bin/redshift \ + ${cfg.package}${cfg.executable} \ -l ${providerString} \ -t ${toString cfg.temperature.day}:${toString cfg.temperature.night} \ -b ${toString cfg.brightness.day}:${toString cfg.brightness.night} \ diff --git a/nixos/modules/services/x11/xserver.nix b/nixos/modules/services/x11/xserver.nix index 55d3e742ef7cd..9e971671c474e 100644 --- a/nixos/modules/services/x11/xserver.nix +++ b/nixos/modules/services/x11/xserver.nix @@ -678,14 +678,14 @@ in script = "${cfg.displayManager.job.execCmd}"; + # Stop restarting if the display manager stops (crashes) 2 times + # in one minute. Starting X typically takes 3-4s. + startLimitIntervalSec = 30; + startLimitBurst = 3; serviceConfig = { Restart = "always"; RestartSec = "200ms"; SyslogIdentifier = "display-manager"; - # Stop restarting if the display manager stops (crashes) 2 times - # in one minute. Starting X typically takes 3-4s. - StartLimitInterval = "30s"; - StartLimitBurst = "3"; }; }; diff --git a/nixos/modules/system/activation/activation-script.nix b/nixos/modules/system/activation/activation-script.nix index ddfd1af4a3190..18c77948cb90b 100644 --- a/nixos/modules/system/activation/activation-script.nix +++ b/nixos/modules/system/activation/activation-script.nix @@ -28,6 +28,20 @@ let utillinux # needed for mount and mountpoint ]; + scriptType = with types; + let scriptOptions = + { deps = mkOption + { type = types.listOf types.str; + default = [ ]; + description = "List of dependencies. The script will run after these."; + }; + text = mkOption + { type = types.lines; + description = "The content of the script."; + }; + }; + in either str (submodule { options = scriptOptions; }); + in { @@ -40,16 +54,14 @@ in default = {}; example = literalExample '' - { stdio = { - text = ''' - # Needed by some programs. - ln -sfn /proc/self/fd /dev/fd - ln -sfn /proc/self/fd/0 /dev/stdin - ln -sfn /proc/self/fd/1 /dev/stdout - ln -sfn /proc/self/fd/2 /dev/stderr - '''; - deps = []; - }; + { stdio.text = + ''' + # Needed by some programs. + ln -sfn /proc/self/fd /dev/fd + ln -sfn /proc/self/fd/0 /dev/stdin + ln -sfn /proc/self/fd/1 /dev/stdout + ln -sfn /proc/self/fd/2 /dev/stderr + '''; } ''; @@ -62,7 +74,7 @@ in idempotent and fast. ''; - type = types.attrsOf types.unspecified; # FIXME + type = types.attrsOf scriptType; apply = set: { script = @@ -125,7 +137,7 @@ in idempotent and fast. ''; - type = types.attrsOf types.unspecified; + type = with types; attrsOf scriptType; apply = set: { script = '' diff --git a/nixos/modules/system/boot/kernel.nix b/nixos/modules/system/boot/kernel.nix index 43871f439f7f3..ed7226331d70e 100644 --- a/nixos/modules/system/boot/kernel.nix +++ b/nixos/modules/system/boot/kernel.nix @@ -227,7 +227,7 @@ in "xhci_pci" "usbhid" "hid_generic" "hid_lenovo" "hid_apple" "hid_roccat" - "hid_logitech_hidpp" "hid_logitech_dj" + "hid_logitech_hidpp" "hid_logitech_dj" "hid_microsoft" ] ++ optionals (pkgs.stdenv.isi686 || pkgs.stdenv.isx86_64) [ # Misc. x86 keyboard stuff. diff --git a/nixos/modules/system/boot/luksroot.nix b/nixos/modules/system/boot/luksroot.nix index 88190e8200b16..8dd2ea20519a1 100644 --- a/nixos/modules/system/boot/luksroot.nix +++ b/nixos/modules/system/boot/luksroot.nix @@ -404,7 +404,7 @@ let echo "Please move your mouse to create needed randomness." ''} echo "Waiting for your FIDO2 device..." - fido2luks -i open ${device} ${name} ${fido2.credential} --await-dev ${toString fido2.gracePeriod} --salt string:$passphrase + fido2luks open ${device} ${name} ${fido2.credential} --await-dev ${toString fido2.gracePeriod} --salt string:$passphrase if [ $? -ne 0 ]; then echo "No FIDO2 key found, falling back to normal open procedure" open_normally diff --git a/nixos/modules/system/boot/networkd.nix b/nixos/modules/system/boot/networkd.nix index 47689b2a4700a..3b01bc00bafa2 100644 --- a/nixos/modules/system/boot/networkd.nix +++ b/nixos/modules/system/boot/networkd.nix @@ -648,11 +648,13 @@ let "RapidCommit" "ForceDHCPv6PDOtherInformation" "PrefixDelegationHint" + "RouteMetric" ]) (assertValueOneOf "UseDNS" boolValues) (assertValueOneOf "UseNTP" boolValues) (assertValueOneOf "RapidCommit" boolValues) (assertValueOneOf "ForceDHCPv6PDOtherInformation" boolValues) + (assertInt "RouteMetric") ]; sectionDHCPServer = checkUnitConfig "DHCPServer" [ diff --git a/nixos/modules/system/boot/pbkdf2-sha512.c b/nixos/modules/system/boot/pbkdf2-sha512.c index b40c383ac0233..67e989957ba6c 100644 --- a/nixos/modules/system/boot/pbkdf2-sha512.c +++ b/nixos/modules/system/boot/pbkdf2-sha512.c @@ -35,4 +35,4 @@ int main(int argc, char** argv) fwrite(key, 1, key_length, stdout); return 0; -} \ No newline at end of file +} diff --git a/nixos/modules/system/boot/plymouth.nix b/nixos/modules/system/boot/plymouth.nix index 55e5b07ed615d..ddf5ef8a0a6af 100644 --- a/nixos/modules/system/boot/plymouth.nix +++ b/nixos/modules/system/boot/plymouth.nix @@ -9,7 +9,7 @@ let cfg = config.boot.plymouth; - nixosBreezePlymouth = pkgs.breeze-plymouth.override { + nixosBreezePlymouth = pkgs.plasma5.breeze-plymouth.override { logoFile = cfg.logo; logoName = "nixos"; osName = "NixOS"; diff --git a/nixos/modules/system/boot/resolved.nix b/nixos/modules/system/boot/resolved.nix index b024f9cf5ee93..84bc9b78076cf 100644 --- a/nixos/modules/system/boot/resolved.nix +++ b/nixos/modules/system/boot/resolved.nix @@ -136,7 +136,7 @@ in } ]; - users.users.resolved.group = "systemd-resolve"; + users.users.systemd-resolve.group = "systemd-resolve"; # add resolve to nss hosts database if enabled and nscd enabled # system.nssModules is configured in nixos/modules/system/boot/systemd.nix diff --git a/nixos/modules/system/boot/stage-1-init.sh b/nixos/modules/system/boot/stage-1-init.sh index f7c2940049e56..e5cafe237cd53 100644 --- a/nixos/modules/system/boot/stage-1-init.sh +++ b/nixos/modules/system/boot/stage-1-init.sh @@ -120,7 +120,7 @@ eval "exec $logOutFd>&1 $logErrFd>&2" if test -w /dev/kmsg; then tee -i < /tmp/stage-1-init.log.fifo /proc/self/fd/"$logOutFd" | while read -r line; do if test -n "$line"; then - echo "<7>stage-1-init: $line" > /dev/kmsg + echo "<7>stage-1-init: [$(date)] $line" > /dev/kmsg fi done & else @@ -356,6 +356,7 @@ mountFS() { case $options in *x-nixos.autoresize*) if [ "$fsType" = ext2 -o "$fsType" = ext3 -o "$fsType" = ext4 ]; then + modprobe "$fsType" echo "resizing $device..." e2fsck -fp "$device" resize2fs "$device" diff --git a/nixos/modules/system/boot/systemd-unit-options.nix b/nixos/modules/system/boot/systemd-unit-options.nix index 5addc6f9ca44d..4154389b2ce5f 100644 --- a/nixos/modules/system/boot/systemd-unit-options.nix +++ b/nixos/modules/system/boot/systemd-unit-options.nix @@ -210,12 +210,21 @@ in rec { ''; }; + startLimitBurst = mkOption { + type = types.int; + description = '' + Configure unit start rate limiting. Units which are started + more than startLimitBurst times within an interval time + interval are not permitted to start any more. + ''; + }; + startLimitIntervalSec = mkOption { type = types.int; description = '' Configure unit start rate limiting. Units which are started - more than burst times within an interval time interval are - not permitted to start any more. + more than startLimitBurst times within an interval time + interval are not permitted to start any more. ''; }; @@ -245,8 +254,7 @@ in rec { serviceConfig = mkOption { default = {}; example = - { StartLimitInterval = 10; - RestartSec = 5; + { RestartSec = 5; }; type = types.addCheck (types.attrsOf unitOption) checkService; description = '' diff --git a/nixos/modules/system/boot/systemd.nix b/nixos/modules/system/boot/systemd.nix index 74d6957678f56..c22264b3e92dd 100644 --- a/nixos/modules/system/boot/systemd.nix +++ b/nixos/modules/system/boot/systemd.nix @@ -243,6 +243,8 @@ let OnFailure = toString config.onFailure; } // optionalAttrs (options.startLimitIntervalSec.isDefined) { StartLimitIntervalSec = toString config.startLimitIntervalSec; + } // optionalAttrs (options.startLimitBurst.isDefined) { + StartLimitBurst = toString config.startLimitBurst; }; }; }; @@ -884,14 +886,25 @@ in config = { - warnings = concatLists (mapAttrsToList (name: service: - let - type = service.serviceConfig.Type or ""; - restart = service.serviceConfig.Restart or "no"; - in optional - (type == "oneshot" && (restart == "always" || restart == "on-success")) - "Service '${name}.service' with 'Type=oneshot' cannot have 'Restart=always' or 'Restart=on-success'") - cfg.services); + warnings = concatLists ( + mapAttrsToList + (name: service: + let + type = service.serviceConfig.Type or ""; + restart = service.serviceConfig.Restart or "no"; + hasDeprecated = builtins.hasAttr "StartLimitInterval" service.serviceConfig; + in + concatLists [ + (optional (type == "oneshot" && (restart == "always" || restart == "on-success")) + "Service '${name}.service' with 'Type=oneshot' cannot have 'Restart=always' or 'Restart=on-success'" + ) + (optional hasDeprecated + "Service '${name}.service' uses the attribute 'StartLimitInterval' in the Service section, which is deprecated. See https://github.com/NixOS/nixpkgs/issues/45786." + ) + ] + ) + cfg.services + ); system.build.units = cfg.units; diff --git a/nixos/modules/tasks/filesystems/zfs.nix b/nixos/modules/tasks/filesystems/zfs.nix index 9ca7c6fb3431b..7b6c2277741b2 100644 --- a/nixos/modules/tasks/filesystems/zfs.nix +++ b/nixos/modules/tasks/filesystems/zfs.nix @@ -175,14 +175,10 @@ in forceImportAll = mkOption { type = types.bool; - default = true; + default = false; description = '' Forcibly import all ZFS pool(s). - This is enabled by default for backwards compatibility purposes, but it is highly - recommended to disable this option, as it bypasses some of the safeguards ZFS uses - to protect your ZFS pools. - If you set this option to <literal>false</literal> and NixOS subsequently fails to import your non-root ZFS pool(s), you should manually import each pool with "zpool import -f <pool-name>", and then reboot. You should only need to do @@ -507,6 +503,7 @@ in Type = "oneshot"; RemainAfterExit = true; }; + environment.ZFS_FORCE = optionalString cfgZfs.forceImportAll "-f"; script = (importLib { # See comments at importLib definition. zpoolCmd="${packages.zfsUser}/sbin/zpool"; diff --git a/nixos/modules/tasks/lvm.nix b/nixos/modules/tasks/lvm.nix index 2c3cc4c5467dc..98a0e2ddef900 100644 --- a/nixos/modules/tasks/lvm.nix +++ b/nixos/modules/tasks/lvm.nix @@ -21,6 +21,10 @@ in { }; config = mkMerge [ + ({ + # minimal configuration file to make lvmconfig/lvm2-activation-generator happy + environment.etc."lvm/lvm.conf".text = "config {}"; + }) (mkIf (!config.boot.isContainer) { systemd.tmpfiles.packages = [ cfg.package.out ]; environment.systemPackages = [ cfg.package ]; diff --git a/nixos/modules/tasks/network-interfaces.nix b/nixos/modules/tasks/network-interfaces.nix index e5bd577536833..53c54c2e3980e 100644 --- a/nixos/modules/tasks/network-interfaces.nix +++ b/nixos/modules/tasks/network-interfaces.nix @@ -1062,7 +1062,6 @@ in ]; boot.kernelModules = [ ] - ++ optional cfg.enableIPv6 "ipv6" ++ optional hasVirtuals "tun" ++ optional hasSits "sit" ++ optional hasBonds "bonding"; diff --git a/nixos/modules/testing/test-instrumentation.nix b/nixos/modules/testing/test-instrumentation.nix index dbbcb0bed5b02..be5fa88b8ade1 100644 --- a/nixos/modules/testing/test-instrumentation.nix +++ b/nixos/modules/testing/test-instrumentation.nix @@ -45,13 +45,22 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; }; systemd.services."serial-getty@${qemuSerialDevice}".enable = false; systemd.services."serial-getty@hvc0".enable = false; - # Only use a serial console, no TTY. - # NOTE: optionalAttrs - # test-instrumentation.nix appears to be used without qemu-vm.nix, so - # we avoid defining consoles if not possible. - # TODO: refactor such that test-instrumentation can import qemu-vm - # or declare virtualisation.qemu.console option in a module that's always imported - virtualisation = lib.optionalAttrs (options ? virtualisation.qemu.consoles) { qemu.consoles = [ qemuSerialDevice ]; }; + # Only set these settings when the options exist. Some tests (e.g. those + # that do not specify any nodes, or an empty attr set as nodes) will not + # have the QEMU module loaded and thuse these options can't and should not + # be set. + virtualisation = lib.optionalAttrs (options ? virtualisation.qemu) { + qemu = { + # Only use a serial console, no TTY. + # NOTE: optionalAttrs + # test-instrumentation.nix appears to be used without qemu-vm.nix, so + # we avoid defining consoles if not possible. + # TODO: refactor such that test-instrumentation can import qemu-vm + # or declare virtualisation.qemu.console option in a module that's always imported + consoles = [ qemuSerialDevice ]; + package = lib.mkDefault pkgs.qemu_test; + }; + }; boot.initrd.preDeviceCommands = '' diff --git a/nixos/modules/virtualisation/amazon-image.nix b/nixos/modules/virtualisation/amazon-image.nix index 20d48add71293..44cb60809452c 100644 --- a/nixos/modules/virtualisation/amazon-image.nix +++ b/nixos/modules/virtualisation/amazon-image.nix @@ -48,7 +48,7 @@ in ]; boot.initrd.kernelModules = [ "xen-blkfront" "xen-netfront" ]; boot.initrd.availableKernelModules = [ "ixgbevf" "ena" "nvme" ]; - boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" ]; + boot.kernelParams = mkIf cfg.hvm [ "console=ttyS0" "random.trust_cpu=on" ]; # Prevent the nouveau kernel module from being loaded, as it # interferes with the nvidia/nvidia-uvm modules needed for CUDA. diff --git a/nixos/modules/virtualisation/ec2-amis.nix b/nixos/modules/virtualisation/ec2-amis.nix index 24de8cf1afbf6..3da63078a2145 100644 --- a/nixos/modules/virtualisation/ec2-amis.nix +++ b/nixos/modules/virtualisation/ec2-amis.nix @@ -329,5 +329,24 @@ let self = { "20.03".ap-east-1.hvm-ebs = "ami-0d18fdd309cdefa86"; "20.03".sa-east-1.hvm-ebs = "ami-09859378158ae971d"; - latest = self."20.03"; + # 20.09.1632.a6a3a368dda + "20.09".eu-west-1.hvm-ebs = "ami-01a79d5ce435f4db3"; + "20.09".eu-west-2.hvm-ebs = "ami-0cbe14f32904e6331"; + "20.09".eu-west-3.hvm-ebs = "ami-07f493412d6213de6"; + "20.09".eu-central-1.hvm-ebs = "ami-01d4a0c2248cbfe38"; + "20.09".eu-north-1.hvm-ebs = "ami-0003f54dd99d68e0f"; + "20.09".us-east-1.hvm-ebs = "ami-068a62d478710462d"; + "20.09".us-east-2.hvm-ebs = "ami-01ac677ff61399caa"; + "20.09".us-west-1.hvm-ebs = "ami-04befdb203b4b17f6"; + "20.09".us-west-2.hvm-ebs = "ami-0fb7bd4a43261c6b2"; + "20.09".ca-central-1.hvm-ebs = "ami-06d5ee429f153f856"; + "20.09".ap-southeast-1.hvm-ebs = "ami-0db0304e23c535b2a"; + "20.09".ap-southeast-2.hvm-ebs = "ami-045983c4db7e36447"; + "20.09".ap-northeast-1.hvm-ebs = "ami-0beb18d632cf64e5a"; + "20.09".ap-northeast-2.hvm-ebs = "ami-0dd0316af578862db"; + "20.09".ap-south-1.hvm-ebs = "ami-008d15ced81c88aed"; + "20.09".ap-east-1.hvm-ebs = "ami-071f49713f86ea965"; + "20.09".sa-east-1.hvm-ebs = "ami-05ded1ae35209b5a8"; + + latest = self."20.09"; }; in self diff --git a/nixos/modules/virtualisation/qemu-vm.nix b/nixos/modules/virtualisation/qemu-vm.nix index 42e43f5ee0233..33da920e94cc7 100644 --- a/nixos/modules/virtualisation/qemu-vm.nix +++ b/nixos/modules/virtualisation/qemu-vm.nix @@ -14,10 +14,11 @@ with import ../../lib/qemu-flags.nix { inherit pkgs; }; let - qemu = config.system.build.qemu or pkgs.qemu_test; cfg = config.virtualisation; + qemu = cfg.qemu.package; + consoles = lib.concatMapStringsSep " " (c: "console=${c}") cfg.qemu.consoles; driveOpts = { ... }: { @@ -401,6 +402,14 @@ in }; virtualisation.qemu = { + package = + mkOption { + type = types.package; + default = pkgs.qemu; + example = "pkgs.qemu_test"; + description = "QEMU package to use."; + }; + options = mkOption { type = types.listOf types.unspecified; @@ -735,16 +744,19 @@ in (isEnabled "VIRTIO_PCI") (isEnabled "VIRTIO_NET") (isEnabled "EXT4_FS") + (isEnabled "NET_9P_VIRTIO") + (isEnabled "9P_FS") (isYes "BLK_DEV") (isYes "PCI") - (isYes "EXPERIMENTAL") (isYes "NETDEVICES") (isYes "NET_CORE") (isYes "INET") (isYes "NETWORK_FILESYSTEMS") - ] ++ optional (!cfg.graphics) [ + ] ++ optionals (!cfg.graphics) [ (isYes "SERIAL_8250_CONSOLE") (isYes "SERIAL_8250") + ] ++ optionals (cfg.writableStore) [ + (isEnabled "OVERLAY_FS") ]; }; diff --git a/nixos/release-combined.nix b/nixos/release-combined.nix index 945ba90e3454c..d8b9a5f9b4bce 100644 --- a/nixos/release-combined.nix +++ b/nixos/release-combined.nix @@ -71,7 +71,6 @@ in rec { (onFullSupported "nixos.tests.fontconfig-default-fonts") (onFullSupported "nixos.tests.gnome3") (onFullSupported "nixos.tests.gnome3-xorg") - (onFullSupported "nixos.tests.hardened") (onSystems ["x86_64-linux"] "nixos.tests.hibernate") (onFullSupported "nixos.tests.i3wm") (onSystems ["x86_64-linux"] "nixos.tests.installer.btrfsSimple") @@ -93,12 +92,12 @@ in rec { (onFullSupported "nixos.tests.keymap.dvp") (onFullSupported "nixos.tests.keymap.neo") (onFullSupported "nixos.tests.keymap.qwertz") - (onFullSupported "nixos.tests.latestKernel.hardened") (onFullSupported "nixos.tests.latestKernel.login") (onFullSupported "nixos.tests.lightdm") (onFullSupported "nixos.tests.login") (onFullSupported "nixos.tests.misc") (onFullSupported "nixos.tests.mutableUsers") + (onFullSupported "nixos.tests.nano") (onFullSupported "nixos.tests.nat.firewall-conntrack") (onFullSupported "nixos.tests.nat.firewall") (onFullSupported "nixos.tests.nat.standalone") diff --git a/nixos/tests/all-tests.nix b/nixos/tests/all-tests.nix index 6564a958d5b4f..640cc84c80326 100644 --- a/nixos/tests/all-tests.nix +++ b/nixos/tests/all-tests.nix @@ -24,6 +24,7 @@ in _3proxy = handleTest ./3proxy.nix {}; acme = handleTest ./acme.nix {}; agda = handleTest ./agda.nix {}; + ammonite = handleTest ./ammonite.nix {}; atd = handleTest ./atd.nix {}; avahi = handleTest ./avahi.nix {}; avahi-with-resolved = handleTest ./avahi.nix { networkd = true; }; @@ -150,7 +151,6 @@ in hostname = handleTest ./hostname.nix {}; hound = handleTest ./hound.nix {}; hydra = handleTest ./hydra {}; - hydra-db-migration = handleTest ./hydra/db-migration.nix {}; i3wm = handleTest ./i3wm.nix {}; icingaweb2 = handleTest ./icingaweb2.nix {}; iftop = handleTest ./iftop.nix {}; @@ -176,6 +176,7 @@ in kernel-latest = handleTest ./kernel-latest.nix {}; kernel-lts = handleTest ./kernel-lts.nix {}; kernel-testing = handleTest ./kernel-testing.nix {}; + keycloak = discoverTests (import ./keycloak.nix); keymap = handleTest ./keymap.nix {}; knot = handleTest ./knot.nix {}; krb5 = discoverTests (import ./krb5 {}); @@ -224,6 +225,8 @@ in mysql-backup = handleTest ./mysql/mysql-backup.nix {}; mysql-replication = handleTest ./mysql/mysql-replication.nix {}; nagios = handleTest ./nagios.nix {}; + nano = handleTest ./nano.nix {}; + nar-serve = handleTest ./nar-serve.nix {}; nat.firewall = handleTest ./nat.nix { withFirewall = true; }; nat.firewall-conntrack = handleTest ./nat.nix { withFirewall = true; withConntrackHelpers = true; }; nat.standalone = handleTest ./nat.nix { withFirewall = false; }; @@ -243,6 +246,7 @@ in nfs4 = handleTest ./nfs { version = 4; }; nghttpx = handleTest ./nghttpx.nix {}; nginx = handleTest ./nginx.nix {}; + nginx-auth = handleTest ./nginx-auth.nix {}; nginx-etag = handleTest ./nginx-etag.nix {}; nginx-pubhtml = handleTest ./nginx-pubhtml.nix {}; nginx-sandbox = handleTestOn ["x86_64-linux"] ./nginx-sandbox.nix {}; @@ -253,6 +257,7 @@ in novacomd = handleTestOn ["x86_64-linux"] ./novacomd.nix {}; nsd = handleTest ./nsd.nix {}; nzbget = handleTest ./nzbget.nix {}; + oh-my-zsh = handleTest ./oh-my-zsh.nix {}; openarena = handleTest ./openarena.nix {}; openldap = handleTest ./openldap.nix {}; opensmtpd = handleTest ./opensmtpd.nix {}; @@ -310,6 +315,9 @@ in rxe = handleTest ./rxe.nix {}; samba = handleTest ./samba.nix {}; sanoid = handleTest ./sanoid.nix {}; + sbt = handleTest ./sbt.nix {}; + sbt-extras = handleTest ./sbt-extras.nix {}; + scala = handleTest ./scala.nix {}; sddm = handleTest ./sddm.nix {}; service-runner = handleTest ./service-runner.nix {}; shadowsocks = handleTest ./shadowsocks {}; @@ -341,6 +349,7 @@ in systemd-binfmt = handleTestOn ["x86_64-linux"] ./systemd-binfmt.nix {}; systemd-boot = handleTest ./systemd-boot.nix {}; systemd-confinement = handleTest ./systemd-confinement.nix {}; + systemd-journal = handleTest ./systemd-journal.nix {}; systemd-timesyncd = handleTest ./systemd-timesyncd.nix {}; systemd-networkd-vrf = handleTest ./systemd-networkd-vrf.nix {}; systemd-networkd = handleTest ./systemd-networkd.nix {}; @@ -362,6 +371,7 @@ in trezord = handleTest ./trezord.nix {}; trickster = handleTest ./trickster.nix {}; tuptime = handleTest ./tuptime.nix {}; + unbound = handleTest ./unbound.nix {}; udisks2 = handleTest ./udisks2.nix {}; unit-php = handleTest ./web-servers/unit-php.nix {}; upnp = handleTest ./upnp.nix {}; @@ -379,6 +389,7 @@ in xmonad = handleTest ./xmonad.nix {}; xrdp = handleTest ./xrdp.nix {}; xss-lock = handleTest ./xss-lock.nix {}; + xterm = handleTest ./xterm.nix {}; yabar = handleTest ./yabar.nix {}; yggdrasil = handleTest ./yggdrasil.nix {}; zfs = handleTest ./zfs.nix {}; diff --git a/nixos/tests/ammonite.nix b/nixos/tests/ammonite.nix index 1955e42be5f02..e9f06358e13f6 100644 --- a/nixos/tests/ammonite.nix +++ b/nixos/tests/ammonite.nix @@ -8,7 +8,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { amm = { pkgs, ... }: { - environment.systemPackages = [ pkgs.ammonite ]; + environment.systemPackages = [ (pkgs.ammonite.override { jre = pkgs.jre8; }) ]; }; }; diff --git a/nixos/tests/bitcoind.nix b/nixos/tests/bitcoind.nix index 09f3e4a6ec078..9068b29b8e5c1 100644 --- a/nixos/tests/bitcoind.nix +++ b/nixos/tests/bitcoind.nix @@ -31,16 +31,16 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.wait_for_unit("bitcoind-testnet.service") machine.wait_until_succeeds( - 'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' + 'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' + 'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:8332 | grep \'"chain":"main"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' + 'curl --fail --user rpc:rpc --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' ) machine.wait_until_succeeds( - 'curl --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' + 'curl --fail --user rpc2:rpc2 --data-binary \'{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }\' -H \'content-type: text/plain;\' localhost:18332 | grep \'"chain":"test"\' ' ) ''; }) diff --git a/nixos/tests/caddy.nix b/nixos/tests/caddy.nix index 445a7fa6b0b42..a21dbec248ab2 100644 --- a/nixos/tests/caddy.nix +++ b/nixos/tests/caddy.nix @@ -1,7 +1,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { name = "caddy"; meta = with pkgs.stdenv.lib.maintainers; { - maintainers = [ xfix filalex77 ]; + maintainers = [ xfix Br1ght0ne ]; }; nodes = { @@ -57,11 +57,13 @@ import ./make-test-python.nix ({ pkgs, ... }: { def check_etag(url): etag = webserver.succeed( - "curl -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format(url) + "curl --fail -v '{}' 2>&1 | sed -n -e \"s/^< [Ee][Tt][Aa][Gg]: *//p\"".format( + url + ) ) etag = etag.replace("\r\n", " ") http_code = webserver.succeed( - "curl --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format( + "curl --fail --silent --show-error -o /dev/null -w \"%{{http_code}}\" --head -H 'If-None-Match: {}' {}".format( etag, url ) ) diff --git a/nixos/tests/cadvisor.nix b/nixos/tests/cadvisor.nix index 60c04f1478003..664aa3ad876af 100644 --- a/nixos/tests/cadvisor.nix +++ b/nixos/tests/cadvisor.nix @@ -19,16 +19,16 @@ import ./make-test-python.nix ({ pkgs, ... } : { testScript = '' start_all() machine.wait_for_unit("cadvisor.service") - machine.succeed("curl http://localhost:8080/containers/") + machine.succeed("curl -f http://localhost:8080/containers/") influxdb.wait_for_unit("influxdb.service") # create influxdb database influxdb.succeed( - 'curl -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"' + 'curl -f -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE root"' ) influxdb.wait_for_unit("cadvisor.service") - influxdb.succeed("curl http://localhost:8080/containers/") + influxdb.succeed("curl -f http://localhost:8080/containers/") ''; }) diff --git a/nixos/tests/certmgr.nix b/nixos/tests/certmgr.nix index ef32f54400e30..8f5b89487793b 100644 --- a/nixos/tests/certmgr.nix +++ b/nixos/tests/certmgr.nix @@ -11,7 +11,7 @@ let file = { group = "nginx"; owner = "nginx"; - path = "/tmp/${host}-ca.pem"; + path = "/var/ssl/${host}-ca.pem"; }; label = "www_ca"; profile = "three-month"; @@ -20,13 +20,13 @@ let certificate = { group = "nginx"; owner = "nginx"; - path = "/tmp/${host}-cert.pem"; + path = "/var/ssl/${host}-cert.pem"; }; private_key = { group = "nginx"; mode = "0600"; owner = "nginx"; - path = "/tmp/${host}-key.pem"; + path = "/var/ssl/${host}-key.pem"; }; request = { CN = host; @@ -57,6 +57,8 @@ let services.cfssl.enable = true; systemd.services.cfssl.after = [ "cfssl-init.service" "networking.target" ]; + systemd.tmpfiles.rules = [ "d /var/ssl 777 root root" ]; + systemd.services.cfssl-init = { description = "Initialize the cfssl CA"; wantedBy = [ "multi-user.target" ]; @@ -87,8 +89,8 @@ let enable = true; virtualHosts = lib.mkMerge (map (host: { ${host} = { - sslCertificate = "/tmp/${host}-cert.pem"; - sslCertificateKey = "/tmp/${host}-key.pem"; + sslCertificate = "/var/ssl/${host}-cert.pem"; + sslCertificateKey = "/var/ssl/${host}-key.pem"; extraConfig = '' ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ''; @@ -124,16 +126,18 @@ in }; testScript = '' machine.wait_for_unit("cfssl.service") - machine.wait_until_succeeds("ls /tmp/decl.example.org-ca.pem") - machine.wait_until_succeeds("ls /tmp/decl.example.org-key.pem") - machine.wait_until_succeeds("ls /tmp/decl.example.org-cert.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-ca.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-key.pem") - machine.wait_until_succeeds("ls /tmp/imp.example.org-cert.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-ca.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-key.pem") + machine.wait_until_succeeds("ls /var/ssl/decl.example.org-cert.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-ca.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-key.pem") + machine.wait_until_succeeds("ls /var/ssl/imp.example.org-cert.pem") machine.wait_for_unit("nginx.service") assert 1 < int(machine.succeed('journalctl -u nginx | grep "Starting Nginx" | wc -l')) - machine.succeed("curl --cacert /tmp/imp.example.org-ca.pem https://imp.example.org") - machine.succeed("curl --cacert /tmp/decl.example.org-ca.pem https://decl.example.org") + machine.succeed("curl --cacert /var/ssl/imp.example.org-ca.pem https://imp.example.org") + machine.succeed( + "curl --cacert /var/ssl/decl.example.org-ca.pem https://decl.example.org" + ) ''; }; diff --git a/nixos/tests/cfssl.nix b/nixos/tests/cfssl.nix index e291fc285fba4..170f09d9b76cc 100644 --- a/nixos/tests/cfssl.nix +++ b/nixos/tests/cfssl.nix @@ -38,7 +38,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { testScript = let cfsslrequest = with pkgs; writeScript "cfsslrequest" '' - curl -X POST -H "Content-Type: application/json" -d @${csr} \ + curl -f -X POST -H "Content-Type: application/json" -d @${csr} \ http://localhost:8888/api/v1/cfssl/newkey | ${cfssl}/bin/cfssljson /tmp/certificate ''; csr = pkgs.writeText "csr.json" (builtins.toJSON { diff --git a/nixos/tests/common/acme/server/README.md b/nixos/tests/common/acme/server/README.md new file mode 100644 index 0000000000000..9de2b2c710292 --- /dev/null +++ b/nixos/tests/common/acme/server/README.md @@ -0,0 +1,21 @@ +# Fake Certificate Authority for ACME testing + +This will set up a test node running [pebble](https://github.com/letsencrypt/pebble) +to serve ACME certificate requests. + +## "Snake oil" certs + +The snake oil certs are hard coded into the repo for reasons explained [here](https://github.com/NixOS/nixpkgs/pull/91121#discussion_r505410235). +The root of the issue is that Nix will hash the derivation based on the arguments +to mkDerivation, not the output. [Minica](https://github.com/jsha/minica) will +always generate a random certificate even if the arguments are unchanged. As a +result, it's possible to end up in a situation where the cached and local +generated certs mismatch and cause issues with testing. + +To generate new certificates, run the following commands: + +```bash +nix-build generate-certs.nix +cp result/* . +rm result +``` diff --git a/nixos/tests/common/acme/server/acme.test.cert.pem b/nixos/tests/common/acme/server/acme.test.cert.pem new file mode 100644 index 0000000000000..76b0d916a8175 --- /dev/null +++ b/nixos/tests/common/acme/server/acme.test.cert.pem @@ -0,0 +1,19 @@ +-----BEGIN CERTIFICATE----- +MIIDLDCCAhSgAwIBAgIIRDAN3FHH//IwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE +AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMB4XDTIwMTAyMTEzMjgzNloXDTIyMTEy +MDEzMjgzNlowFDESMBAGA1UEAxMJYWNtZS50ZXN0MIIBIjANBgkqhkiG9w0BAQEF +AAOCAQ8AMIIBCgKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9 +Z4Xu5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeH +pImHO/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCN +Xf/LjIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/l +EnHrkcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOY +H+RfQfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABo3YwdDAOBgNVHQ8B +Af8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB +/wQCMAAwHwYDVR0jBBgwFoAU+8IZlLV/Qp5CXqpXMLvtxWlxcJwwFAYDVR0RBA0w +C4IJYWNtZS50ZXN0MA0GCSqGSIb3DQEBCwUAA4IBAQB0pe8I5/VDkB5VMgQB2GJV +GKzyigfWbVez9uLmqMj9PPP/zzYKSYeq+91aMuOZrnH7NqBxSTwanULkmqAmhbJJ +YkXw+FlFekf9FyxcuArzwzzNZDSGcjcdXpN8S2K1qkBd00iSJF9kU7pdZYCIKR20 +QirdBrELEfsJ3GU62a6N3a2YsrisZUvq5TbjGJDcytAtt+WG3gmV7RInLdFfPwbw +bEHPCnx0uiV0nxLjd/aVT+RceVrFQVt4hR99jLoMlBitSKluZ1ljsrpIyroBhQT0 +pp/pVi6HJdijG0fsPrC325NEGAwcpotLUhczoeM/rffKJd54wLhDkfYxOyRZXivs +-----END CERTIFICATE----- diff --git a/nixos/tests/common/acme/server/acme.test.key.pem b/nixos/tests/common/acme/server/acme.test.key.pem new file mode 100644 index 0000000000000..741df99a372e3 --- /dev/null +++ b/nixos/tests/common/acme/server/acme.test.key.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAo8XjMVUaljcaqQ5MFhfPuQgSwdyXEUbpSHz+5yPkE0h9Z4Xu +5BJF1Oq7h5ggCtadVsIspiY6Jm6aWDOjlh4myzW5UNBNUG3OPEk50vmmHFeHpImH +O/d8yb33QoF9VRcTZs4tuJYg7l9bSs4jNG72vYvv2YiGAcmjJcsmAZIfniCNXf/L +jIm+Cxykn+Vo3UuzO1w5/iuofdgWO/aZxMezmXUivlL3ih4cNzCJei8WlB/lEnHr +kcy3ogRmmynP5zcz7vmGIJX2ji6dhCa4Got5B7eZK76o2QglhQXqPatG0AOYH+Rf +QfzKemqPG5om9MgJtwFtTOU1LoaiBw//jXKESQIDAQABAoIBADox/2FwVFo8ioS4 +R+Ex5OZjMAcjU6sX/516jTmlT05q2+UFerYgqB/YqXqtW/V9/brulN8VhmRRuRbO +grq9TBu5o3hMDK0f18EkZB/MBnLbx594H033y6gEkPBZAyhRYtuNOEH3VwxdZhtW +1Lu1EoiYSUqLcNMBy6+KWJ8GRaXyacMYBlj2lMHmyzkA/t1+2mwTGC3lT6zN0F5Y +E5umXOxsn6Tb6q3KM9O5IvtmMMKpgj4HIHZLZ6j40nNgHwGRaAv4Sha/vx0DeBw3 +6VlNiTTPdShEkhESlM5/ocqTfI92VHJpM5gkqTYOWBi2aKIPfAopXoqoJdWl4pQ/ +NCFIu2ECgYEAzntNKIcQtf0ewe0/POo07SIFirvz6jVtYNMTzeQfL6CoEjYArJeu +Vzc4wEQfA4ZFVerBb1/O6M449gI3zex1PH4AX0h8q8DSjrppK1Jt2TnpVh97k7Gg +Tnat/M/yW3lWYkcMVJJ3AYurXLFTT1dYP0HvBwZN04yInrEcPNXKfmcCgYEAywyJ +51d4AE94PrANathKqSI/gk8sP+L1gzylZCcUEAiGk/1r45iYB4HN2gvWbS+CvSdp +F7ShlDWrTaNh2Bm1dgTjc4pWb4J+CPy/KN2sgLwIuM4+ZWIZmEDcio6khrM/gNqK +aR7xUsvWsqU26O84woY/xR8IHjSNF7cFWE1H2c8CgYEAt6SSi2kVQ8dMg84uYE8t +o3qO00U3OycpkOQqyQQLeKC62veMwfRl6swCfX4Y11mkcTXJtPTRYd2Ia8StPUkB +PDwUuKoPt/JXUvoYb59wc7M+BIsbrdBdc2u6cw+/zfutCNuH6/AYSBeg4WAVaIuW +wSwzG1xP+8cR+5IqOzEqWCECgYATweeVTCyQEyuHJghYMi2poXx+iIesu7/aAkex +pB/Oo5W8xrb90XZRnK7UHbzCqRHWqAQQ23Gxgztk9ZXqui2vCzC6qGZauV7cLwPG +zTMg36sVmHP314DYEM+k59ZYiQ6P0jQPoIQo407D2VGrfsOOIhQIcUmP7tsfyJ5L +hlGMfwKBgGq4VNnnuX8I5kl03NpaKfG+M8jEHmVwtI9RkPTCCX9bMjeG0cDxqPTF +TRkf3r8UWQTZ5QfAfAXYAOlZvmGhHjSembRbXMrMdi3rGsYRSrQL6n5NHnORUaMy +FCWo4gyAnniry7tx9dVNgmHmbjEHuQnf8AC1r3dibRCjvJWUiQ8H +-----END RSA PRIVATE KEY----- diff --git a/nixos/tests/common/acme/server/ca.cert.pem b/nixos/tests/common/acme/server/ca.cert.pem new file mode 100644 index 0000000000000..5c33e879b675b --- /dev/null +++ b/nixos/tests/common/acme/server/ca.cert.pem @@ -0,0 +1,20 @@ +-----BEGIN CERTIFICATE----- +MIIDSzCCAjOgAwIBAgIIeHRvRrNvbGQwDQYJKoZIhvcNAQELBQAwIDEeMBwGA1UE +AxMVbWluaWNhIHJvb3QgY2EgNzg3NDZmMCAXDTIwMTAyMTEzMjgzNloYDzIxMjAx +MDIxMTMyODM2WjAgMR4wHAYDVQQDExVtaW5pY2Egcm9vdCBjYSA3ODc0NmYwggEi +MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCrNTzVLDJOKtGYGLU98EEcLKps +tXHCLC6G54LKbEcU80fn+ArX8qsPSHyhdXQkcYjq6Vh/EDJ1TctyRSnvAjwyG4Aa +1Zy1QFc/JnjMjvzimCkUc9lQ+wkLwHSM/KGwR1cGjmtQ/EMClZTA0NwulJsXMKVz +bd5asXbq/yJTQ5Ww25HtdNjwRQXTvB7r3IKcY+DsED9CvFvC9oG/ZhtZqZuyyRdC +kFUrrv8WNUDkWSN+lMR6xMx8v0583IN6f11IhX0b+svK98G81B2eswBdkzvVyv9M +unZBO0JuJG8sdM502KhWLmzBC1ZbvgUBF9BumDRpMFH4DCj7+qQ2taWeGyc7AgMB +AAGjgYYwgYMwDgYDVR0PAQH/BAQDAgKEMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr +BgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1UdDgQWBBT7whmUtX9CnkJe +qlcwu+3FaXFwnDAfBgNVHSMEGDAWgBT7whmUtX9CnkJeqlcwu+3FaXFwnDANBgkq +hkiG9w0BAQsFAAOCAQEARMe1wKmF33GjEoLLw0oDDS4EdAv26BzCwtrlljsEtwQN +95oSzUNd6o4Js7WCG2o543OX6cxzM+yju8TES3+vJKDgsbNMU0bWCv//tdrb0/G8 +OkU3Kfi5q4fOauZ1pqGv/pXdfYhZ5ieB/zwis3ykANe5JfB0XqwCb1Vd0C3UCIS2 +NPKngRwNSzphIsbzfvxGDkdM1enuGl5CVyDhrwTMqGaJGDSOv6U5jKFxKRvigqTN +Ls9lPmT5NXYETduWLBR3yUIdH6kZXrcozZ02B9vjOB2Cv4RMDc+9eM30CLIWpf1I +097e7JkhzxFhfC/bMMt3P1FeQc+fwH91wdBmNi7tQw== +-----END CERTIFICATE----- diff --git a/nixos/tests/common/acme/server/ca.key.pem b/nixos/tests/common/acme/server/ca.key.pem new file mode 100644 index 0000000000000..ed46f5dccf467 --- /dev/null +++ b/nixos/tests/common/acme/server/ca.key.pem @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAqzU81SwyTirRmBi1PfBBHCyqbLVxwiwuhueCymxHFPNH5/gK +1/KrD0h8oXV0JHGI6ulYfxAydU3LckUp7wI8MhuAGtWctUBXPyZ4zI784pgpFHPZ +UPsJC8B0jPyhsEdXBo5rUPxDApWUwNDcLpSbFzClc23eWrF26v8iU0OVsNuR7XTY +8EUF07we69yCnGPg7BA/QrxbwvaBv2YbWambsskXQpBVK67/FjVA5FkjfpTEesTM +fL9OfNyDen9dSIV9G/rLyvfBvNQdnrMAXZM71cr/TLp2QTtCbiRvLHTOdNioVi5s +wQtWW74FARfQbpg0aTBR+Awo+/qkNrWlnhsnOwIDAQABAoIBAA3ykVkgd5ysmlSU +trcsCnHcJaojgff6l3PACoSpG4VWaGY6a8+54julgRm6MtMBONFCX0ZCsImj484U +Wl0xRmwil2YYPuL5MeJgJPktMObY1IfpBCw3tz3w2M3fiuCMf0d2dMGtO1xLiUnH ++hgFXTkfamsj6ThkOrbcQBSebeRxbKM5hqyCaQoieV+0IJnyxUVq/apib8N50VsH +SHd4oqLUuEZgg6N70+l5DpzedJUb4nrwS/KhUHUBgnoPItYBCiGPmrwLk7fUhPs6 +kTDqJDtc/xW/JbjmzhWEpVvtumcC/OEKULss7HLdeQqwVBrRQkznb0M9AnSra3d0 +X11/Y4ECgYEA3FC8SquLPFb2lHK4+YbJ4Ac6QVWeYFEHiZ0Rj+CmONmjcAvOGLPE +SblRLm3Nbrkxbm8FF6/AfXa/rviAKEVPs5xqGfSDw/3n1uInPcmShiBCLwM/jHH5 +NeVG+R5mTg5zyQ/pQMLWRcs+Ail+ZAnZuoGpW3Cdc8OtCUYFQ7XB6nsCgYEAxvBJ +zFxcTtsDzWbMWXejugQiUqJcEbKWwEfkRbf3J2rAVO2+EFr7LxdRfN2VwPiTQcWc +LnN2QN+ouOjqBMTh3qm5oQY+TLLHy86k9g1k0gXWkMRQgP2ZdfWH1HyrwjLUgLe1 +VezFN7N1azgy6xFkInAAvuA4loxElZNvkGBgekECgYA/Xw26ILvNIGqO6qzgQXAh ++5I7JsiGheg4IjDiBMlrQtbrLMoceuD0H9UFGNplhel9DXwWgxxIOncKejpK2x0A +2fX+/0FDh+4+9hA5ipiV8gN3iGSoHkSDxy5yC9d7jlapt+TtFt4Rd1OfxZWwatDw +/8jaH3t6yAcmyrhK8KYVrwKBgAE5KwsBqmOlvyE9N5Z5QN189wUREIXfVkP6bTHs +jq2EX4hmKdwJ4y+H8i1VY31bSfSGlY5HkXuWpH/2lrHO0CDBZG3UDwADvWzIaYVF +0c/kz0v2mRQh+xaZmus4lQnNrDbaalgL666LAPbW0qFVaws3KxoBYPe0BxvwWyhF +H3LBAoGBAKRRNsq2pWQ8Gqxc0rVoH0FlexU9U2ci3lsLmgEB0A/o/kQkSyAxaRM+ +VdKp3sWfO8o8lX5CVQslCNBSjDTNcat3Co4NEBLg6Xv1yKN/WN1GhusnchP9szsP +oU47gC89QhUyWSd6vvr2z2NG9C3cACxe4dhDSHQcE4nHSldzCKv2 +-----END RSA PRIVATE KEY----- diff --git a/nixos/tests/common/acme/server/default.nix b/nixos/tests/common/acme/server/default.nix index cea10c16900db..1c3bfdf76b7e7 100644 --- a/nixos/tests/common/acme/server/default.nix +++ b/nixos/tests/common/acme/server/default.nix @@ -51,10 +51,7 @@ # that it has to be started _before_ the ACME service. { config, pkgs, lib, ... }: let - testCerts = import ./snakeoil-certs.nix { - minica = pkgs.minica; - mkDerivation = pkgs.stdenv.mkDerivation; - }; + testCerts = import ./snakeoil-certs.nix; domain = testCerts.domain; resolver = let diff --git a/nixos/tests/common/acme/server/generate-certs.nix b/nixos/tests/common/acme/server/generate-certs.nix new file mode 100644 index 0000000000000..cd8fe0dffca17 --- /dev/null +++ b/nixos/tests/common/acme/server/generate-certs.nix @@ -0,0 +1,29 @@ +# Minica can provide a CA key and cert, plus a key +# and cert for our fake CA server's Web Front End (WFE). +{ + pkgs ? import <nixpkgs> {}, + minica ? pkgs.minica, + mkDerivation ? pkgs.stdenv.mkDerivation +}: +let + conf = import ./snakeoil-certs.nix; + domain = conf.domain; +in mkDerivation { + name = "test-certs"; + buildInputs = [ minica ]; + phases = [ "buildPhase" "installPhase" ]; + + buildPhase = '' + minica \ + --ca-key ca.key.pem \ + --ca-cert ca.cert.pem \ + --domains ${domain} + ''; + + installPhase = '' + mkdir -p $out + mv ca.*.pem $out/ + mv ${domain}/key.pem $out/${domain}.key.pem + mv ${domain}/cert.pem $out/${domain}.cert.pem + ''; +} diff --git a/nixos/tests/common/acme/server/snakeoil-certs.nix b/nixos/tests/common/acme/server/snakeoil-certs.nix index 4b6a38b8fa307..11c3f7fc9290c 100644 --- a/nixos/tests/common/acme/server/snakeoil-certs.nix +++ b/nixos/tests/common/acme/server/snakeoil-certs.nix @@ -1,37 +1,13 @@ -# Minica can provide a CA key and cert, plus a key -# and cert for our fake CA server's Web Front End (WFE). -{ minica, mkDerivation }: let domain = "acme.test"; - - selfSignedCertData = mkDerivation { - name = "test-certs"; - buildInputs = [ minica ]; - phases = [ "buildPhase" "installPhase" ]; - - buildPhase = '' - mkdir ca - minica \ - --ca-key ca/key.pem \ - --ca-cert ca/cert.pem \ - --domains ${domain} - chmod 600 ca/* - chmod 640 ${domain}/*.pem - ''; - - installPhase = '' - mkdir -p $out - mv ${domain} ca $out/ - ''; - }; in { inherit domain; ca = { - cert = "${selfSignedCertData}/ca/cert.pem"; - key = "${selfSignedCertData}/ca/key.pem"; + cert = ./ca.cert.pem; + key = ./ca.key.pem; }; "${domain}" = { - cert = "${selfSignedCertData}/${domain}/cert.pem"; - key = "${selfSignedCertData}/${domain}/key.pem"; + cert = ./. + "/${domain}.cert.pem"; + key = ./. + "/${domain}.key.pem"; }; } diff --git a/nixos/tests/convos.nix b/nixos/tests/convos.nix index b4ff1188fd8be..af2758c857d09 100644 --- a/nixos/tests/convos.nix +++ b/nixos/tests/convos.nix @@ -25,6 +25,6 @@ in machine.wait_for_unit("convos") machine.wait_for_open_port("${toString port}") machine.succeed("journalctl -u convos | grep -q 'Listening at.*${toString port}'") - machine.succeed("curl http://localhost:${toString port}/") + machine.succeed("curl -f http://localhost:${toString port}/") ''; }) diff --git a/nixos/tests/corerad.nix b/nixos/tests/corerad.nix index 37a1e90477a82..638010f92f445 100644 --- a/nixos/tests/corerad.nix +++ b/nixos/tests/corerad.nix @@ -80,7 +80,7 @@ import ./make-test-python.nix ( ), "SLAAC temporary address was not configured on client after router advertisement" with subtest("Verify HTTP debug server is configured"): - out = router.succeed("curl localhost:9430/metrics") + out = router.succeed("curl -f localhost:9430/metrics") assert ( "corerad_build_info" in out diff --git a/nixos/tests/docker-edge.nix b/nixos/tests/docker-edge.nix index 96de885a554ab..703179eef1956 100644 --- a/nixos/tests/docker-edge.nix +++ b/nixos/tests/docker-edge.nix @@ -43,7 +43,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { docker.fail("sudo -u noprivs docker ps") docker.succeed("docker stop sleeping") - # Must match version twice to ensure client and server versions are correct - docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "2" ]') + # Must match version 4 times to ensure client and server git commits and versions are correct + docker.succeed('[ $(docker version | grep ${pkgs.docker-edge.version} | wc -l) = "4" ]') ''; }) diff --git a/nixos/tests/docker-tools.nix b/nixos/tests/docker-tools.nix index edb9aec62db30..c1c41b0fc1100 100644 --- a/nixos/tests/docker-tools.nix +++ b/nixos/tests/docker-tools.nix @@ -115,7 +115,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { "docker load --input='${examples.nginx}'", "docker run --name nginx -d -p 8000:80 ${examples.nginx.imageName}", ) - docker.wait_until_succeeds("curl http://localhost:8000/") + docker.wait_until_succeeds("curl -f http://localhost:8000/") docker.succeed( "docker rm --force nginx", "docker rmi '${examples.nginx.imageName}'", ) diff --git a/nixos/tests/ferm.nix b/nixos/tests/ferm.nix index a73c9ce739cf1..112b5f19a7dea 100644 --- a/nixos/tests/ferm.nix +++ b/nixos/tests/ferm.nix @@ -56,6 +56,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { start_all() client.wait_for_unit("network-online.target") + server.wait_for_unit("network-online.target") server.wait_for_unit("ferm.service") server.wait_for_unit("nginx.service") server.wait_until_succeeds("ss -ntl | grep -q 80") diff --git a/nixos/tests/firefox.nix b/nixos/tests/firefox.nix index 7071baceba739..07e25bd4ca72d 100644 --- a/nixos/tests/firefox.nix +++ b/nixos/tests/firefox.nix @@ -11,26 +11,105 @@ import ./make-test-python.nix ({ pkgs, esr ? false, ... }: { environment.systemPackages = (if esr then [ pkgs.firefox-esr ] else [ pkgs.firefox ]) ++ [ pkgs.xdotool ]; + + # Need some more memory to record audio. + virtualisation.memorySize = "500"; + + # Create a virtual sound device, with mixing + # and all, for recording audio. + boot.kernelModules = [ "snd-aloop" ]; + sound.enable = true; + sound.extraConfig = '' + pcm.!default { + type plug + slave.pcm pcm.dmixer + } + pcm.dmixer { + type dmix + ipc_key 1 + slave { + pcm "hw:Loopback,0,0" + rate 48000 + periods 128 + period_time 0 + period_size 1024 + buffer_size 8192 + } + } + pcm.recorder { + type hw + card "Loopback" + device 1 + subdevice 0 + } + ''; + + systemd.services.audio-recorder = { + description = "Record NixOS test audio to /tmp/record.wav"; + script = "${pkgs.alsaUtils}/bin/arecord -D recorder -f S16_LE -r48000 /tmp/record.wav"; + }; + }; testScript = '' + from contextlib import contextmanager + + + @contextmanager + def audio_recording(machine: Machine) -> None: + """ + Perform actions while recording the + machine audio output. + """ + machine.systemctl("start audio-recorder") + yield + machine.systemctl("stop audio-recorder") + + + def wait_for_sound(machine: Machine) -> None: + """ + Wait until any sound has been emitted. + """ + machine.wait_for_file("/tmp/record.wav") + while True: + # Get at most 2M of the recording + machine.execute("tail -c 2M /tmp/record.wav > /tmp/last") + # Get the exact size + size = int(machine.succeed("stat -c '%s' /tmp/last").strip()) + # Compare it against /dev/zero using `cmp` (skipping 50B of WAVE header). + # If some non-NULL bytes are found it returns 1. + status, output = machine.execute( + f"cmp -i 50 -n {size - 50} /tmp/last /dev/zero 2>&1" + ) + if status == 1: + break + machine.sleep(2) + + machine.wait_for_x() - with subtest("wait until Firefox has finished loading the Valgrind docs page"): + with subtest("Wait until Firefox has finished loading the Valgrind docs page"): machine.execute( "xterm -e 'firefox file://${pkgs.valgrind.doc}/share/doc/valgrind/html/index.html' &" ) machine.wait_for_window("Valgrind") machine.sleep(40) + with subtest("Check whether Firefox can play sound"): + with audio_recording(machine): + machine.succeed( + "firefox file://${pkgs.sound-theme-freedesktop}/share/sounds/freedesktop/stereo/phone-incoming-call.oga &" + ) + wait_for_sound(machine) + machine.copy_from_vm("/tmp/record.wav") + + with subtest("Close sound test tab"): + machine.execute("xdotool key ctrl+w") + with subtest("Close default browser prompt"): machine.execute("xdotool key space") - with subtest("Hide default browser window"): - machine.sleep(2) - machine.execute("xdotool key F12") - - with subtest("wait until Firefox draws the developer tool panel"): + with subtest("Wait until Firefox draws the developer tool panel"): machine.sleep(10) machine.succeed("xwininfo -root -tree | grep Valgrind") machine.screenshot("screen") diff --git a/nixos/tests/go-neb.nix b/nixos/tests/go-neb.nix index d9e5db0b4a534..531ab5a66714e 100644 --- a/nixos/tests/go-neb.nix +++ b/nixos/tests/go-neb.nix @@ -34,7 +34,7 @@ import ./make-test-python.nix ({ pkgs, ... }: start_all() server.wait_for_unit("go-neb.service") server.wait_until_succeeds( - "curl -L http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U" + "curl -fL http://localhost:4050/services/hooks/d2lraXBlZGlhX3NlcnZpY2U" ) server.wait_until_succeeds( "journalctl -eu go-neb -o cat | grep -q service_id=wikipedia_service" diff --git a/nixos/tests/hadoop/hdfs.nix b/nixos/tests/hadoop/hdfs.nix index 85aaab34b158f..f1f98ed42eb31 100644 --- a/nixos/tests/hadoop/hdfs.nix +++ b/nixos/tests/hadoop/hdfs.nix @@ -48,7 +48,7 @@ import ../make-test-python.nix ({...}: { datanode.wait_for_open_port(9866) datanode.wait_for_open_port(9867) - namenode.succeed("curl http://namenode:9870") - datanode.succeed("curl http://datanode:9864") + namenode.succeed("curl -f http://namenode:9870") + datanode.succeed("curl -f http://datanode:9864") ''; }) diff --git a/nixos/tests/hadoop/yarn.nix b/nixos/tests/hadoop/yarn.nix index 2264ecaff1555..01077245d3973 100644 --- a/nixos/tests/hadoop/yarn.nix +++ b/nixos/tests/hadoop/yarn.nix @@ -40,7 +40,7 @@ import ../make-test-python.nix ({...}: { nodemanager.wait_for_open_port(8042) nodemanager.wait_for_open_port(8041) - resourcemanager.succeed("curl http://localhost:8088") - nodemanager.succeed("curl http://localhost:8042") + resourcemanager.succeed("curl -f http://localhost:8088") + nodemanager.succeed("curl -f http://localhost:8042") ''; }) diff --git a/nixos/tests/haproxy.nix b/nixos/tests/haproxy.nix index ffb77c052a24b..2c3878131b68b 100644 --- a/nixos/tests/haproxy.nix +++ b/nixos/tests/haproxy.nix @@ -39,9 +39,9 @@ import ./make-test-python.nix ({ pkgs, ...}: { machine.wait_for_unit("multi-user.target") machine.wait_for_unit("haproxy.service") machine.wait_for_unit("httpd.service") - assert "We are all good!" in machine.succeed("curl -k http://localhost:80/index.txt") + assert "We are all good!" in machine.succeed("curl -fk http://localhost:80/index.txt") assert "haproxy_process_pool_allocated_bytes" in machine.succeed( - "curl -k http://localhost:80/metrics" + "curl -fk http://localhost:80/metrics" ) with subtest("reload"): @@ -49,7 +49,7 @@ import ./make-test-python.nix ({ pkgs, ...}: { # wait some time to ensure the following request hits the reloaded haproxy machine.sleep(5) assert "We are all good!" in machine.succeed( - "curl -k http://localhost:80/index.txt" + "curl -fk http://localhost:80/index.txt" ) ''; }) diff --git a/nixos/tests/hitch/default.nix b/nixos/tests/hitch/default.nix index 904d12619d70e..8a2193e75f2ac 100644 --- a/nixos/tests/hitch/default.nix +++ b/nixos/tests/hitch/default.nix @@ -28,6 +28,6 @@ import ../make-test-python.nix ({ pkgs, ... }: machine.wait_for_unit("multi-user.target") machine.wait_for_unit("hitch.service") machine.wait_for_open_port(443) - assert "We are all good!" in machine.succeed("curl -k https://localhost:443/index.txt") + assert "We are all good!" in machine.succeed("curl -fk https://localhost:443/index.txt") ''; }) diff --git a/nixos/tests/hound.nix b/nixos/tests/hound.nix index 27c65abdf27c3..b8b10022bd920 100644 --- a/nixos/tests/hound.nix +++ b/nixos/tests/hound.nix @@ -53,7 +53,7 @@ import ./make-test-python.nix ({ pkgs, ... } : { machine.wait_for_unit("hound.service") machine.wait_for_open_port(6080) machine.wait_until_succeeds( - "curl http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'" + "curl -f http://127.0.0.1:6080/api/v1/search\?stats\=fosho\&repos\=\*\&rng=%3A20\&q\=hi\&files\=\&i=nope | grep 'Filename' | grep 'hello'" ) ''; }) diff --git a/nixos/tests/hydra/db-migration.nix b/nixos/tests/hydra/db-migration.nix deleted file mode 100644 index ca65e2e66aa1a..0000000000000 --- a/nixos/tests/hydra/db-migration.nix +++ /dev/null @@ -1,92 +0,0 @@ -{ system ? builtins.currentSystem -, pkgs ? import ../../.. { inherit system; } -, ... -}: - -let inherit (import ./common.nix { inherit system; }) baseConfig; in - -with import ../../lib/testing-python.nix { inherit system pkgs; }; -with pkgs.lib; - -{ mig = makeTest { - name = "hydra-db-migration"; - meta = with pkgs.stdenv.lib.maintainers; { - maintainers = [ ma27 ]; - }; - - nodes = { - original = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - - # An older version of Hydra before the db change - # for testing purposes. - services.hydra.package = pkgs.hydra-migration.overrideAttrs (old: { - inherit (old) pname; - version = "2020-02-06"; - src = pkgs.fetchFromGitHub { - owner = "NixOS"; - repo = "hydra"; - rev = "2b4f14963b16b21ebfcd6b6bfa7832842e9b2afc"; - sha256 = "16q0cffcsfx5pqd91n9k19850c1nbh4vvbd9h8yi64ihn7v8bick"; - }; - }); - }; - - migration_phase1 = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - services.hydra.package = pkgs.hydra-migration; - }; - - finished = { pkgs, lib, ... }: { - imports = [ baseConfig ]; - services.hydra.package = pkgs.hydra-unstable; - }; - }; - - testScript = { nodes, ... }: let - next = nodes.migration_phase1.config.system.build.toplevel; - finished = nodes.finished.config.system.build.toplevel; - in '' - original.start() - original.wait_for_unit("multi-user.target") - original.wait_for_unit("postgresql.service") - original.wait_for_unit("hydra-init.service") - original.require_unit_state("hydra-queue-runner.service") - original.require_unit_state("hydra-evaluator.service") - original.require_unit_state("hydra-notify.service") - original.succeed("hydra-create-user admin --role admin --password admin") - original.wait_for_open_port(3000) - original.succeed("create-trivial-project.sh") - original.wait_until_succeeds( - 'curl -L -s http://localhost:3000/build/1 -H "Accept: application/json" | jq .buildstatus | xargs test 0 -eq' - ) - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id" not in out - - original.succeed( - "${next}/bin/switch-to-configuration test >&2" - ) - original.wait_for_unit("hydra-init.service") - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id|integer|||" in out - - original.succeed("hydra-backfill-ids") - - original.succeed( - "${finished}/bin/switch-to-configuration test >&2" - ) - original.wait_for_unit("hydra-init.service") - - out = original.succeed("su -l postgres -c 'psql -d hydra <<< \"\\d+ builds\" -A'") - assert "jobset_id|integer||not null|" in out - - original.wait_until_succeeds( - 'curl -L -s http://localhost:3000/build/1 -H "Accept: application/json" | jq .buildstatus | xargs test 0 -eq' - ) - - original.shutdown() - ''; - }; -} diff --git a/nixos/tests/hydra/default.nix b/nixos/tests/hydra/default.nix index 2336e4033d6dc..e91a1cd3359d9 100644 --- a/nixos/tests/hydra/default.nix +++ b/nixos/tests/hydra/default.nix @@ -11,7 +11,7 @@ let inherit (import ./common.nix { inherit system; }) baseConfig; hydraPkgs = { - inherit (pkgs) hydra-migration hydra-unstable; + inherit (pkgs) hydra-unstable; }; makeHydraTest = with pkgs.lib; name: package: makeTest { diff --git a/nixos/tests/initrd-network-ssh/default.nix b/nixos/tests/initrd-network-ssh/default.nix index 017de6882081d..0ad0563b0ce15 100644 --- a/nixos/tests/initrd-network-ssh/default.nix +++ b/nixos/tests/initrd-network-ssh/default.nix @@ -22,6 +22,10 @@ import ../make-test-python.nix ({ lib, ... }: hostKeys = [ ./ssh_host_ed25519_key ]; }; }; + boot.initrd.extraUtilsCommands = '' + mkdir -p $out/secrets/etc/ssh + cat "${./ssh_host_ed25519_key}" > $out/secrets/etc/ssh/sh_host_ed25519_key + ''; boot.initrd.preLVMCommands = '' while true; do if [ -f fnord ]; then diff --git a/nixos/tests/installed-tests/default.nix b/nixos/tests/installed-tests/default.nix index 889a00d4b5686..e5d7009bb7b9a 100644 --- a/nixos/tests/installed-tests/default.nix +++ b/nixos/tests/installed-tests/default.nix @@ -94,6 +94,7 @@ in glib-networking = callInstalledTest ./glib-networking.nix {}; gnome-photos = callInstalledTest ./gnome-photos.nix {}; graphene = callInstalledTest ./graphene.nix {}; + gsconnect = callInstalledTest ./gsconnect.nix {}; ibus = callInstalledTest ./ibus.nix {}; libgdata = callInstalledTest ./libgdata.nix {}; glib-testing = callInstalledTest ./glib-testing.nix {}; @@ -101,5 +102,6 @@ in libxmlb = callInstalledTest ./libxmlb.nix {}; malcontent = callInstalledTest ./malcontent.nix {}; ostree = callInstalledTest ./ostree.nix {}; + pipewire = callInstalledTest ./pipewire.nix {}; xdg-desktop-portal = callInstalledTest ./xdg-desktop-portal.nix {}; } diff --git a/nixos/tests/installed-tests/fwupd.nix b/nixos/tests/installed-tests/fwupd.nix index 6a0ceb57dda48..a8a683a1af7b0 100644 --- a/nixos/tests/installed-tests/fwupd.nix +++ b/nixos/tests/installed-tests/fwupd.nix @@ -5,7 +5,7 @@ makeInstalledTest { testConfig = { services.fwupd.enable = true; - services.fwupd.blacklistPlugins = lib.mkForce []; # don't blacklist test plugin + services.fwupd.disabledPlugins = lib.mkForce []; # don't disable test plugin services.fwupd.enableTestRemote = true; virtualisation.memorySize = 768; }; diff --git a/nixos/tests/installed-tests/gsconnect.nix b/nixos/tests/installed-tests/gsconnect.nix new file mode 100644 index 0000000000000..ac39f7435786b --- /dev/null +++ b/nixos/tests/installed-tests/gsconnect.nix @@ -0,0 +1,7 @@ +{ pkgs, makeInstalledTest, ... }: + +makeInstalledTest { + tested = pkgs.gnomeExtensions.gsconnect; + + withX11 = true; +} diff --git a/nixos/tests/installed-tests/pipewire.nix b/nixos/tests/installed-tests/pipewire.nix new file mode 100644 index 0000000000000..f4154b5d2fd7d --- /dev/null +++ b/nixos/tests/installed-tests/pipewire.nix @@ -0,0 +1,5 @@ +{ pkgs, lib, makeInstalledTest, ... }: + +makeInstalledTest { + tested = pkgs.pipewire; +} diff --git a/nixos/tests/keycloak.nix b/nixos/tests/keycloak.nix new file mode 100644 index 0000000000000..f448a0f7095f6 --- /dev/null +++ b/nixos/tests/keycloak.nix @@ -0,0 +1,144 @@ +# This tests Keycloak: it starts the service, creates a realm with an +# OIDC client and a user, and simulates the user logging in to the +# client using their Keycloak login. + +let + frontendUrl = "http://keycloak/auth"; + initialAdminPassword = "h4IhoJFnt2iQIR9"; + + keycloakTest = import ./make-test-python.nix ( + { pkgs, databaseType, ... }: + { + name = "keycloak"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ talyz ]; + }; + + nodes = { + keycloak = { ... }: { + virtualisation.memorySize = 1024; + services.keycloak = { + enable = true; + inherit frontendUrl databaseType initialAdminPassword; + databasePasswordFile = pkgs.writeText "dbPassword" "wzf6vOCbPp6cqTH"; + }; + environment.systemPackages = with pkgs; [ + xmlstarlet + libtidy + jq + ]; + }; + }; + + testScript = + let + client = { + clientId = "test-client"; + name = "test-client"; + redirectUris = [ "urn:ietf:wg:oauth:2.0:oob" ]; + }; + + user = { + firstName = "Chuck"; + lastName = "Testa"; + username = "chuck.testa"; + email = "chuck.testa@example.com"; + }; + + password = "password1234"; + + realm = { + enabled = true; + realm = "test-realm"; + clients = [ client ]; + users = [( + user // { + enabled = true; + credentials = [{ + type = "password"; + temporary = false; + value = password; + }]; + } + )]; + }; + + realmDataJson = pkgs.writeText "realm-data.json" (builtins.toJSON realm); + + jqCheckUserinfo = pkgs.writeText "check-userinfo.jq" '' + if { + "firstName": .given_name, + "lastName": .family_name, + "username": .preferred_username, + "email": .email + } != ${builtins.toJSON user} then + error("Wrong user info!") + else + empty + end + ''; + in '' + keycloak.start() + keycloak.wait_for_unit("keycloak.service") + keycloak.wait_until_succeeds("curl -sSf ${frontendUrl}") + + + ### Realm Setup ### + + # Get an admin interface access token + keycloak.succeed( + "curl -sSf -d 'client_id=admin-cli' -d 'username=admin' -d 'password=${initialAdminPassword}' -d 'grant_type=password' '${frontendUrl}/realms/master/protocol/openid-connect/token' | jq -r '\"Authorization: bearer \" + .access_token' >admin_auth_header" + ) + + # Publish the realm, including a test OIDC client and user + keycloak.succeed( + "curl -sSf -H @admin_auth_header -X POST -H 'Content-Type: application/json' -d @${realmDataJson} '${frontendUrl}/admin/realms/'" + ) + + # Generate and save the client secret. To do this we need + # Keycloak's internal id for the client. + keycloak.succeed( + "curl -sSf -H @admin_auth_header '${frontendUrl}/admin/realms/${realm.realm}/clients?clientId=${client.name}' | jq -r '.[].id' >client_id", + "curl -sSf -H @admin_auth_header -X POST '${frontendUrl}/admin/realms/${realm.realm}/clients/'$(<client_id)'/client-secret' | jq -r .value >client_secret", + ) + + + ### Authentication Testing ### + + # Start the login process by sending an initial request to the + # OIDC authentication endpoint, saving the returned page. Tidy + # up the HTML (XmlStarlet is picky) and extract the login form + # post url. + keycloak.succeed( + "curl -sSf -c cookie '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/auth?client_id=${client.name}&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=openid+email&response_type=code&response_mode=query&nonce=qw4o89g3qqm' >login_form", + "tidy -q -m login_form || true", + "xml sel -T -t -m \"_:html/_:body/_:div/_:div/_:div/_:div/_:div/_:div/_:form[@id='kc-form-login']\" -v @action login_form >form_post_url", + ) + + # Post the login form and save the response. Once again tidy up + # the HTML, then extract the authorization code. + keycloak.succeed( + "curl -sSf -L -b cookie -d 'username=${user.username}' -d 'password=${password}' -d 'credentialId=' \"$(<form_post_url)\" >auth_code_html", + "tidy -q -m auth_code_html || true", + "xml sel -T -t -m \"_:html/_:body/_:div/_:div/_:div/_:div/_:div/_:input[@id='code']\" -v @value auth_code_html >auth_code", + ) + + # Exchange the authorization code for an access token. + keycloak.succeed( + "curl -sSf -d grant_type=authorization_code -d code=$(<auth_code) -d client_id=${client.name} -d client_secret=$(<client_secret) -d redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/token' | jq -r '\"Authorization: bearer \" + .access_token' >auth_header" + ) + + # Use the access token on the OIDC userinfo endpoint and check + # that the returned user info matches what we initialized the + # realm with. + keycloak.succeed( + "curl -sSf -H @auth_header '${frontendUrl}/realms/${realm.realm}/protocol/openid-connect/userinfo' | jq -f ${jqCheckUserinfo}" + ) + ''; + } + ); +in +{ + postgres = keycloakTest { databaseType = "postgresql"; }; + mysql = keycloakTest { databaseType = "mysql"; }; +} diff --git a/nixos/tests/leaps.nix b/nixos/tests/leaps.nix index ac0c602d44504..ec5b69a76290f 100644 --- a/nixos/tests/leaps.nix +++ b/nixos/tests/leaps.nix @@ -26,7 +26,7 @@ import ./make-test-python.nix ({ pkgs, ... }: server.wait_for_open_port(6666) client.wait_for_unit("network.target") assert "leaps" in client.succeed( - "${pkgs.curl}/bin/curl http://server:6666/leaps/" + "${pkgs.curl}/bin/curl -f http://server:6666/leaps/" ) ''; }) diff --git a/nixos/tests/limesurvey.nix b/nixos/tests/limesurvey.nix index 7228fcb833155..dad807fb73300 100644 --- a/nixos/tests/limesurvey.nix +++ b/nixos/tests/limesurvey.nix @@ -20,7 +20,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { machine.wait_for_unit("phpfpm-limesurvey.service") assert "The following surveys are available" in machine.succeed( - "curl http://example.local/" + "curl -f http://example.local/" ) ''; }) diff --git a/nixos/tests/loki.nix b/nixos/tests/loki.nix index dbf1e8a650f5d..bede775b7d3c3 100644 --- a/nixos/tests/loki.nix +++ b/nixos/tests/loki.nix @@ -12,15 +12,28 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: enable = true; configFile = "${pkgs.grafana-loki.src}/cmd/loki/loki-local-config.yaml"; }; - systemd.services.promtail = { - description = "Promtail service for Loki test"; - wantedBy = [ "multi-user.target" ]; - - serviceConfig = { - ExecStart = '' - ${pkgs.grafana-loki}/bin/promtail --config.file ${pkgs.grafana-loki.src}/cmd/promtail/promtail-local-config.yaml - ''; - DynamicUser = true; + services.promtail = { + enable = true; + configuration = { + server = { + http_listen_port = 9080; + grpc_listen_port = 0; + }; + clients = [ { url = "http://localhost:3100/loki/api/v1/push"; } ]; + scrape_configs = [ + { + job_name = "system"; + static_configs = [ + { + targets = [ "localhost" ]; + labels = { + job = "varlogs"; + __path__ = "/var/log/*log"; + }; + } + ]; + } + ]; }; }; }; @@ -32,6 +45,8 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: machine.wait_for_open_port(3100) machine.wait_for_open_port(9080) machine.succeed("echo 'Loki Ingestion Test' > /var/log/testlog") + # should not have access to journal unless specified + machine.fail("systemctl show --property=SupplementaryGroups promtail | grep -q systemd-journal") machine.wait_until_succeeds( "${pkgs.grafana-loki}/bin/logcli --addr='http://localhost:3100' query --no-labels '{job=\"varlogs\",filename=\"/var/log/testlog\"}' | grep -q 'Loki Ingestion Test'" ) diff --git a/nixos/tests/mailcatcher.nix b/nixos/tests/mailcatcher.nix index 2ef38544fe0a3..a55fba8a9950b 100644 --- a/nixos/tests/mailcatcher.nix +++ b/nixos/tests/mailcatcher.nix @@ -24,7 +24,7 @@ import ./make-test-python.nix ({ lib, ... }: 'echo "this is the body of the email" | mail -s "subject" root@example.org' ) assert "this is the body of the email" in machine.succeed( - "curl http://localhost:1080/messages/1.source" + "curl -f http://localhost:1080/messages/1.source" ) ''; }) diff --git a/nixos/tests/matrix-synapse.nix b/nixos/tests/matrix-synapse.nix index 9ca808721763d..6c8f1e188d528 100644 --- a/nixos/tests/matrix-synapse.nix +++ b/nixos/tests/matrix-synapse.nix @@ -77,12 +77,12 @@ in { start_all() serverpostgres.wait_for_unit("matrix-synapse.service") serverpostgres.wait_until_succeeds( - "curl -L --cacert ${ca_pem} https://localhost:8448/" + "curl --fail -L --cacert ${ca_pem} https://localhost:8448/" ) serverpostgres.require_unit_state("postgresql.service") serversqlite.wait_for_unit("matrix-synapse.service") serversqlite.wait_until_succeeds( - "curl -L --cacert ${ca_pem} https://localhost:8448/" + "curl --fail -L --cacert ${ca_pem} https://localhost:8448/" ) serversqlite.succeed("[ -e /var/lib/matrix-synapse/homeserver.db ]") ''; diff --git a/nixos/tests/mediawiki.nix b/nixos/tests/mediawiki.nix index 008682310cf65..702fefefa1610 100644 --- a/nixos/tests/mediawiki.nix +++ b/nixos/tests/mediawiki.nix @@ -22,7 +22,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: { machine.wait_for_unit("phpfpm-mediawiki.service") - page = machine.succeed("curl -L http://localhost/") + page = machine.succeed("curl -fL http://localhost/") assert "MediaWiki has been installed" in page ''; }) diff --git a/nixos/tests/metabase.nix b/nixos/tests/metabase.nix index 1450a4e9086f1..65619cc793a74 100644 --- a/nixos/tests/metabase.nix +++ b/nixos/tests/metabase.nix @@ -15,6 +15,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("metabase.service") machine.wait_for_open_port(3000) - machine.wait_until_succeeds("curl -L http://localhost:3000/setup | grep Metabase") + machine.wait_until_succeeds("curl -fL http://localhost:3000/setup | grep Metabase") ''; }) diff --git a/nixos/tests/morty.nix b/nixos/tests/morty.nix index 64c5a27665d6c..ff30b7c072bb1 100644 --- a/nixos/tests/morty.nix +++ b/nixos/tests/morty.nix @@ -24,7 +24,7 @@ import ./make-test-python.nix ({ pkgs, ... }: '' mortyProxyWithKey.wait_for_unit("default.target") mortyProxyWithKey.wait_for_open_port(3001) - mortyProxyWithKey.succeed("curl -L 127.0.0.1:3001 | grep MortyProxy") + mortyProxyWithKey.succeed("curl -fL 127.0.0.1:3001 | grep MortyProxy") ''; }) diff --git a/nixos/tests/nano.nix b/nixos/tests/nano.nix new file mode 100644 index 0000000000000..9e0a9e147f2c5 --- /dev/null +++ b/nixos/tests/nano.nix @@ -0,0 +1,44 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "nano"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { lib, ... }: { + environment.systemPackages = [ pkgs.nano ]; + }; + + testScript = { ... }: '' + start_all() + + with subtest("Create user and log in"): + machine.wait_for_unit("multi-user.target") + machine.wait_until_succeeds("pgrep -f 'agetty.*tty1'") + machine.succeed("useradd -m alice") + machine.succeed("(echo foobar; echo foobar) | passwd alice") + machine.wait_until_tty_matches(1, "login: ") + machine.send_chars("alice\n") + machine.wait_until_tty_matches(1, "login: alice") + machine.wait_until_succeeds("pgrep login") + machine.wait_until_tty_matches(1, "Password: ") + machine.send_chars("foobar\n") + machine.wait_until_succeeds("pgrep -u alice bash") + machine.screenshot("prompt") + + with subtest("Use nano"): + machine.send_chars("nano /tmp/foo") + machine.send_key("ret") + machine.sleep(2) + machine.send_chars("42") + machine.sleep(1) + machine.send_key("ctrl-x") + machine.sleep(1) + machine.send_key("y") + machine.sleep(1) + machine.screenshot("nano") + machine.sleep(1) + machine.send_key("ret") + machine.wait_for_file("/tmp/foo") + assert "42" in machine.succeed("cat /tmp/foo") + ''; +}) diff --git a/nixos/tests/nar-serve.nix b/nixos/tests/nar-serve.nix new file mode 100644 index 0000000000000..9ee738ffb170e --- /dev/null +++ b/nixos/tests/nar-serve.nix @@ -0,0 +1,48 @@ +import ./make-test-python.nix ( + { pkgs, lib, ... }: + { + name = "nar-serve"; + meta.maintainers = [ lib.maintainers.rizary ]; + nodes = + { + server = { pkgs, ... }: { + services.nginx = { + enable = true; + virtualHosts.default.root = "/var/www"; + }; + services.nar-serve = { + enable = true; + # Connect to the localhost nginx instead of the default + # https://cache.nixos.org + cacheURL = "http://localhost/"; + }; + environment.systemPackages = [ + pkgs.hello + pkgs.curl + ]; + + networking.firewall.allowedTCPPorts = [ 8383 ]; + + # virtualisation.diskSize = 2 * 1024; + }; + }; + testScript = '' + start_all() + + # Create a fake cache with Nginx service the static files + server.succeed( + "nix copy --to file:///var/www ${pkgs.hello}" + ) + server.wait_for_unit("nginx.service") + server.wait_for_open_port(80) + + # Check that nar-serve can return the content of the derivation + drvName = os.path.basename("${pkgs.hello}") + drvHash = drvName.split("-")[0] + server.wait_for_unit("nar-serve.service") + server.succeed( + "curl -o hello -f http://localhost:8383/nix/store/{}/bin/hello".format(drvHash) + ) + ''; + } +) diff --git a/nixos/tests/neo4j.nix b/nixos/tests/neo4j.nix index 32ee7f501b8bc..8329e5630d7af 100644 --- a/nixos/tests/neo4j.nix +++ b/nixos/tests/neo4j.nix @@ -15,6 +15,6 @@ import ./make-test-python.nix { master.wait_for_unit("neo4j") master.wait_for_open_port(7474) - master.succeed("curl http://localhost:7474/") + master.succeed("curl -f http://localhost:7474/") ''; } diff --git a/nixos/tests/nginx-auth.nix b/nixos/tests/nginx-auth.nix new file mode 100644 index 0000000000000..c0d24a20ddbcc --- /dev/null +++ b/nixos/tests/nginx-auth.nix @@ -0,0 +1,47 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "nginx-auth"; + + nodes = { + webserver = { pkgs, lib, ... }: { + services.nginx = let + root = pkgs.runCommand "testdir" {} '' + mkdir "$out" + echo hello world > "$out/index.html" + ''; + in { + enable = true; + + virtualHosts.lockedroot = { + inherit root; + basicAuth.alice = "jane"; + }; + + virtualHosts.lockedsubdir = { + inherit root; + locations."/sublocation/" = { + alias = "${root}/"; + basicAuth.bob = "john"; + }; + }; + }; + }; + }; + + testScript = '' + webserver.wait_for_unit("nginx") + webserver.wait_for_open_port(80) + + webserver.fail("curl --fail --resolve lockedroot:80:127.0.0.1 http://lockedroot") + webserver.succeed( + "curl --fail --resolve lockedroot:80:127.0.0.1 http://alice:jane@lockedroot" + ) + + webserver.succeed("curl --fail --resolve lockedsubdir:80:127.0.0.1 http://lockedsubdir") + webserver.fail( + "curl --fail --resolve lockedsubdir:80:127.0.0.1 http://lockedsubdir/sublocation/index.html" + ) + webserver.succeed( + "curl --fail --resolve lockedsubdir:80:127.0.0.1 http://bob:john@lockedsubdir/sublocation/index.html" + ) + ''; +}) diff --git a/nixos/tests/nixos-generate-config.nix b/nixos/tests/nixos-generate-config.nix index 6c83ccecc70a0..5daa55a8abbea 100644 --- a/nixos/tests/nixos-generate-config.nix +++ b/nixos/tests/nixos-generate-config.nix @@ -7,8 +7,16 @@ import ./make-test-python.nix ({ lib, ... } : { { config, pkgs, ... }: { imports = [ ./hardware-configuration.nix ]; $bootLoaderConfig + $desktopConfiguration } ''; + + system.nixos-generate-config.desktopConfiguration = '' + # DESKTOP + # services.xserver.enable = true; + # services.xserver.displayManager.gdm.enable = true; + # services.xserver.desktopManager.gnome3.enable = true; + ''; }; testScript = '' start_all() @@ -18,9 +26,17 @@ import ./make-test-python.nix ({ lib, ... } : { # Test if the configuration really is overridden machine.succeed("grep 'OVERRIDDEN' /etc/nixos/configuration.nix") + # Test if desktop configuration really is overridden + machine.succeed("grep 'DESKTOP' /etc/nixos/configuration.nix") + # Test of if the Perl variable $bootLoaderConfig is spliced correctly: machine.succeed( "grep 'boot\\.loader\\.grub\\.enable = true;' /etc/nixos/configuration.nix" ) + + # Test if the Perl variable $desktopConfiguration is spliced correctly + machine.succeed( + "grep 'services\\.xserver\\.desktopManager\\.gnome3\\.enable = true;' /etc/nixos/configuration.nix" + ) ''; }) diff --git a/nixos/tests/nzbget.nix b/nixos/tests/nzbget.nix index 12d8ed6ea8da1..b39c9b035e612 100644 --- a/nixos/tests/nzbget.nix +++ b/nixos/tests/nzbget.nix @@ -21,7 +21,7 @@ import ./make-test-python.nix ({ pkgs, ...} : { server.wait_for_unit("network.target") server.wait_for_open_port(6789) assert "This file is part of nzbget" in server.succeed( - "curl -s -u nzbget:tegbzn6789 http://127.0.0.1:6789" + "curl -f -s -u nzbget:tegbzn6789 http://127.0.0.1:6789" ) server.succeed( "${pkgs.nzbget}/bin/nzbget -n -o Control_iP=127.0.0.1 -o Control_port=6789 -o Control_password=tegbzn6789 -V" diff --git a/nixos/tests/oci-containers.nix b/nixos/tests/oci-containers.nix index bb6c019f07c99..0dfc7ffb276be 100644 --- a/nixos/tests/oci-containers.nix +++ b/nixos/tests/oci-containers.nix @@ -32,7 +32,7 @@ let start_all() ${backend}.wait_for_unit("${backend}-nginx.service") ${backend}.wait_for_open_port(8181) - ${backend}.wait_until_succeeds("curl http://localhost:8181 | grep Hello") + ${backend}.wait_until_succeeds("curl -f http://localhost:8181 | grep Hello") ''; }; diff --git a/nixos/tests/oh-my-zsh.nix b/nixos/tests/oh-my-zsh.nix new file mode 100644 index 0000000000000..57a073b086e88 --- /dev/null +++ b/nixos/tests/oh-my-zsh.nix @@ -0,0 +1,18 @@ +import ./make-test-python.nix ({ pkgs, ... }: { + name = "oh-my-zsh"; + + machine = { pkgs, ... }: + + { + programs.zsh = { + enable = true; + ohMyZsh.enable = true; + }; + }; + + testScript = '' + start_all() + machine.succeed("touch ~/.zshrc") + machine.succeed("zsh -c 'source /etc/zshrc && echo $ZSH | grep oh-my-zsh-${pkgs.oh-my-zsh.version}'") + ''; +}) diff --git a/nixos/tests/osrm-backend.nix b/nixos/tests/osrm-backend.nix index db67a5a589f9d..4067d5b1a239a 100644 --- a/nixos/tests/osrm-backend.nix +++ b/nixos/tests/osrm-backend.nix @@ -48,10 +48,10 @@ in { machine.wait_for_unit("osrm.service") machine.wait_for_open_port(${toString port}) assert "Boulevard Rainier III" in machine.succeed( - "curl --silent '${query}' | jq .waypoints[0].name" + "curl --fail --silent '${query}' | jq .waypoints[0].name" ) assert "Avenue de la Costa" in machine.succeed( - "curl --silent '${query}' | jq .waypoints[1].name" + "curl --fail --silent '${query}' | jq .waypoints[1].name" ) ''; }) diff --git a/nixos/tests/paperless.nix b/nixos/tests/paperless.nix index 355e7041d3fe2..fb83e6f976de6 100644 --- a/nixos/tests/paperless.nix +++ b/nixos/tests/paperless.nix @@ -23,14 +23,14 @@ import ./make-test-python.nix ({ lib, ... } : { with subtest("Service gets ready"): machine.wait_for_unit("paperless-server.service") # Wait until server accepts connections - machine.wait_until_succeeds("curl -s localhost:28981") + machine.wait_until_succeeds("curl -fs localhost:28981") with subtest("Test document is consumed"): machine.wait_until_succeeds( - "(($(curl -s localhost:28981/api/documents/ | jq .count) == 1))" + "(($(curl -fs localhost:28981/api/documents/ | jq .count) == 1))" ) assert "2005-10-16" in machine.succeed( - "curl -s localhost:28981/api/documents/ | jq '.results | .[0] | .created'" + "curl -fs localhost:28981/api/documents/ | jq '.results | .[0] | .created'" ) ''; }) diff --git a/nixos/tests/peerflix.nix b/nixos/tests/peerflix.nix index 37628604d49b5..6e534dedc4715 100644 --- a/nixos/tests/peerflix.nix +++ b/nixos/tests/peerflix.nix @@ -18,6 +18,6 @@ import ./make-test-python.nix ({ pkgs, ...} : { start_all() peerflix.wait_for_unit("peerflix.service") - peerflix.wait_until_succeeds("curl localhost:9000") + peerflix.wait_until_succeeds("curl -f localhost:9000") ''; }) diff --git a/nixos/tests/php/fpm.nix b/nixos/tests/php/fpm.nix index 513abd9437378..9ad515ebdde06 100644 --- a/nixos/tests/php/fpm.nix +++ b/nixos/tests/php/fpm.nix @@ -43,7 +43,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: { machine.wait_for_unit("phpfpm-foobar.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/") assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/nixos/tests/php/httpd.nix b/nixos/tests/php/httpd.nix index 1092e0ecadd39..27ea7a24e3a99 100644 --- a/nixos/tests/php/httpd.nix +++ b/nixos/tests/php/httpd.nix @@ -21,7 +21,7 @@ import ../make-test-python.nix ({pkgs, lib, ...}: { machine.wait_for_unit("httpd.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/") assert "PHP Version ${pkgs.php.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/nixos/tests/php/pcre.nix b/nixos/tests/php/pcre.nix index 3dd0964e60fbe..3ea19304bffd5 100644 --- a/nixos/tests/php/pcre.nix +++ b/nixos/tests/php/pcre.nix @@ -32,7 +32,7 @@ in import ../make-test-python.nix ({lib, ...}: { '' machine.wait_for_unit("httpd.service") # Ensure php evaluation by matching on the var_dump syntax - response = machine.succeed("curl -vvv -s http://127.0.0.1:80/index.php") + response = machine.succeed("curl -fvvv -s http://127.0.0.1:80/index.php") expected = 'string(${toString (builtins.stringLength testString)}) "${testString}"' assert expected in response, "Does not appear to be able to use subgroups." ''; diff --git a/nixos/tests/postfix.nix b/nixos/tests/postfix.nix index 37ae76afec107..6d22b4edba0a2 100644 --- a/nixos/tests/postfix.nix +++ b/nixos/tests/postfix.nix @@ -11,9 +11,9 @@ import ./make-test-python.nix { enable = true; enableSubmission = true; enableSubmissions = true; - sslCACert = certs.ca.cert; - sslCert = certs.${domain}.cert; - sslKey = certs.${domain}.key; + tlsTrustedAuthorities = "${certs.ca.cert}"; + sslCert = "${certs.${domain}.cert}"; + sslKey = "${certs.${domain}.key}"; submissionsOptions = { smtpd_sasl_auth_enable = "yes"; smtpd_client_restrictions = "permit"; diff --git a/nixos/tests/powerdns.nix b/nixos/tests/powerdns.nix index 75d71315e644d..d025934ad2b37 100644 --- a/nixos/tests/powerdns.nix +++ b/nixos/tests/powerdns.nix @@ -1,13 +1,65 @@ -import ./make-test-python.nix ({ pkgs, ... }: { +# This test runs PowerDNS authoritative server with the +# generic MySQL backend (gmysql) to connect to a +# MariaDB server using UNIX sockets authentication. + +import ./make-test-python.nix ({ pkgs, lib, ... }: { name = "powerdns"; nodes.server = { ... }: { services.powerdns.enable = true; - environment.systemPackages = [ pkgs.dnsutils ]; + services.powerdns.extraConfig = '' + launch=gmysql + gmysql-user=pdns + ''; + + services.mysql = { + enable = true; + package = pkgs.mariadb; + ensureDatabases = [ "powerdns" ]; + ensureUsers = lib.singleton + { name = "pdns"; + ensurePermissions = { "powerdns.*" = "ALL PRIVILEGES"; }; + }; + }; + + environment.systemPackages = with pkgs; + [ dnsutils powerdns mariadb ]; }; testScript = '' - server.wait_for_unit("pdns") - server.succeed("dig version.bind txt chaos \@127.0.0.1") + import re + + with subtest("PowerDNS database exists"): + server.wait_for_unit("mysql") + server.succeed("echo 'SHOW DATABASES;' | sudo -u pdns mysql -u pdns >&2") + + with subtest("Loading the MySQL schema works"): + server.succeed( + "sudo -u pdns mysql -u pdns -D powerdns <" + "${pkgs.powerdns}/share/doc/pdns/schema.mysql.sql" + ) + + with subtest("PowerDNS server starts"): + server.wait_for_unit("pdns") + server.succeed("dig version.bind txt chaos @127.0.0.1 >&2") + + with subtest("Adding an example zone works"): + # Extract configuration file needed by pdnsutil + unit = server.succeed("systemctl cat pdns") + conf = re.search("(--config-dir=[^ ]+)", unit).group(1) + pdnsutil = "sudo -u pdns pdnsutil " + conf + server.succeed(f"{pdnsutil} create-zone example.com ns1.example.com") + server.succeed(f"{pdnsutil} add-record example.com ns1 A 192.168.1.2") + + with subtest("Querying the example zone works"): + reply = server.succeed("dig +noall +answer ns1.example.com @127.0.0.1") + assert ( + "192.168.1.2" in reply + ), f"""" + The reply does not contain the expected IP address: + Expected: + ns1.example.com. 3600 IN A 192.168.1.2 + Reply: + {reply}""" ''; }) diff --git a/nixos/tests/prometheus-exporters.nix b/nixos/tests/prometheus-exporters.nix index ad2fff2b01f66..0b9957404f3b4 100644 --- a/nixos/tests/prometheus-exporters.nix +++ b/nixos/tests/prometheus-exporters.nix @@ -563,6 +563,37 @@ let ''; }; + rtl_433 = { + exporterConfig = { + enable = true; + }; + metricProvider = { + # Mock rtl_433 binary to return a dummy metric stream. + nixpkgs.overlays = [ (self: super: { + rtl_433 = self.runCommand "rtl_433" {} '' + mkdir -p "$out/bin" + cat <<EOF > "$out/bin/rtl_433" + #!/bin/sh + while true; do + printf '{"time" : "2020-04-26 13:37:42", "model" : "zopieux", "id" : 55, "channel" : 3, "temperature_C" : 18.000}\n' + sleep 4 + done + EOF + chmod +x "$out/bin/rtl_433" + ''; + }) ]; + }; + exporterTest = '' + wait_for_unit("prometheus-rtl_433-exporter.service") + wait_for_open_port(9550) + wait_until_succeeds( + "curl -sSf localhost:9550/metrics | grep -q '{}'".format( + 'rtl_433_temperature_celsius{channel="3",id="55",location="",model="zopieux"} 18' + ) + ) + ''; + }; + snmp = { exporterConfig = { enable = true; @@ -578,6 +609,50 @@ let ''; }; + sql = { + exporterConfig = { + configuration.jobs.points = { + interval = "1m"; + connections = [ + "postgres://prometheus-sql-exporter@/data?host=/run/postgresql&sslmode=disable" + ]; + queries = { + points = { + labels = [ "name" ]; + help = "Amount of points accumulated per person"; + values = [ "amount" ]; + query = "SELECT SUM(amount) as amount, name FROM points GROUP BY name"; + }; + }; + }; + enable = true; + user = "prometheus-sql-exporter"; + }; + metricProvider = { + services.postgresql = { + enable = true; + initialScript = builtins.toFile "init.sql" '' + CREATE DATABASE data; + \c data; + CREATE TABLE points (amount INT, name TEXT); + INSERT INTO points(amount, name) VALUES (1, 'jack'); + INSERT INTO points(amount, name) VALUES (2, 'jill'); + INSERT INTO points(amount, name) VALUES (3, 'jack'); + + CREATE USER "prometheus-sql-exporter"; + GRANT ALL PRIVILEGES ON DATABASE data TO "prometheus-sql-exporter"; + GRANT SELECT ON points TO "prometheus-sql-exporter"; + ''; + }; + systemd.services.prometheus-sql-exporter.after = [ "postgresql.service" ]; + }; + exporterTest = '' + wait_for_unit("prometheus-sql-exporter.service") + wait_for_open_port(9237) + succeed("curl http://localhost:9237/metrics | grep -c 'sql_points{' | grep -q 2") + ''; + }; + surfboard = { exporterConfig = { enable = true; diff --git a/nixos/tests/prometheus.nix b/nixos/tests/prometheus.nix index af2aa66a55266..6881c659e6d0c 100644 --- a/nixos/tests/prometheus.nix +++ b/nixos/tests/prometheus.nix @@ -19,7 +19,6 @@ let secret_key = s3.secretKey; insecure = true; signature_version2 = false; - encrypt_sse = false; put_user_metadata = {}; http_config = { idle_conn_timeout = "0s"; @@ -193,13 +192,13 @@ in import ./make-test-python.nix { # Check if prometheus responds to requests: prometheus.wait_for_unit("prometheus.service") prometheus.wait_for_open_port(${toString queryPort}) - prometheus.succeed("curl -s http://127.0.0.1:${toString queryPort}/metrics") + prometheus.succeed("curl -sf http://127.0.0.1:${toString queryPort}/metrics") # Let's test if pushing a metric to the pushgateway succeeds: prometheus.wait_for_unit("pushgateway.service") prometheus.succeed( "echo 'some_metric 3.14' | " - + "curl --data-binary \@- " + + "curl -f --data-binary \@- " + "http://127.0.0.1:${toString pushgwPort}/metrics/job/some_job" ) diff --git a/nixos/tests/riak.nix b/nixos/tests/riak.nix index 6915779e7e9c2..3dd4e333d6691 100644 --- a/nixos/tests/riak.nix +++ b/nixos/tests/riak.nix @@ -1,7 +1,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { name = "riak"; meta = with lib.maintainers; { - maintainers = [ filalex77 ]; + maintainers = [ Br1ght0ne ]; }; machine = { diff --git a/nixos/tests/sanoid.nix b/nixos/tests/sanoid.nix index 284b38932cceb..66ddaad60ea25 100644 --- a/nixos/tests/sanoid.nix +++ b/nixos/tests/sanoid.nix @@ -38,7 +38,7 @@ in { services.syncoid = { enable = true; - sshKey = "/root/.ssh/id_ecdsa"; + sshKey = "/var/lib/syncoid/id_ecdsa"; commonArgs = [ "--no-sync-snap" ]; commands."pool/test".target = "root@target:pool/test"; }; @@ -69,11 +69,12 @@ in { "udevadm settle", ) - source.succeed("mkdir -m 700 /root/.ssh") source.succeed( - "cat '${snakeOilPrivateKey}' > /root/.ssh/id_ecdsa" + "mkdir -m 700 -p /var/lib/syncoid", + "cat '${snakeOilPrivateKey}' > /var/lib/syncoid/id_ecdsa", + "chmod 600 /var/lib/syncoid/id_ecdsa", + "chown -R syncoid:syncoid /var/lib/syncoid/", ) - source.succeed("chmod 600 /root/.ssh/id_ecdsa") source.succeed("touch /tmp/mnt/test.txt") source.systemctl("start --wait sanoid.service") diff --git a/nixos/tests/sbt-extras.nix b/nixos/tests/sbt-extras.nix new file mode 100644 index 0000000000000..d63113f943e4e --- /dev/null +++ b/nixos/tests/sbt-extras.nix @@ -0,0 +1,16 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "sbt-extras"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + environment.systemPackages = [ pkgs.sbt-extras ]; + }; + + testScript = + '' + machine.succeed("(sbt -h)") + ''; +}) diff --git a/nixos/tests/sbt.nix b/nixos/tests/sbt.nix new file mode 100644 index 0000000000000..004d9c2e140a0 --- /dev/null +++ b/nixos/tests/sbt.nix @@ -0,0 +1,18 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "sbt"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + environment.systemPackages = [ pkgs.sbt ]; + }; + + testScript = + '' + machine.succeed( + "(sbt --offline --version 2>&1 || true) | grep 'getting org.scala-sbt sbt ${pkgs.sbt.version} (this may take some time)'" + ) + ''; +}) diff --git a/nixos/tests/scala.nix b/nixos/tests/scala.nix new file mode 100644 index 0000000000000..f99d9e563ffe3 --- /dev/null +++ b/nixos/tests/scala.nix @@ -0,0 +1,33 @@ +{ system ? builtins.currentSystem, + config ? {}, + pkgs ? import ../.. { inherit system config; } +}: + +with pkgs.lib; + +let + common = name: package: (import ./make-test-python.nix ({ + inherit name; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + nodes = { + scala = { ... }: { + environment.systemPackages = [ package ]; + }; + }; + + testScript = '' + start_all() + + scala.succeed("scalac -version 2>&1 | grep '^Scala compiler version ${package.version}'") + ''; + }) { inherit system; }); + +in with pkgs; { + scala_2_10 = common "scala_2_10" scala_2_10; + scala_2_11 = common "scala_2_11" scala_2_11; + scala_2_12 = common "scala_2_12" scala_2_12; + scala_2_13 = common "scala_2_13" scala_2_13; +} diff --git a/nixos/tests/service-runner.nix b/nixos/tests/service-runner.nix index 39ae66fe1116e..55fbbb7293445 100644 --- a/nixos/tests/service-runner.nix +++ b/nixos/tests/service-runner.nix @@ -29,7 +29,7 @@ import ./make-test-python.nix ({ pkgs, ... }: { """ ) machine.wait_for_open_port(80) - machine.succeed(f"curl {url}") + machine.succeed(f"curl -f {url}") machine.succeed("kill -INT $(cat my-nginx.pid)") machine.wait_for_closed_port(80) ''; diff --git a/nixos/tests/spacecookie.nix b/nixos/tests/spacecookie.nix index 6eff32a2e75d7..5b5022a742787 100644 --- a/nixos/tests/spacecookie.nix +++ b/nixos/tests/spacecookie.nix @@ -32,7 +32,7 @@ in ${gopherHost}.wait_for_unit("spacecookie.service") client.wait_for_unit("network.target") - fileResponse = client.succeed("curl -s gopher://${gopherHost}//${fileName}") + fileResponse = client.succeed("curl -f -s gopher://${gopherHost}//${fileName}") # the file response should return our created file exactly if not (fileResponse == "${fileContent}\n"): @@ -41,7 +41,7 @@ in # sanity check on the directory listing: we serve a directory and a file # via gopher, so the directory listing should have exactly two entries, # one with gopher file type 0 (file) and one with file type 1 (directory). - dirResponse = client.succeed("curl -s gopher://${gopherHost}") + dirResponse = client.succeed("curl -f -s gopher://${gopherHost}") dirEntries = [l[0] for l in dirResponse.split("\n") if len(l) > 0] dirEntries.sort() diff --git a/nixos/tests/sslh.nix b/nixos/tests/sslh.nix index 2a800aa52d0aa..17094606e8e6b 100644 --- a/nixos/tests/sslh.nix +++ b/nixos/tests/sslh.nix @@ -78,6 +78,6 @@ import ./make-test-python.nix { server.succeed(f"grep '{ip}' /tmp/foo{arg}") # check that http through sslh works - assert client.succeed(f"curl {arg} http://server:443").strip() == "hello world" + assert client.succeed(f"curl -f {arg} http://server:443").strip() == "hello world" ''; } diff --git a/nixos/tests/sympa.nix b/nixos/tests/sympa.nix index 280691f7cb402..eb38df180a789 100644 --- a/nixos/tests/sympa.nix +++ b/nixos/tests/sympa.nix @@ -30,7 +30,7 @@ import ./make-test-python.nix ({ pkgs, lib, ... }: { machine.wait_for_unit("sympa.service") machine.wait_for_unit("wwsympa.service") assert "Mailing lists service" in machine.succeed( - "curl --insecure -L http://localhost/" + "curl --fail --insecure -L http://localhost/" ) ''; }) diff --git a/nixos/tests/syncthing-relay.nix b/nixos/tests/syncthing-relay.nix index cd72ef1cbe1de..c144bf7fca372 100644 --- a/nixos/tests/syncthing-relay.nix +++ b/nixos/tests/syncthing-relay.nix @@ -19,7 +19,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { machine.wait_for_open_port(12346) out = machine.succeed( - "curl -sS http://localhost:12346/status | jq -r '.options.\"provided-by\"'" + "curl -sSf http://localhost:12346/status | jq -r '.options.\"provided-by\"'" ) assert "nixos-test" in out ''; diff --git a/nixos/tests/syncthing.nix b/nixos/tests/syncthing.nix index 9e2a8e01e3fb8..ac9df5e50c8c1 100644 --- a/nixos/tests/syncthing.nix +++ b/nixos/tests/syncthing.nix @@ -25,7 +25,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { "xmllint --xpath 'string(configuration/gui/apikey)' %s/config.xml" % confdir ).strip() oldConf = host.succeed( - "curl -Ss -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config" % APIKey + "curl -Ssf -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config" % APIKey ) conf = json.loads(oldConf) conf["devices"].append({"deviceID": deviceID, "id": name}) @@ -39,7 +39,7 @@ import ./make-test-python.nix ({ lib, pkgs, ... }: { ) newConf = json.dumps(conf) host.succeed( - "curl -Ss -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config -d %s" + "curl -Ssf -H 'X-API-Key: %s' 127.0.0.1:8384/rest/system/config -d %s" % (APIKey, shlex.quote(newConf)) ) diff --git a/nixos/tests/systemd-journal.nix b/nixos/tests/systemd-journal.nix new file mode 100644 index 0000000000000..c50c151ae10d8 --- /dev/null +++ b/nixos/tests/systemd-journal.nix @@ -0,0 +1,20 @@ +import ./make-test-python.nix ({ pkgs, ... }: + +{ + name = "systemd-journal"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ lewo ]; + }; + + machine = { pkgs, lib, ... }: { + services.journald.enableHttpGateway = true; + }; + + testScript = '' + machine.wait_for_unit("multi-user.target") + + machine.succeed( + "${pkgs.curl}/bin/curl -s localhost:19531/machine | ${pkgs.jq}/bin/jq -e '.hostname == \"machine\"'" + ) + ''; +}) diff --git a/nixos/tests/telegraf.nix b/nixos/tests/telegraf.nix index 73f741b113574..7f4b367525821 100644 --- a/nixos/tests/telegraf.nix +++ b/nixos/tests/telegraf.nix @@ -6,12 +6,15 @@ import ./make-test-python.nix ({ pkgs, ...} : { machine = { ... }: { services.telegraf.enable = true; + services.telegraf.environmentFiles = [(pkgs.writeText "secrets" '' + SECRET=example + '')]; services.telegraf.extraConfig = { agent.interval = "1s"; agent.flush_interval = "1s"; inputs.exec = { commands = [ - "${pkgs.runtimeShell} -c 'echo example,tag=a i=42i'" + "${pkgs.runtimeShell} -c 'echo $SECRET,tag=a i=42i'" ]; timeout = "5s"; data_format = "influx"; diff --git a/nixos/tests/trac.nix b/nixos/tests/trac.nix index 7953f8d41f771..af7182d1e185c 100644 --- a/nixos/tests/trac.nix +++ b/nixos/tests/trac.nix @@ -14,6 +14,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("trac.service") machine.wait_for_open_port(8000) - machine.wait_until_succeeds("curl -L http://localhost:8000/ | grep 'Trac Powered'") + machine.wait_until_succeeds("curl -fL http://localhost:8000/ | grep 'Trac Powered'") ''; }) diff --git a/nixos/tests/trezord.nix b/nixos/tests/trezord.nix index b7b3dd31942bb..7c8370f409ed4 100644 --- a/nixos/tests/trezord.nix +++ b/nixos/tests/trezord.nix @@ -14,6 +14,6 @@ import ./make-test-python.nix ({ pkgs, ... }: { start_all() machine.wait_for_unit("trezord.service") machine.wait_for_open_port(21325) - machine.wait_until_succeeds("curl -L http://localhost:21325/status/ | grep Version") + machine.wait_until_succeeds("curl -fL http://localhost:21325/status/ | grep Version") ''; }) diff --git a/nixos/tests/trickster.nix b/nixos/tests/trickster.nix index 713ac8f0b2fae..e32f919a1ada9 100644 --- a/nixos/tests/trickster.nix +++ b/nixos/tests/trickster.nix @@ -19,19 +19,19 @@ import ./make-test-python.nix ({ pkgs, ... }: { prometheus.wait_for_unit("prometheus.service") prometheus.wait_for_open_port(9090) prometheus.wait_until_succeeds( - "curl -L http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_for_unit("trickster.service") trickster.wait_for_open_port(8082) trickster.wait_for_open_port(9090) trickster.wait_until_succeeds( - "curl -L http://localhost:8082/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:8082/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_until_succeeds( - "curl -L http://prometheus:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://prometheus:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) trickster.wait_until_succeeds( - "curl -L http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" + "curl -fL http://localhost:9090/metrics | grep 'promhttp_metric_handler_requests_total{code=\"500\"} 0'" ) ''; }) diff --git a/nixos/tests/unbound.nix b/nixos/tests/unbound.nix new file mode 100644 index 0000000000000..dc8e5a9d3ed8c --- /dev/null +++ b/nixos/tests/unbound.nix @@ -0,0 +1,278 @@ +/* + Test that our unbound module indeed works as most users would expect. + There are a few settings that we must consider when modifying the test. The + ususal use-cases for unbound are + * running a recursive DNS resolver on the local machine + * running a recursive DNS resolver on the local machine, forwarding to a local DNS server via UDP/53 & TCP/53 + * running a recursive DNS resolver on the local machine, forwarding to a local DNS server via TCP/853 (DoT) + * running a recursive DNS resolver on a machine in the network awaiting input from clients over TCP/53 & UDP/53 + * running a recursive DNS resolver on a machine in the network awaiting input from clients over TCP/853 (DoT) + + In the below test setup we are trying to implement all of those use cases. + + Another aspect that we cover is access to the local control UNIX socket. It + can optionally be enabled and users can optionally be in a group to gain + access. Users that are not in the group (except for root) should not have + access to that socket. Also, when there is no socket configured, users + shouldn't be able to access the control socket at all. Not even root. +*/ +import ./make-test-python.nix ({ pkgs, lib, ... }: + let + # common client configuration that we can just use for the multitude of + # clients we are constructing + common = { lib, pkgs, ... }: { + config = { + environment.systemPackages = [ pkgs.knot-dns ]; + + # disable the root anchor update as we do not have internet access during + # the test execution + services.unbound.enableRootTrustAnchor = false; + }; + }; + + cert = pkgs.runCommandNoCC "selfSignedCerts" { buildInputs = [ pkgs.openssl ]; } '' + openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -nodes -subj '/CN=dns.example.local' + mkdir -p $out + cp key.pem cert.pem $out + ''; + in + { + name = "unbound"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ andir ]; + }; + + nodes = { + + # The server that actually serves our zones, this tests unbounds authoriative mode + authoritative = { lib, pkgs, config, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.1"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::1"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ 53 ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + interfaces = [ "192.168.0.1" "fd21::1" "::1" "127.0.0.1" ]; + allowedAccess = [ "192.168.0.0/24" "fd21::/64" "::1" "127.0.0.0/8" ]; + extraConfig = '' + server: + local-data: "example.local. IN A 1.2.3.4" + local-data: "example.local. IN AAAA abcd::eeff" + ''; + }; + }; + + # The resolver that knows that fowards (only) to the authoritative server + # and listens on UDP/53, TCP/53 & TCP/853. + resolver = { lib, nodes, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.2"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::2"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ + 53 # regular DNS + 853 # DNS over TLS + ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + allowedAccess = [ "192.168.0.0/24" "fd21::/64" "::1" "127.0.0.0/8" ]; + interfaces = [ "::1" "127.0.0.1" "192.168.0.2" "fd21::2" "192.168.0.2@853" "fd21::2@853" "::1@853" "127.0.0.1@853" ]; + forwardAddresses = [ + (lib.head nodes.authoritative.config.networking.interfaces.eth1.ipv6.addresses).address + (lib.head nodes.authoritative.config.networking.interfaces.eth1.ipv4.addresses).address + ]; + extraConfig = '' + server: + tls-service-pem: ${cert}/cert.pem + tls-service-key: ${cert}/key.pem + ''; + }; + }; + + # machine that runs a local unbound that will be reconfigured during test execution + local_resolver = { lib, nodes, config, ... }: { + imports = [ common ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ + { address = "192.168.0.3"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = lib.mkForce [ + { address = "fd21::3"; prefixLength = 64; } + ]; + networking.firewall.allowedTCPPorts = [ + 53 # regular DNS + ]; + networking.firewall.allowedUDPPorts = [ 53 ]; + + services.unbound = { + enable = true; + allowedAccess = [ "::1" "127.0.0.0/8" ]; + interfaces = [ "::1" "127.0.0.1" ]; + localControlSocketPath = "/run/unbound/unbound.ctl"; + extraConfig = '' + include: "/etc/unbound/extra*.conf" + ''; + }; + + users.users = { + # user that is permitted to access the unix socket + someuser.extraGroups = [ + config.users.users.unbound.group + ]; + + # user that is not permitted to access the unix socket + unauthorizeduser = {}; + }; + + environment.etc = { + "unbound-extra1.conf".text = '' + forward-zone: + name: "example.local." + forward-addr: ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address} + forward-addr: ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address} + ''; + "unbound-extra2.conf".text = '' + auth-zone: + name: something.local. + zonefile: ${pkgs.writeText "zone" '' + something.local. IN A 3.4.5.6 + ''} + ''; + }; + }; + + + # plain node that only has network access and doesn't run any part of the + # resolver software locally + client = { lib, nodes, ... }: { + imports = [ common ]; + networking.nameservers = [ + (lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address + (lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address + ]; + networking.interfaces.eth1.ipv4.addresses = [ + { address = "192.168.0.10"; prefixLength = 24; } + ]; + networking.interfaces.eth1.ipv6.addresses = [ + { address = "fd21::10"; prefixLength = 64; } + ]; + }; + }; + + testScript = { nodes, ... }: '' + import typing + import json + + zone = "example.local." + records = [("AAAA", "abcd::eeff"), ("A", "1.2.3.4")] + + + def query( + machine, + host: str, + query_type: str, + query: str, + expected: typing.Optional[str] = None, + args: typing.Optional[typing.List[str]] = None, + ): + """ + Execute a single query and compare the result with expectation + """ + text_args = "" + if args: + text_args = " ".join(args) + + out = machine.succeed( + f"kdig {text_args} {query} {query_type} @{host} +short" + ).strip() + machine.log(f"{host} replied with {out}") + if expected: + assert expected == out, f"Expected `{expected}` but got `{out}`" + + + def test(machine, remotes, /, doh=False, zone=zone, records=records, args=[]): + """ + Run queries for the given remotes on the given machine. + """ + for query_type, expected in records: + for remote in remotes: + query(machine, remote, query_type, zone, expected, args) + query(machine, remote, query_type, zone, expected, ["+tcp"] + args) + if doh: + query( + machine, + remote, + query_type, + zone, + expected, + ["+tcp", "+tls"] + args, + ) + + + client.start() + authoritative.wait_for_unit("unbound.service") + + # verify that we can resolve locally + with subtest("test the authoritative servers local responses"): + test(authoritative, ["::1", "127.0.0.1"]) + + resolver.wait_for_unit("unbound.service") + + with subtest("root is unable to use unbounc-control when the socket is not configured"): + resolver.succeed("which unbound-control") # the binary must exist + resolver.fail("unbound-control list_forwards") # the invocation must fail + + # verify that the resolver is able to resolve on all the local protocols + with subtest("test that the resolver resolves on all protocols and transports"): + test(resolver, ["::1", "127.0.0.1"], doh=True) + + resolver.wait_for_unit("multi-user.target") + + with subtest("client should be able to query the resolver"): + test(client, ["${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address}", "${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address}"], doh=True) + + # discard the client we do not need anymore + client.shutdown() + + local_resolver.wait_for_unit("multi-user.target") + + # link a new config file to /etc/unbound/extra.conf + local_resolver.succeed("ln -s /etc/unbound-extra1.conf /etc/unbound/extra1.conf") + + # reload the server & ensure the forwarding works + with subtest("test that the local resolver resolves on all protocols and transports"): + local_resolver.succeed("systemctl reload unbound") + print(local_resolver.succeed("journalctl -u unbound -n 1000")) + test(local_resolver, ["::1", "127.0.0.1"], args=["+timeout=60"]) + + with subtest("test that we can use the unbound control socket"): + out = local_resolver.succeed( + "sudo -u someuser -- unbound-control list_forwards" + ).strip() + + # Thank you black! Can't really break this line into a readable version. + expected = "example.local. IN forward ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv6.addresses).address} ${(lib.head nodes.resolver.config.networking.interfaces.eth1.ipv4.addresses).address}" + assert out == expected, f"Expected `{expected}` but got `{out}` instead." + local_resolver.fail("sudo -u unauthorizeduser -- unbound-control list_forwards") + + + # link a new config file to /etc/unbound/extra.conf + local_resolver.succeed("ln -sf /etc/unbound-extra2.conf /etc/unbound/extra2.conf") + + # reload the server & ensure the new local zone works + with subtest("test that we can query the new local zone"): + local_resolver.succeed("unbound-control reload") + r = [("A", "3.4.5.6")] + test(local_resolver, ["::1", "127.0.0.1"], zone="something.local.", records=r) + ''; + }) diff --git a/nixos/tests/upnp.nix b/nixos/tests/upnp.nix index a7d837ea0708e..046c0a56b2a76 100644 --- a/nixos/tests/upnp.nix +++ b/nixos/tests/upnp.nix @@ -90,7 +90,7 @@ in client1.succeed("upnpc -a ${internalClient1Address} 9000 9000 TCP") client1.wait_for_unit("httpd") - client2.wait_until_succeeds("curl http://${externalRouterAddress}:9000/") + client2.wait_until_succeeds("curl -f http://${externalRouterAddress}:9000/") ''; }) diff --git a/nixos/tests/uwsgi.nix b/nixos/tests/uwsgi.nix index 78a87147f55cd..7f4945a88030f 100644 --- a/nixos/tests/uwsgi.nix +++ b/nixos/tests/uwsgi.nix @@ -33,6 +33,6 @@ import ./make-test-python.nix ({ pkgs, ... }: machine.wait_for_unit("multi-user.target") machine.wait_for_unit("uwsgi.service") machine.wait_for_open_port(8000) - assert "Hello World" in machine.succeed("curl -v 127.0.0.1:8000") + assert "Hello World" in machine.succeed("curl -fv 127.0.0.1:8000") ''; }) diff --git a/nixos/tests/victoriametrics.nix b/nixos/tests/victoriametrics.nix index 73ef8b7286153..fff8d7005da17 100644 --- a/nixos/tests/victoriametrics.nix +++ b/nixos/tests/victoriametrics.nix @@ -19,9 +19,11 @@ import ./make-test-python.nix ({ pkgs, ...} : { # write some points and run simple query out = one.succeed( - "curl -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'" + "curl -f -d 'measurement,tag1=value1,tag2=value2 field1=123,field2=1.23' -X POST 'http://localhost:8428/write'" + ) + cmd = ( + """curl -f -s -G 'http://localhost:8428/api/v1/export' -d 'match={__name__!=""}'""" ) - cmd = """curl -s -G 'http://localhost:8428/api/v1/export' -d 'match={__name__!=""}'""" # data takes a while to appear one.wait_until_succeeds(f"[[ $({cmd} | wc -l) -ne 0 ]]") out = one.succeed(cmd) diff --git a/nixos/tests/web-servers/unit-php.nix b/nixos/tests/web-servers/unit-php.nix index 2a0a5bdaa5d51..033036ee76673 100644 --- a/nixos/tests/web-servers/unit-php.nix +++ b/nixos/tests/web-servers/unit-php.nix @@ -47,7 +47,7 @@ in { machine.wait_for_unit("unit.service") # Check so we get an evaluated PHP back - response = machine.succeed("curl -vvv -s http://127.0.0.1:9074/") + response = machine.succeed("curl -f -vvv -s http://127.0.0.1:9074/") assert "PHP Version ${pkgs.unit.usedPhp74.version}" in response, "PHP version not detected" # Check so we have database and some other extensions loaded diff --git a/nixos/tests/wordpress.nix b/nixos/tests/wordpress.nix index b7449859f7e62..5d740502bb572 100644 --- a/nixos/tests/wordpress.nix +++ b/nixos/tests/wordpress.nix @@ -40,7 +40,7 @@ import ./make-test-python.nix ({ pkgs, ... }: with subtest("website returns welcome screen"): for site_name in site_names: - assert "Welcome to the famous" in machine.succeed(f"curl -L {site_name}") + assert "Welcome to the famous" in machine.succeed(f"curl -fL {site_name}") with subtest("wordpress-init went through"): for site_name in site_names: diff --git a/nixos/tests/xterm.nix b/nixos/tests/xterm.nix new file mode 100644 index 0000000000000..9f30543bf3859 --- /dev/null +++ b/nixos/tests/xterm.nix @@ -0,0 +1,23 @@ +import ./make-test-python.nix ({ pkgs, ...} : { + name = "xterm"; + meta = with pkgs.stdenv.lib.maintainers; { + maintainers = [ nequissimus ]; + }; + + machine = { pkgs, ... }: + { + imports = [ ./common/x11.nix ]; + services.xserver.desktopManager.xterm.enable = false; + }; + + testScript = + '' + machine.wait_for_x() + machine.succeed("DISPLAY=:0 xterm -title testterm -class testterm -fullscreen &") + machine.sleep(2) + machine.send_chars("echo $XTERM_VERSION >> /tmp/xterm_version\n") + machine.wait_for_file("/tmp/xterm_version") + assert "${pkgs.xterm.version}" in machine.succeed("cat /tmp/xterm_version") + machine.screenshot("window") + ''; +}) diff --git a/nixos/tests/zfs.nix b/nixos/tests/zfs.nix index 87e6c900c98e0..e05cd540227a6 100644 --- a/nixos/tests/zfs.nix +++ b/nixos/tests/zfs.nix @@ -18,7 +18,7 @@ let maintainers = [ adisbladis ]; }; - machine = { pkgs, ... }: { + machine = { pkgs, lib, ... }: { virtualisation.emptyDiskImages = [ 4096 ]; networking.hostId = "deadbeef"; boot.kernelPackages = kernelPackage; @@ -26,6 +26,24 @@ let boot.zfs.enableUnstable = enableUnstable; environment.systemPackages = [ pkgs.parted ]; + + # Setup regular fileSystems machinery to ensure forceImportAll can be + # tested via the regular service units. + fileSystems = lib.mkVMOverride { + "/forcepool" = { + device = "forcepool"; + fsType = "zfs"; + options = [ "noauto" ]; + }; + }; + + # forcepool doesn't exist at first boot, and we need to manually test + # the import after tweaking the hostId. + systemd.services.zfs-import-forcepool.wantedBy = lib.mkVMOverride []; + systemd.targets.zfs.wantedBy = lib.mkVMOverride []; + boot.zfs.forceImportAll = true; + # /dev/disk/by-id doesn't get populated in the NixOS test framework + boot.zfs.devNodes = "/dev/disk/by-uuid"; }; testScript = '' @@ -57,6 +75,21 @@ let "zpool destroy rpool", "udevadm settle", ) + + with subtest("boot.zfs.forceImportAll works"): + machine.succeed( + "rm /etc/hostid", + "zgenhostid deadcafe", + "zpool create forcepool /dev/vdb1 -O mountpoint=legacy", + ) + machine.shutdown() + machine.start() + machine.succeed("udevadm settle") + machine.fail("zpool import forcepool") + machine.succeed( + "systemctl start zfs-import-forcepool.service", + "mount -t zfs forcepool /tmp/mnt", + ) '' + extraTest; }; |