about summary refs log tree commit diff
path: root/pkgs/tools/nix
AgeCommit message (Collapse)AuthorFilesLines
2023-03-27Merge remote-tracking branch 'origin/master' into haskell-updatesDennis Gosnell2-1/+316
2023-03-26tree-wide: convert rust with git deps to importCargoLockYureka2-1/+316
2023-03-23Merge remote-tracking branch 'origin/master' into haskell-updatesDennis Gosnell1-4/+3
2023-03-22nix-output-monitor: use hermes-json-0.2.0.1Dennis Gosnell1-3/+11
2023-03-21nix-init: 0.2.0 -> 0.2.1figsoda1-4/+3
Diff: https://github.com/nix-community/nix-init/compare/v0.2.0...v0.2.1 Changelog: https://github.com/nix-community/nix-init/blob/v0.2.1/CHANGELOG.md
2023-03-20nix-init: 0.1.1 -> 0.2.0figsoda2-22/+56
Diff: https://github.com/nix-community/nix-init/compare/v0.1.1...v0.2.0 Changelog: https://github.com/nix-community/nix-init/blob/v0.2.0/CHANGELOG.md
2023-03-14nixos-render-docs: use packageOverrides to construct python packagesSandro Jäckel1-25/+17
2023-03-07pkgs/tools/nix: enable strictDepsArtturin5-3/+10
`fd "\.nix" pkgs/tools/nix | xargs rg "strictDeps" --files-without-match | xargs rg "buildRust|buildGo|buildPyt|buildNim" --files-without-match | xargs vim -p` checked with Artturin/diffing
2023-02-21nixos-render-docs: add manual html converterpennae2-23/+433
this converter is currently supposed to be able to reproduce the docbook-generated html DOMs exactly, though not necessarily the html *files*. it mirrors many docbook behaviours that seem rather odd, such as top-level sections in chapters using the same heading depth as understood by html as their parent chapters do. over time we can hopefully remove all special casing needed to reproduce docbook rendering, but for now at least it doesn't hurt *too* much.
2023-02-21nixos-render-docs: render directly from file to filepennae1-7/+6
this will be necessary for html since there we have to do chunking into multiple files ourselves. writing one file from the caller of the converter and all others from within the converter is unnecessarily spread out, and returning a dict of file names and their contents is not quite as meaningful for docbook (which has only one file to begin with).
2023-02-21nixos-render-docs: add options html rendererpennae1-5/+111
it's not hooked up to anything yet, but that will come soon. there's a bit of docbook compat here that must be interoperable with the actual docbook exporter, but luckily it's not all that much.
2023-02-21nixos-render-docs: add html rendererpennae3-0/+425
the basic html renderer. it doesn't have all the docbook compatibility codes embedded into it, but there is a good amount. this renderer is unaware of manual structure and does not traverse structural include tokens (if it finds any it'll just fail), that task falls to derived classes. once we have more uses for structural includes than just the manual we may revisit this decision.
2023-02-21nixos-render-docs: add toc generatorpennae1-3/+100
the docbook toolchain uses docbook-xsl to generate its TOC, our html renderer will have to do this on its own. this generator uses a very straight-forward algorithm of only inspecting headings, but anything else could be inspected as well. (examples come to mind, but those do not have titles and would thus make for bad toc entries) we also use path information (that will be taken from include block args in the html renderer) to produce navigation information. the algorithm we use mirrors what docbook does, linking to the next/previous files in depth-first toc order. toc entries are linked to the tokens they refer to for easy use later.
2023-02-21nixos-render-docs: allow for options in include blockspennae1-6/+30
while docbook relies on external chunk-toc info to do chunking of the rendered manual we have nothing of the sort for html. there it seems easiest to add annotations to blocks to create new chunks. such annotations could be extended to docbook to create the chunk-toc instead of passing it in externally, but with docbook on the way out that seems like a waste of effort.
2023-02-21nixos-render-docs: check book structurepennae2-4/+45
text content in the toplevel file of a book will not render properly. the first proper element will be a preface, part, or chapter anyway, and those require includes to produce. parts do not currently allow headings in the part file itself, but that's mainly a renderer limitation. we can add support for headings in part intros when we need them in all other cases includes must be followed by either another include, a heading, or end of file. text content could not be properly linked to from a TOC without a preceding heading.
2023-02-21nixos-render-docs: require headings to have idspennae1-0/+8
without this we cannot build a TOC to arbitrary depth without generating ids for headings, but generated ids are fragile and liable to either break or point to different things if the manual changes shape. we already have the convention that all headings should have an id, this formalizes it.
2023-02-21nixos-render-docs: check heading continuitypennae1-0/+11
while not technically necessary for correct rendering of *contents* we do need to disallow heading levels being skipped to build a correct TOC. treating headings that have skipped a number of levels to actually be headings that many levels up only gets confusing, and inserting artifical intermediate headings suffers from problems, such as which ids to use and what to call them.
2023-02-21nixos-render-docs: check heading presence during parsingpennae2-23/+40
check that all required headings are present during parsing, not during rendering. building a correct TOC will need this since every TOC entry needs a heading to set its title, and every included substructure needs a title. also improve the error message on repeated title headings slightly, giving the end line turns out to not be very useful.
2023-02-21nixos-render-docs: add Freezable classpennae1-0/+21
for most of our data classes we can use dataclasses.dataclass with frozen=True or even plain named tuples. the TOC structure we'll need to generate proper navigation links is most easily represented and used as a cyclic structure though, and for that we can use neither. if we want to make the TOC structures immutable (which seems like a good idea) we'll need a hack of *some* kind, and this hack seems like the least intrusive.
2023-02-21nixos-render-docs: move recursive manual parsing to base classpennae1-76/+77
the html renderer will need all of these functions as well. some extensions will be needed, but we'll add those as they become necessary.
2023-02-21nixos-render-docs: don't render options during manual parsingpennae1-5/+7
we should really be rendering options at *rendering* time, not at parse time. currently this is just an academic exercise, but the html renderer will have to inspect the options.json data after the entire document has been parsed, but before anything gets rendered.
2023-02-21nixos-render-docs: keep revision in renderer, not converterpennae1-6/+6
ultimately it's the renderer that needs it, for the options rendering that will be simplified in a bit.
2023-02-21nixos-render-docs: drop options, env parameterspennae8-538/+288
these weren't used for anything. options never was (and does not contain any information for the renderer that we *want* to honor), and env is not used because typed renderer state is much more useful for all our cases.
2023-02-21nixos-render-docs: don't use markdown-it RendererProtocolpennae13-94/+101
our renderers carry significantly more state than markdown-it wants to easily cater for, and the html renderer will need even more state still. relying on the markdown-it-provided rendering functions has already proven to be a nuisance, and since parsing and rendering are split well enough we can just replace the rendering part with our own stuff outright. this also frees us from the tyranny of having to set instance variables before calling super().__init__ just to make sure that the renderer creation callback has access to everything it needs.
2023-02-21nixos-render-docs: add options asciidoc converterpennae3-0/+482
same reasoning as for the earlier commonmark converter.
2023-02-21nixos-render-docs: add options commonmark converterpennae3-0/+399
the old method of pasting parts of options.json into a markdown document and hoping for the best no longer works now that options.json contains more than just docbook. given the infrastructure we have now we can actually render options.md properly, so we may as well do that.
2023-02-21nixos-render-docs: forbid attrspans and examples in optionspennae1-0/+6
inline anchors are not allowed in option docs per the manual, and the sole class we current have (.keycap) is never used anyway. disallow them for now to avoid future surprises. the same goes for examples, which aren't even documented in the manual yet.
2023-02-21nixos-render-docs: refactor option docs restrictionspennae1-2/+3
move the restrictions we care about into a mixin class. a few more restrictions will appear soon and a few new converters as well, the renderers of which need not have these restrictions already baked in by accident (like the manpage renderer does right now).
2023-02-21nixos-render-docs: extend md_make_codepennae1-3/+3
add the ability to set the info string for a newly created fenced code block, and a flag to always emit a fenced block. the commonmark converter will need this to faithfully recreate fenced and indented code blocks.
2023-02-21nixos-render-docs: extract md code block factorypennae2-13/+15
the commonmark exporter can make good use once it appears.
2023-02-21nixos-render-docs: drop frozendictpennae4-16/+11
with mypy type checking and Mapping types this is a lot less useful than anticipated. let's drop it for simplicity and having fewer dependencies. frozendict 2.3.5 also broke the mypy checks.
2023-02-18nixos-render-docs: use multiprocessing for optionspennae3-4/+102
options processing is pretty slow right now, mostly because the markdown-it-py parser is pure python (and with performance pessimizations at that). options parsing *is* embarassingly parallel though, so we can just fork out all the work to worker processes and collect the results. multiprocessing probably has a greater benefit on linux than on darwin since the worker spawning method darwin uses is less efficient than fork() on linux. this hasn't been tested on darwin, only on linux, but if anything darwin will be faster with its preferred method.
2023-02-17Merge pull request #215973 from pennae/nrd-manual-structureRobert Hensing4-155/+196
nixos/manual: specify manual structure in markdown
2023-02-14treewide: use lib.optionalsFelix Buehler1-1/+1
2023-02-12nixos-render-docs: add structural includes, use for manualpennae2-149/+175
this adds support for structural includes to nixos-render-docs. structural includes provide a way to denote the (sub)structure of the nixos manual in the markdown source files, very similar to how we used literal docbook blocks before, and are processed by nixos-render-docs without involvement of xml tooling. this will ultimately allow us to emit the nixos manual in other formats as well, e.g. html, without going through docbook at all. alternatives to this source layout were also considered: a parallel structure using e.g. toml files that describe the document tree and links to each part is possible, but much more complicated to implement than the solution chosen here and makes it harder to follow which files have what substructure. it also makes it much harder to include a substructure in the middle of a file. much the same goes for command-line arguments to the converter, only that command-lined arguments are even harder to specify correctly and cannot be reasonably pulled together from many places without involving another layer of tooling. cli arguments would also mean that the manual structure would be fixed in default.nix, which is also not ideal.
2023-02-12nixos-render-docs: add support for <part>pennae1-3/+18
<part> is different from all other blocks we care about in that it requires textual content to be wrapped in <partintro>. add support for this to the generic docbook renderer, which will just assume that a part is the whole document start to finish. we do make provision for the manual renderer to close a partintro tag early though.
2023-02-12nixos-render-docs: use Mapping for options converterpennae1-1/+1
this way we can pass in frozendicts from other converters.
2023-02-12nixos-render-docs: print exception trees by __cause__pennae1-2/+2
__context__ is always set to the prior exception, even when not using the raise from form. __cause__ is only set during raise from. use __cause__ so we can override a leaf exception (eg KeyError to something more meaningful).
2023-02-10nixos-render-docs: render manual chapters during manual buildpennae1-0/+52
render all manual chapters to docbook from scratch every time the manual is built. nixos-render-docs is quick enough at this to not worry about the cost (needing only about a second), and it means we can remove md-to-db.sh in the next commit. no changes to the rendered html manual except for replacements and smartquotes.
2023-02-10nixos-render-docs: rename manual docbook converter to docbook-sectionpennae1-7/+7
we'll soon add another docbook converter that does not emit a section as a collection of chapters, but sections or chapters on their own. this should clarify naming a bit before there can be any confusion.
2023-02-10nixos-render-docs: add example blockspennae3-1/+52
this is currently only supported by the docbook exporter, and even the docbook exporter doesn't do much with them. we mirror the conversion pandoc did for consistency with the previous manual chapter conversion, which is to add just an anchor with the given id. future exporters that go directly to html might want to do more.
2023-02-10nixos-render-docs: add generic attributed-block parsingpennae2-15/+96
this is a subset of pandoc's fenced divs. currently we only use this for admonitions (which get a new name to differentiate them from other kinds of blocks), but more users will appear soon.
2023-02-10nixos-render-docs: promote compact-list attrs to core rulepennae1-25/+30
rules are a better place for this. since _post_parse is now empty (and presumably will never grow) we'll remove that as well.
2023-02-10nixos-render-docs: allow dots in heading idspennae1-1/+1
this is used by release notes (and we don't want to break links to those), and is also technically allowed anyway. we will *not* extend the regex to allow more characters just yet due to a mozilla recommendation against it (cf https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/id)
2023-02-10nixos-render-docs: promote heading id extraction to a core rulepennae2-12/+53
this should've been a core rule from the beginning. not being a core rule made it always run after smartquotes and replacements, which could've wrecked the id.
2023-02-10nixos-render-docs: add the .keycap classpennae1-6/+15
this lets us parse the `[F12]{.keycap}` syntax we recently introduced to the nixos manual markdown sources. the docbook renderer emits the keycap element for this class, the manpage renderer will reject it because it's not entirely clear what to do with it: while html has <kbd> mandoc has nothing of the sort, and with no current occurences in options doc we don't have to settle on a (potentially bad) way to render these.
2023-02-10nixos-render-docs: add support for full attributed spanspennae4-51/+165
this is pretty much what pandoc calls bracketed spans. since we only want to support ids and classes it doesn't seem fair to copy the name, so we'll call them "attributed span" for now. renderers are expected to know about *all* classes they could encounter and act appropriately, and since there are currently no classes with any defined behavior the most appropriate thing to do for now is to reject all classes.
2023-02-08nixos-render-docs: add all-features manpage renderer testpennae2-0/+196
now that the renderer produces the output we want to keep for the future we can add a test that checks all of its features. this test notably does not include markdown headings since we don't want to have those in manpages (at least right now), but tests for other converters may add headings for themselves.
2023-02-08nixos-render-docs: track links in manpagespennae4-7/+58
for the longest time we completely dropped link targets in configuration.nix.5. let's stop doing this now and instead provide a footnote for each link in a given option, numbered locally per option. we will currently duplicate the link for <labelless-links> because it makes it easier to get the collection of all links in a given option. this may not be useful enough, so over time we might decide to drop the footnotes for such links.
2023-02-08nixos-render-docs: indent and embolden list item heads in manpagespennae1-2/+2
this matches what html outputs do more closely, and feels like it'll be easier to read because it looks less like just another paragraph.
generated by cgit-pink 1.4.1 (git 2.36.1) at 2024-09-14 06:41:30 +0000