I feel like DocBook is in a bad place compared to alternatives - it's to esoteric, too hard to write, and then there's the fact that it's XML.
TeX and friends are first class for paper docs and publication, but given their pre-web origins don't work well in that medium.
Markdown is just too limited and primitive for anything more than a blog post or basic README file - no captions on figures, hyperlinking to headings/sections isn't formalized, etc.
I tend to think that Sphinx + ReStructuredText makes the right balance in terms of publishing targets, and with all of Python, ReadTheDocs, and the Linux Kernel making the same decision, it seems like the right one. Intersphinx is pretty sweet if you have to link multiple sites together.
DocBook was SGML for most of its lifetime, and only became XML-only with the newest version 5, I believe. So you can use custom Wiki syntax if your concern is XML's verbosity and unfriendliness as an authoring format.
Other than that, I don't have a particular bad experience using DocBook for the one conference paper where I used it. Especially compared to LaTeX' limited options for rendering copies to HTML, I'd say DocBook is far better, like you say.
I have seen some places that use asciidoc as the interface, and then transpile asciidoc to docbook by using custom transformations. This works really well.
It's a huge PITA to write anything but basic markdown for books or long articles (30+ pages) that you need to output in multiple formats (e.g. one source to PDF, epub, and html) to be honest.
What I'd love to see is a general-purpose human readable format for documents. Markdown is handicapped by being (effectively) an HTML pre-processor for a tiny subset of HTML. Poor support for math, no captions, but more importantly very little semantics. You can often write a valid Markdown document that does what you want, but any significant information in the document (like "Author: My Name") just becomes part of the resulting HTML, not something that has any meaning in the Markdown spec.
RST strikes me as rather domain specific with its emphasis on technical writing, and IMO it also suffers from being intended as a preprocessor (the ugly link syntax for example).
I'm probably asking for too much, but what I'm looking for is something semantic enough to write a book in, but with viewing as plain text as the primary intended output. Gemini's gemtext format has very different goals, but it captures what I mean by "text as the primary intended output" very well.
> Markdown is just too limited and primitive
On the other hand, if you think of Markdown as just being a preprocessor, not a format that has to have everything built in, it's really not that bad if your renderer is good. For example: python-markdown has built-in plugins that make everything much nicer: meta, extra, and smarty are the one's I'd recommend. 'yafg' is a plugin that solves the figure support problem. 'markdown_vidify' allows embedding HTML5 videos.
I’ve heard people talking about reStructuredText as having ugly link syntax before, and I’m curious what specifically it is that you’re finding ugly. Are you talking about inline links, `link text <href>`_? In that, certainly I think the underscore was a mistake, though I’d say it’s good apart from that (angle brackets are good for URL delimition matching a long-standing historical convention as well as not using valid URL characters; parentheses as in Markdown are fundamentally particularly bad because parentheses are valid URL characters, so you can’t just drop a valid URL in willy-nilly). Or is it the reference style, with text_ or `link text`_ followed by .. _text: href or .. _link text: href? The reference definition is definitely… syntactically idiosyncratic, even if it’s quite consistent through the language.
Good question! Keep in mind that I'm not a regular rST user, so I'm not the best person to make complaints about it.
I dislike both the inline links and the footnotes. I agree with you about the underscore and that is the main aesthetic critique. I also rather dislike the use of backticks here, though. They have the smell of code especially with the nested <href>. If you have to have inline links I think Markdown's aren't that bad, especially because they lend themselves to easily adding inline images (exclamation point before the opening bracket). But both clearly function as markup, which is what would make them inappropriate for my envisioned format that is intended to be primarily rendered as plain text.
Additional complaints about rST syntax are the mixture of semantic document elements with code. For example rST has a glossary directive (great!) but unfortunately it is (and reads as) a piece of code (just like literalinclude or highlight).
In some ways what I'm looking for is even more constrained than Markdown. For example,
# GLOSSARY:
is valid markdown, but I'd like a standardized format that imputes semantic meaning to constructs like this and has rules for how the glossary section of the document must be formatted. The result would (hopefully) read without any hint that it is markup, but on the other hand make it possible to convert the document to PDF, HTML, EPUB, etc.
A format intended to be read as text probably wouldn't have inline links at all. A cleanly readable syntax for footnotes would probably make more sense, and with the proper semantics, a viewer / converter for the format could convert the link to inline if desired.
Interesting. I’m curious in significant part because I’m in the process of designing a lightweight markup language of my own, in which I am caring a lot about the ergonomics of the source text for reading, writing and tooling (I’d say all existing LMLs fail significantly on at least one of those).
Your remarks against inline links surprise me, because I am rather inclined to approve of inline links, because otherwise the href tends to get too far separated from the place it is, yet it is regularly important to the understanding of the text. Similarly endnotes as widely practised on the web (extremely commonly misnamed footnotes) are horrible for reading, as they’re in a completely different location, requiring jumping back and forth—whereas in the source, there’s a fair chance you wrote the footnote near the source. I’m rather partial to side notes, and have in the past set such a thing up in Sphinx so I could write the |main column content| like that, followed by `.. |main column content| sidenote:: the note text` below. Meanwhile on my website, currently Zola-powered, I write {{ sidenote(src="main column content", note="the note text") }} which invokes a shortcode that transforms it into the required HTML. Anyway, enough of that.
I despise Markdown’s inline link syntax, because it has no rhyme or reason: square brackets around the text, then parentheses around the href and possibly toooltip. Not only are parentheses around the link terrible due to being URL characters (thereby complicating the parser in an attempt to mitigate the damage by allowing matching parentheses, yet still requiring something beyond a simple URL serialiser for unmatched parentheses), the entire […](…) syntax is quite unlike anything used by anything else, and devoid of any rationale (you could just about describe the URL as parenthetical to the prose, but the claim is weak at best and perhaps harmed by the lack of space before it, though that would be a problem of its own)—people that don’t work in Markdown extremely extensively get the order wrong ((text)[href], [href](text) or (href)[text]) fairly regularly. As for Markdown’s inline image syntax, it’s possibly even more atrocious by virtue of being utterly opaque—it’s something that has to be taught and can’t be guessed, and is surprising when encountered even when you see that it is an image.
In my own markup language (code name T), I currently have [link text <href>] as the inline link syntax, quite similar to reStructuredText’s, but using matching outer delimiters. ([link text] <href> or word <href> are almost tempting, but definitely conflict with other parts of my goals.)
With your glossary heading, you may be interested in AsciiDoc (which incidentally for this submission has DocBook semantics); it’s big on the sort of semantics it looks like you may be interested in. (I’m not so fond of quite a bit of its syntax, but its semantics are robust like reStructuredText’s.) https://docs.asciidoctor.org/asciidoc/latest/sections/glossa... is its glossary stuff. Note that it requires more than just a heading with the text “glossary”, which would be problematic to operate on for linguistic reasons (though you could perhaps separate such style commands in a CSS-like way, `section:heading("GLOSSARY:") { role: glossary; }`).
Even that is enough to show that Markdown is bad by design. But when you actually use it for any serious document, you face the bad design and struggle with the "easy" tool :)
Asciidoc and Asciidoctor can produce DocBook, and from DocBook you can do most formats.
Working with the DocBook toolchain is rather unpleasant and brittle (depends a lot on the versions), so my company ended up packaging everything in a Docker container https://github.com/l3nz/dockbooker - this has worked pretty well for many years but now we mostly use Asciidoctor by itself.
AsciiDoctor has been designed so that its syntax elements map well to Docbook. You can use pretty much every feature of Docbook with the much nicer AsciiDoctor syntax on top.
I've the same experience. Soem years ago I tried DocBook to render HTML and PDF documents, but the whole setup was so unpleasant that I gave up. Writing the syntax is not overly hard, but also not pleasant. I ended up with AsciiDoc, what is nice to read and write and which has a rich set of features. It is also very easy to set up: just download AsciiDoctor and you're ready to go. https://asciidoctor.org/
I find DocBook (and XML in general) too cumbersome. I've been using Asciidoc in conjunction with Pandoc to create Word documents, Asciidoctor-PDF to generate PDF documents, and Antora to generate websites. This combination works well for me, anyway.
DocBook has served me very well. For my books I put together an automated publishing workflow that starts with DocBook and produces output for both print and digital (in my case, PDF, EPUB, and HTML). For context, I wrote and published two books (4 editions) with more than a 1,000 pages combined. My goal was always continuous writing and publishing where everything is fully automated.
Conceptually, XML is an excellent choice for this sort of thing. If you want to produce a good quality book, you need many features and markup that can support them. For a technically minded person, being able to programmatically inspect and modify the manuscript is awesome.
Unfortunately, the tooling, although very capable and very well documented (via Bob's free book http://www.sagehill.net/docbookxsl/), is very difficult to use unless you're an XML expert already. XSLT really isn't a good choice for this sort of thing. That's what I had to use because CSS-styling wasn't mature enough. These days I'd start with CSS instead of XSLT.
Crucially, I would use a commercial FO processor. I spent a lot of time with Apache FOP and, even though it's a great tool, it lacked a number of features that one needed for a professional use. (It may have improved by now, my experience is from a decade ago.) I eventually purchased a licence for Antenna Formatter, https://www.antennahouse.com/, which has saved me so much time.
You're not meant to write XML by hand, but there aren't that many great editors out there. I use OxygenXML, which is all right. It's expensive. The best thing about OxygenXML is that it supports change tracking. My copyeditor also uses OxygenXML and we have a great workflow that would be impossible to implement in any other way. For me this is the killer feature. A good copyeditor will make thousands of edits to your manuscript. Having an easy way to handle those, respond, and communicate... is key.
There are some other interesting tools out there, for example DocBook Compare, which we use to show exactly what changed between two editions. I have some screenshots of what that looks like in this blog post: https://blog.ivanristic.com/2022/02/bulletproof-tls-and-pki-...
To sum up: Do use DocBook. Use OxygenXML to write your books and collaborate with your coauthors and editors. Use Antenna Formatter and CSS for styling. It's going to cost you about $2,000 in software licences, but it's worth it and it will pay back loads in time saved. I understand that no one will spend this much money for a hobby, but for any sort of professional activity, it's a no-brainer.
It was flamed pretty badly when first introduced to HN, but I still think Steve Howell's Shpaml (http://shpaml.com/) completely removes the pain of writing angle brackets and keeping track of closing tags. Plus, it makes writing arbitrary XML really easy, just the thing for writing DocBook, I'll bet. And no need for a specific editor, just use your favorite.
I should add that you don't have to run Django to use Shpaml, just the Python script itself.
I wonder if Texinfo became a victim to naming that is easy to misunderstand. Based on many comments i've seen online, especially on Reddit, etc, whenever Texinfo is mentioned many people seem to read "info" which they then associate with the "GNU info" program and so they dismiss Texinfo as this weird documentation viewer that GNU insists on using instead of manpages - all of it being layers upon layers of misunderstandings that somehow (well, naming, i guess) persist.
"Texinfo" is basically a structured document format, somewhat similar to DocBook, except instead of XML it uses a more compact and (IMO) easier to read and write syntax that @looks{ like this }. It is largely meant for full manuals, guides, etc. There are a bunch of conversion tools (AFAIK written in Perl) to convert Texinfo documents to other formats, like TeX, HTML and of course info.
"info" is a hypertext document format, similar to CHM, though it predates it (and HTML) by many years. It is quite primitive though it has some neat indexing functionality which improves searching it (assuming good indices exist, but Texinfo has support for making those easily). One of its biggest issues is that it is a preformatted document and image support is "hacked in"..
"GNU info" is GNU's viewer for info documents which uses some Emacs-like shortcuts which trips people. However there are other viewers like pinfo and tkinfo, the latter providing a lightweight graphical viewer and some help systems (e.g. KHelpCenter) also support viewing info files. AFAIK there is also a cgi-based viewer so one can also use their browser. On the other hand most viewers do not seem to support images.
I'm not sure if i forgot any other use of "info" in there :-P
There is a relatively new "js-info" option for viewing texinfo output. It is regular rich HTML along with some JavaScript for convenient navigation: a dynamic sidebar, a keyboard bindings (based on traditional info mode). To see it in action, check out the DomTerm home-page (https://domterm.org/index.html), which is written in TexInfo with a little custom styling.
"js-info" is a name that's defeating attempts to search for it. Would you mind elaborating?
Texinfo does seem like something in between (La)TeX and AsciiDoc in difficulty, while hypothetically being just as rich in markup as almost any need requires. I think it does suffer a bit from seeming outdated in present HTML outputs, but you do demonstrate that it doesn't need to be so.
It's a document interchange format, and the code that persists it does not have feelings.
Use it to write out (externalize) the model structure of your document to persistent storage.
The structure reflected in the XML form of a document could either be hidden below a near-WYSIWYG GUI layer, or it may be more exposed in an XML editor: XML editors let you edit XML in a way that "normal" text editors don't (e.g. show markup and text directly together while still protecting mark-up tags instead of treating them as character-wise editable text).
28 comments
[ 0.23 ms ] story [ 82.6 ms ] threadTeX and friends are first class for paper docs and publication, but given their pre-web origins don't work well in that medium.
Markdown is just too limited and primitive for anything more than a blog post or basic README file - no captions on figures, hyperlinking to headings/sections isn't formalized, etc.
I tend to think that Sphinx + ReStructuredText makes the right balance in terms of publishing targets, and with all of Python, ReadTheDocs, and the Linux Kernel making the same decision, it seems like the right one. Intersphinx is pretty sweet if you have to link multiple sites together.
MS Word and Wikis can go die in a fire.
DocBook was SGML for most of its lifetime, and only became XML-only with the newest version 5, I believe. So you can use custom Wiki syntax if your concern is XML's verbosity and unfriendliness as an authoring format.
Other than that, I don't have a particular bad experience using DocBook for the one conference paper where I used it. Especially compared to LaTeX' limited options for rendering copies to HTML, I'd say DocBook is far better, like you say.
Also, DocBook is supported by pandoc.
It's a huge PITA to write anything but basic markdown for books or long articles (30+ pages) that you need to output in multiple formats (e.g. one source to PDF, epub, and html) to be honest.
RST strikes me as rather domain specific with its emphasis on technical writing, and IMO it also suffers from being intended as a preprocessor (the ugly link syntax for example).
I'm probably asking for too much, but what I'm looking for is something semantic enough to write a book in, but with viewing as plain text as the primary intended output. Gemini's gemtext format has very different goals, but it captures what I mean by "text as the primary intended output" very well.
> Markdown is just too limited and primitive
On the other hand, if you think of Markdown as just being a preprocessor, not a format that has to have everything built in, it's really not that bad if your renderer is good. For example: python-markdown has built-in plugins that make everything much nicer: meta, extra, and smarty are the one's I'd recommend. 'yafg' is a plugin that solves the figure support problem. 'markdown_vidify' allows embedding HTML5 videos.
I dislike both the inline links and the footnotes. I agree with you about the underscore and that is the main aesthetic critique. I also rather dislike the use of backticks here, though. They have the smell of code especially with the nested <href>. If you have to have inline links I think Markdown's aren't that bad, especially because they lend themselves to easily adding inline images (exclamation point before the opening bracket). But both clearly function as markup, which is what would make them inappropriate for my envisioned format that is intended to be primarily rendered as plain text.
Additional complaints about rST syntax are the mixture of semantic document elements with code. For example rST has a glossary directive (great!) but unfortunately it is (and reads as) a piece of code (just like literalinclude or highlight).
In some ways what I'm looking for is even more constrained than Markdown. For example,
is valid markdown, but I'd like a standardized format that imputes semantic meaning to constructs like this and has rules for how the glossary section of the document must be formatted. The result would (hopefully) read without any hint that it is markup, but on the other hand make it possible to convert the document to PDF, HTML, EPUB, etc.A format intended to be read as text probably wouldn't have inline links at all. A cleanly readable syntax for footnotes would probably make more sense, and with the proper semantics, a viewer / converter for the format could convert the link to inline if desired.
Your remarks against inline links surprise me, because I am rather inclined to approve of inline links, because otherwise the href tends to get too far separated from the place it is, yet it is regularly important to the understanding of the text. Similarly endnotes as widely practised on the web (extremely commonly misnamed footnotes) are horrible for reading, as they’re in a completely different location, requiring jumping back and forth—whereas in the source, there’s a fair chance you wrote the footnote near the source. I’m rather partial to side notes, and have in the past set such a thing up in Sphinx so I could write the |main column content| like that, followed by `.. |main column content| sidenote:: the note text` below. Meanwhile on my website, currently Zola-powered, I write {{ sidenote(src="main column content", note="the note text") }} which invokes a shortcode that transforms it into the required HTML. Anyway, enough of that.
I despise Markdown’s inline link syntax, because it has no rhyme or reason: square brackets around the text, then parentheses around the href and possibly toooltip. Not only are parentheses around the link terrible due to being URL characters (thereby complicating the parser in an attempt to mitigate the damage by allowing matching parentheses, yet still requiring something beyond a simple URL serialiser for unmatched parentheses), the entire […](…) syntax is quite unlike anything used by anything else, and devoid of any rationale (you could just about describe the URL as parenthetical to the prose, but the claim is weak at best and perhaps harmed by the lack of space before it, though that would be a problem of its own)—people that don’t work in Markdown extremely extensively get the order wrong ((text)[href], [href](text) or (href)[text]) fairly regularly. As for Markdown’s inline image syntax, it’s possibly even more atrocious by virtue of being utterly opaque—it’s something that has to be taught and can’t be guessed, and is surprising when encountered even when you see that it is an image.
In my own markup language (code name T), I currently have [link text <href>] as the inline link syntax, quite similar to reStructuredText’s, but using matching outer delimiters. ([link text] <href> or word <href> are almost tempting, but definitely conflict with other parts of my goals.)
With your glossary heading, you may be interested in AsciiDoc (which incidentally for this submission has DocBook semantics); it’s big on the sort of semantics it looks like you may be interested in. (I’m not so fond of quite a bit of its syntax, but its semantics are robust like reStructuredText’s.) https://docs.asciidoctor.org/asciidoc/latest/sections/glossa... is its glossary stuff. Note that it requires more than just a heading with the text “glossary”, which would be problematic to operate on for linguistic reasons (though you could perhaps separate such style commands in a CSS-like way, `section:heading("GLOSSARY:") { role: glossary; }`).
Github Flavored Markdown Stack Exchange Flavored Markdown Markdown Extra LiaScript MultiMarkdown LeanPub Flavored Markdown Markua Python-Markdown Kramdown
Even that is enough to show that Markdown is bad by design. But when you actually use it for any serious document, you face the bad design and struggle with the "easy" tool :)
Working with the DocBook toolchain is rather unpleasant and brittle (depends a lot on the versions), so my company ended up packaging everything in a Docker container https://github.com/l3nz/dockbooker - this has worked pretty well for many years but now we mostly use Asciidoctor by itself.
[1] https://pandoc.org/
Conceptually, XML is an excellent choice for this sort of thing. If you want to produce a good quality book, you need many features and markup that can support them. For a technically minded person, being able to programmatically inspect and modify the manuscript is awesome.
Unfortunately, the tooling, although very capable and very well documented (via Bob's free book http://www.sagehill.net/docbookxsl/), is very difficult to use unless you're an XML expert already. XSLT really isn't a good choice for this sort of thing. That's what I had to use because CSS-styling wasn't mature enough. These days I'd start with CSS instead of XSLT.
Crucially, I would use a commercial FO processor. I spent a lot of time with Apache FOP and, even though it's a great tool, it lacked a number of features that one needed for a professional use. (It may have improved by now, my experience is from a decade ago.) I eventually purchased a licence for Antenna Formatter, https://www.antennahouse.com/, which has saved me so much time.
You're not meant to write XML by hand, but there aren't that many great editors out there. I use OxygenXML, which is all right. It's expensive. The best thing about OxygenXML is that it supports change tracking. My copyeditor also uses OxygenXML and we have a great workflow that would be impossible to implement in any other way. For me this is the killer feature. A good copyeditor will make thousands of edits to your manuscript. Having an easy way to handle those, respond, and communicate... is key.
There are some other interesting tools out there, for example DocBook Compare, which we use to show exactly what changed between two editions. I have some screenshots of what that looks like in this blog post: https://blog.ivanristic.com/2022/02/bulletproof-tls-and-pki-...
To sum up: Do use DocBook. Use OxygenXML to write your books and collaborate with your coauthors and editors. Use Antenna Formatter and CSS for styling. It's going to cost you about $2,000 in software licences, but it's worth it and it will pay back loads in time saved. I understand that no one will spend this much money for a hobby, but for any sort of professional activity, it's a no-brainer.
It was flamed pretty badly when first introduced to HN, but I still think Steve Howell's Shpaml (http://shpaml.com/) completely removes the pain of writing angle brackets and keeping track of closing tags. Plus, it makes writing arbitrary XML really easy, just the thing for writing DocBook, I'll bet. And no need for a specific editor, just use your favorite.
I should add that you don't have to run Django to use Shpaml, just the Python script itself.
Consistent, fairly easy syntax, exports to DocBook, PDF, HTML and can be read interactively from terminal as well.
In contrast to markdown|asciidoc|reStructuredText|… and LaTeX, it handles more complex, non-linear documentation very well.
https://www.gnu.org/software/texinfo/manual/texinfo/texinfo....
"Texinfo" is basically a structured document format, somewhat similar to DocBook, except instead of XML it uses a more compact and (IMO) easier to read and write syntax that @looks{ like this }. It is largely meant for full manuals, guides, etc. There are a bunch of conversion tools (AFAIK written in Perl) to convert Texinfo documents to other formats, like TeX, HTML and of course info.
"info" is a hypertext document format, similar to CHM, though it predates it (and HTML) by many years. It is quite primitive though it has some neat indexing functionality which improves searching it (assuming good indices exist, but Texinfo has support for making those easily). One of its biggest issues is that it is a preformatted document and image support is "hacked in"..
"GNU info" is GNU's viewer for info documents which uses some Emacs-like shortcuts which trips people. However there are other viewers like pinfo and tkinfo, the latter providing a lightweight graphical viewer and some help systems (e.g. KHelpCenter) also support viewing info files. AFAIK there is also a cgi-based viewer so one can also use their browser. On the other hand most viewers do not seem to support images.
I'm not sure if i forgot any other use of "info" in there :-P
Texinfo does seem like something in between (La)TeX and AsciiDoc in difficulty, while hypothetically being just as rich in markup as almost any need requires. I think it does suffer a bit from seeming outdated in present HTML outputs, but you do demonstrate that it doesn't need to be so.
Specifically, what I call js-into is the "js" directory in the source code: info.js, modernizr.js, and info.css.
Use it to write out (externalize) the model structure of your document to persistent storage.
The structure reflected in the XML form of a document could either be hidden below a near-WYSIWYG GUI layer, or it may be more exposed in an XML editor: XML editors let you edit XML in a way that "normal" text editors don't (e.g. show markup and text directly together while still protecting mark-up tags instead of treating them as character-wise editable text).