I was looking at RST as an answer for a documentation system with similar complexity to the system he's building and an intense need to mark stuff up with definite semantics, both in the case of capturing structures in the RST file in a database and mixing database results into the content.
Two problems I had were: (1) common with a lot of systems, the RST tools don't contain an RST unparser. I really wanted to be able to generate RST files automatically by merging content from other RST files and other sources and working with them via a document API. The RST tools don't support that. (2) The RST tools expect a certain defined set of blocks for a particular document; it would be very possible for a system to represent blocks in a generic way which would make possible tools that can transform documents without necessarily knowing the definitions of blocks inside it.
These are problems w/ the tools, not problems with RST itself but every time I'm forced to strip my code back to the walls is an opportunity to think about some other markup system such as something HTML-based.
My beef is that it doesn't have an RST writer. What I want is a central hub where I can suck in documentation that was originally in different formats and then manually edit it in a format where I can capture essential semantics and then feed it through the system again to produce output documents.
If it had that it would be a 200 foot tall giant robot, as it is it's just another documentation generator.
Yes, and with a somewhat different focus. I wish I could throw in 1000 pages of reference documentation, align it semantically with a few pages of documentation about a particular topic and then enrich those pages with snippets pulled out of the reference material. And things like that.
Instead of HTML you can use XML to write structured documents. With XML you can define whatever custom tags you need with schema validation if desired. The advantage of this approach is you have full control over the input schema and output, but the downside is XML is much more syntactically noisy compared to Markdown or RST and you will need a script to parse & convert the XML to your preferred output format(s).
> Markdown or RST and you will need a script to parse & convert the XML to your preferred output format(s).
XSLT will happily output any number of formats from an XML doc. There's also a number of existing and very capable XML schemas for documentation like DocBook. You can even go from a lightweight markup like RST or Markdown to DocBook and then have an XSLT pipeline that delivers that intermediary to any number of formats. There's also very capable graphical editors that will spit out DocBook directly.
I find the aversion to XML very strange. It was definitely applied in some places poorly by products with "Enterprise" somewhere in the name but the language itself is very useful. The aversion to XML has led to a lot of work re-learning of the lessons that led to some XML features. It's also led to implementing XML features poorly in other languages.
What I really like with XSLT is what you can do with user-defined functions. Not only are they helpful in implementing transforms but you can also write stateful functions that insert rows into a database or something.
You're not alone. I find XML to be massively more readable and flexible. I work in the macOS/iOS IT space, and our bread and butter is the Property List, which tends to either actually be XML underneath or is at the very least rendered like XML, all just conforming to the plist DTD (which is pretty short). These plists that conform to this fairly simple DTD are enough to describe every init task and background process to the OS, every preference in applications that use UserDefaults and CFPrefs (which is most), and I use them for storing structured data in and out of scripts that I throw together.
Loads of other systems I work with still output XML, and I have fast, powerful tools to work with them pre-installed on any Mac I work on, and it doesn't even matter that Apple hasn't updated them in a while since XML hasn't really been changed either. I can look at an XML doc and have so much information about the data inside just telegraphed into my head, just by the way the things are shaped. I don't have to read the tags like a book, I can just gloss over them and still get enough to understand the strings/numbers inside. Yet, they're still on the page, out of focus, in my periphery, providing anchoring and structure and navigation, I can find my place. Maybe I'm just a freak, or that comes from 20 years of staring at them. Whatever reason, I just can't do that with JSON as easily or as fast.
I don't hate on any of the other formats (save for YAML, not interested), and I really appreciate JSON's brevity sometimes for simple data and small responses, but that brevity quickly turns into wisps of formatting and text seemingly floating out in the middle of nowhere.
> I don't hate on any of the other formats (save for YAML, not interested), and I really appreciate JSON's brevity sometimes for simple data and small responses, but that brevity quickly turns into wisps of formatting and text seemingly floating out in the middle of nowhere.
Same. Every back end project I work on I recommend XML as the transport and every time I get a reflexive "ugh, XML". Then I spend an inordinate amount of time making Swagger endpoints so a consumer can figure out how the JSON output is supposed to behave.
It really isn't. Both namespaces and schema cause far more problems than they solve. And the theoretical advantages of having a declarative language for expressing transformations (and schema) is outweighed by how awful it actually is to write XSLT (and schema).
People hate namespaces wherever they are used, but they are one of the keys to "programming in the large". The first year after Java came out the official docs for namespaces weren't complete and I had to go to a web page at NASA to understand exactly how they work, particularly in the strange case where you don't include a package statement -- people got used it though. (I'll make the case that Java was a software reuse revolution not because it was OO or had a particular implementation of OO but because of getting a bunch of "little" features right such as namespaces and memory management.)
If we had namespaces for classes and id's in CSS we'd be able to transclude content from one HTML to another HTML page easily but we don't so we can't. (Shadow DOM helps a bit though) The "hygenic macro problem" is one of the many barriers people face doing metaprogramming. In the RDF world it's refreshing to be able to import facts from multiple systems into multiple namespaces so you can put data you got from different sources into the same database and "it just works".
A certain amount of namespacing is useful, but the XML model where you can't even write a hello world xpath query until you've set up all your namespaces and used them in your query is something that clearly does more harm than good. Java is I think widely recognised as having gone too far on the namespacing front; most post-Java languages have a much less strictly hierarchical namespacing model and I don't think any other package registry has adopted the fully nested style that Maven does. And even Java's heavyweight namespacing is much easier to work with than XML's thanks to things like auto-import in IDEs.
I don't think namespaces solve the macro hygiene problem, because if you're going to have macros then they need to be able to interact with and manipulate your namespaces, and sooner or later you need your macro to e.g. generate a fresh identifier for use within an existing namespace and then you're right back where you started.
Being able to disambiguate when you have one document including another would be useful. But the cost/benefit on XML-style big namespacing up-front just doesn't stack up.
My take is that XSLT comes from a different universe where Prolog became a mainstream programming language.
That is, XSLT is based on pattern matching rules that most people find strange and unfamiliar. It wouldn't be so strange to people if this kind of system was more widespread, but as an island that's different from everything else I think people struggle to wrap their heads around it.
- is a superset of XML (XML is derived from SGML as a simplified, proper subset)
- can parse HTML precisely with all bells and whistles such as tag omission, enumerated attributes and other shortforms (which are based on SGML after all)
- can define markdown syntax as an SGML SHORTREF customization (note that SGML can not cover all of markdown syntax; for example, defining reference links somewhere at the end of a document and expecting SGML to pull a href URL into the place where the link is referenced won't work with SGML SHORTREF)
The combination of markdown-as-sgml and inline HTML or HTML blocks is particularly nice and predictable, and also informs further customization such as using entities (text variables), custom elements, and other advanced SGML stuff with markdown that comes up frequently as a requirement.
I am (was?) a committer to docutils (the main tooling for rST). One of the reasons I moved my tooling to md was because docutils was such a pain to work with (refusing to move to something like GitHub illustrates how hostile working with it is).
An interesting commonality among the three examples you listed: they're all designed for a world when computing was a more solitary activity than it is now.
For Lisp and Smalltalk, image-based development is a genuine hurdle in the modern world. It complicates many aspects of modern highly-collaborative software development, including version control and continuous integration. I think that the language's respective hacker ethoses might also be a challenge in light of how often people switch jobs nowadays. I haven't used Smalltalk for pay money, but my experience has been that initially getting settled into a large pre-existing Lisp project does take more brain effort than it seems to with a language like Java. (In the long run I'd rather inherit a Lisp project, but I also acknowledge that first impressions are important.)
And BeOS was not a multiuser OS and didn't seem to have a clear path to becoming one. I'm pretty sure the main thing preventing it from becoming as much of a cybersecurity disaster as Windows 95 was is the simple fact that there wasn't any honey in that pot.
Image-based development was never good. So we pushed it back again and again, so that we nowadays have a single image component we named "database" and most of the work is free from its problems.
This largely occurs because the only real spec of rST is the Python implementation. And tools written in other languages are loath to include a Python library just to support one document format. While writing a new compatible implementation is much more work given the larger scope than it is for Markdown.
So every language has a markdown implementation, but if you want to use rst in your Rust based static site generator or your PHP based CMS or whatever, it's a lot more work.
Every language has its down markdown - but they are not compatible with each other unless you stick to the very basic which doesn't do much. Which is why every implementation has extensions - it isn't hard to add extensions and there are a lot of obviously missing things from the basic.
Yet at the same time I'm able to write Markdown that parses and renders correctly in VS Code, Obsidian, and the Rust-based static site generator I use. Sure, Obsidian has incompatible features I could use like [[Wiki Links]] but I don't have to.
That is easy enough as long as you are only writing a simple one page readme type document. To be fair that is the majority of what people write. However if you write anything more complex you will run into issues.
Is there any rich text format which is well-supported across multiple languages? The closest I can think of is HTML, but even that has its fair number of issues...
> markdown is a lightweight representation of html, while rst is a midweight representation of an abstract documentation tree
To me, the whole point of markdown is to allow you to do simple things faster than writing raw HTML, but with the ability to intersperse raw HTML when needed.
In projects where I've needed the power of RST over markdown, I've felt more comfortable just writing raw HTML directly.
That’s pretty much how I feel. When the author writes:
> This means you can extend Sphinx with new text objects! Say you that instead of an <image>, you want a <figure> with a <figcaption>. In basic markdown you have to manually insert the html
I can’t help but wonder what’s wrong with just writing HTML if you want those features? Why the extra layer?
The author wrote an entire article explaining their use case which answers your question. Short version: they’re writing a book which they want to render to multiple formats with semantic linking between items and different rendering based on formats.
I get that, but why not use HTML as the base to render to other formats? I guess I'm not 100% clear what you mean by semantic linking and how could that not be achieved through HTML.
If I understand you correctly, you want to throw out rST, and just have users write HTML directly? I see two issues with that:
1. HTML is designed for representing webpages, not books. Your intermediate representation will become HTML's DOM, which is likely not well suited to the book use case. You'll either need to use an existing HTML-to-DOM parser or build your own. One big limitation of the DOM, as I see it, is that it doesn't have the concept of a page number.
2. HTML is not extensible in the way the author talks about. This is the "semantic linking" thing I meant. In rST, you can define entirely new top-level constructs: he defines an exercise which has a solution and will be a part of a solution list. The semantic linking part is that other parts of the document will refer back to these solutions. Some output formats will have all solutions inline, with the exercises, while other output formats will collect all solutions in one place at the end. HTML is not flexible in that manner.
What is extensible is XML. It's entirely feasible for XML to be the input, but for better-or-worse, most people are even more turned off by XML than they are by something like rST.
> If I understand you correctly, you want to throw out rST, and just have users write HTML directly?
Yep!
> 1. HTML is designed for representing webpages, not books. Your intermediate representation will become HTML's DOM, which is likely not well suited to the book use case. You'll either need to use an existing HTML-to-DOM parser or build your own. One big limitation of the DOM, as I see it, is that it doesn't have the concept of a page number.
I guess my point is that while the main use of HTML indeed to render webpage, it seems pretty well suited at rendering most if not all types of documents. The page number issue you mention is typically only part of the presentation and is handled by the CSS paged media module (though admittedly, support of this isn't universal in my experience).
> 2. HTML is not extensible in the way the author talks about. [...]
Except it is! This is what HTML Custom Elements are all about. Combined with CSS, it can handled to different rendering formats you mention pretty well.
> What is extensible is XML. It's entirely feasible for XML to be the input [...]
I'd actually argue that raw XML is the wrong approach for most things, as the goal of XML is to be extended (e.g.: SVG, MathML, SAML, etc.). But I feel like that's a whole other discussion.
Anyways, I think this is a good conversation and wouldn't mind continuing over email (see my profile). It's fine if some people like rST, but HTML seems better suited for these use case than people make it out to be.
I'm less likely to continue this conversation over email. Such is life. :)
I think that bringing in CSS is a bridge too far for the case of writing a book. Authors are no longer just writing HTML, they're writing HTML and CSS. And CSS is intimately tied to actual web browsers - you're now going to need a browser display engine to be an integral part of how you render your book. (Or you're going to need to implement just enough of that yourself, which is a big undertaking.)
The point of comparison is Markdown, which is relatively straight-forward and simple. HTML and CSS is not that.
That's like saying why use C++ templates if you can just write the same function for every type that needs it. Now you don't need template instantiation!
The point is that it's objectively better when you want to separate the concerns of writing content and rendering it. It makes writing much easier and keeping the layout consistent as well.
Isn’t this just a case of different tools for different jobs (or the right tool for the right job)? Rst clearly has some advantages for this use case but, at least for me, markdown is much faster and clearer to understand when jotting down my daily notes.
Absolutely. I prefer rst for the use case of writing a technical book, because of the specific benefits I get from its additional complexity. My blog is still in markdown.
Having done a little of both, I find RST is faster just because I can lookup whatever syntax I need to keep going while with markdown I often stop to scream because I can't make it do what I want. Either is easy enough to understand when reading raw text - this is mostly about good style though, RST gives you more options for bad style that is hard to read.
> markdown is a lightweight representation of html
This is my main problem with the article, as this is decidedly incorrect.
Markdown was designed as a tool of conversion of de facto standard of formatting text in email messages and Usenet posts of early nineties. The basic 7-bit ASCII nature of those technologies inspired people to informally annotate the text with special signs to denote formatting like emphasis, section titles and the like.
True, HTML shares a lot of similarities with that nameless standard, so John Gruber in 2004 wrote a basic script [0] to convert the latter to the former, never expecting it to become such a universally used actual standard.
I disagree. Markdown has always been about HTML - to the point that Markdown parsers support dropping actual HTML tags mixed in the the markup.
The fact that Markdown was inspired by email conventions doesn't make the statement "markdown is a lightweight representation of html" any less correct.
I thought a goal of Markdown was to be a set of conventions that were readable on their own as well as being transformable to a rich text/HTML representation.
You can read a Markdown document pretty easily using "less" or any other character-based interface.
The same is not really true of reST.
Maybe I'm confusing that with some other plain-text markup convention?
reST is also designed to have human readable source. You might not like it compared to markdown but it has to serve as a the docstring format of Python and does so fairly well.
all of that is true, but markdown targets HTML as its "native backend", if you will. it is not feasible to render markdown -> pdf without having an intermediate HTML representation.
this is clear from the fact that HTML can be inlined in a markdown document; this is part of the spec (in so much as there even is a "spec" for markdown, anyway).
> to the point that Markdown parsers support dropping actual HTML tags mixed in the the markup.
Some Markdown parsers support some HTML. You cannot, for example, add a YouTube video embed to a GitHub comment. Webpages that allow Markdown quite reasonably limit what’s accepted.
Usually, the colloquial conflation of Markdown with CommonMark and the numerous Markdown-esque dialects is helpful language, I've defended it here before.
> Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML).
It's not possible to read the original specification without agreeing that, yes, Markdown is specifically a way to generate HTML. It also has a nice syntax which has become popular in contexts where embedding HTML isn't a good idea, and there are some tools which allow Markdown to be converted to other formats as well (usually, but not always, these disallow most HTML tags, meaning that they aren't Markdown sensu stricto either).
I would say that all of this supports Hillel's basic point here: Markdown is in fact tied to HTML semantics, in a way which makes it painful to do things with it which aren't tied to those semantics in the same way. Dialects in the Markdown family exist which make this less painful, and they do so by diverging from the original Markdown spec.
> Markdown qua Markdown, unambiguously supports HTML tags.
The section I quoted (and responded to) didn’t say “Markdown supports HTML tags”, it said (emphasis added) “Markdown parsers support (…)”.
So unless you’re arguing that a tool cannot be considered a true Markdown parser¹ unless it parses HTML, the distinction is irrelevant. I wasn’t making a general argument, I was responding to a specific claim.
> You can verify that yourself
Yes, I am quite familiar with that subpar piece of code and specification. I have spent some time implementing a Markdown parser myself (the output wasn’t HTML) and Gruber’s resources were by far the worst. For various reasons he has been an awful steward of Markdown; its ubiquity is despite him, not because of. Forgive me if I don’t find the author to be the authoritative source anymore.
I think you’ve kinda lost the thread. From the top comment:
> Markdown was designed…
Surely Gruber is an authority on that.
And anyway, “ markdown is a lightweight representation of html” is true enough that calling it “decidedly incorrect” is wrong. Certainly it’s not a point that’s supported by going to author intent.
The very first sentence in that link says “Markdown is a text-to-HTML conversion tool for web writers.”
Gruber did not take a "de facto standard" from Usenet and simply write an HTML converter for it. He designed his own markup, borrowing from Usenet and other conventions. The “Acknowledgements” section at the bottom of your link alludes to this fact.
Markdown was from the beginning intended as a markup syntax for web CMSs. To say it is a lightweight representation of HTML is correct. The whole point was that every part of the syntax produce a direct HTML equivalent.
Gruber writes that Markdown is two things: (1) a plain text formatting syntax; and (2) a software tool, written in Perl, that converts the plain text formatting to HTML.
> To say it is a lightweight representation of HTML is correct.
It's correct in the context of (2) but not (1). In the context of (1), Markdown is a lightweight markup language for rich text. For example, Pandoc can convert Markdown to formats like PDF, LaTeX, and ePub.
It's also true in the context of (1), since the syntax specification specifically falls back to HTML for difficult things like tables.
This is made abundantly clear on the Syntax page[0]:
> Markdown’s syntax is intended for one purpose: to be used as a format for writing for the web.
> Markdown is not a replacement for HTML, or even close to it. Its syntax is very small, corresponding only to a very small subset of HTML tags. The idea is not to create a syntax that makes it easier to insert HTML tags. In my opinion, HTML tags are already easy to insert. The idea for Markdown is to make it easy to read, write, and edit prose. HTML is a publishing format; Markdown is a writing format. Thus, Markdown’s formatting syntax only addresses issues that can be conveyed in plain text.
> For any markup that is not covered by Markdown’s syntax, you simply use HTML itself. There’s no need to preface it or delimit it to indicate that you’re switching from Markdown to HTML; you just use the tags.
Markdown is very coupled to HTML, and is hobbled as a result compared to formats like Tiddlywiki and Org that not only have sophisticated syntax for all kinds of content like tables, but also directives to the parser itself to render particular blocks using an external tool. I wrote a bit about this in 2021 in response to query about why Org mode doesn't use Markdown[1].
> Markdown was from the beginning intended as a markup syntax for web CMSs.
As a matter of history you're right, just as JSON was intended as nothing more than the literal syntax for JavaScript programs. But it turned out to be a lot more widely applicable.
> To say it is a lightweight representation of HTML is correct.
No, it's a representation of writing. It represents the input not the output.
The first few years after I released Markdown, it was far less popular than I expected and hoped it would be. But it grew, and then for a years it was as popular as I’d expected. But then it kept growing, and now it’s far more popular and widely used than I’d ever thought possible. I genuinely now think it’s used in many cases where it shouldn’t be, and exposed to non-technical users who should be using something WYSIWYG, not markup.
Markdown, Markdown, is nothing to do with email and Usenet formatting. It was a specific syntax, poorly defined, which has since become a broad family of mostly similar syntaxes; and it was inspired by various conventions from things like email and Usenet, some of which predate computers (e.g. pretty sure I’ve seen or heard of asterisks being used for italics in old typewritten stuff); but Markdown is all about HTML, its syntax is extremely constrained by HTML, and any attempts to separate it from HTML are fairly thoroughly doomed.
Saying that it has nothing to do with email/usenet is a bridge too far in the other direction, it was very much inspired by such formatting, something it never hid: https://daringfireball.net/projects/markdown/
> the single biggest source of inspiration for Markdown’s syntax is the format of plain text email
Yeah, it was inspired by those things, as I said, but it was never connected to them, and is quite incompatible with them.
I also express things that strongly because I’ve observed a distinct tendency to interpret Markdown as the successor of those things or the inventor of such syntax ideas. Whereas in fact, Markdown was not the first lightweight markup language, and is not the only one, and isn’t even the LML unquestionably closest to plaintext email conventions (reStructuredText is an obvious contender; overall, I’d consider reStructuredText a little closer than Markdown, but it’s subjective).
Perhaps my intent would be more clearly expressed by inverting it: email formatting has nothing to do with Markdown.
Dear HN: stop leaving comments like this. It's arguing over semantics and makes for boring conversation. It also violates HN guidelines for leaving comments:
- Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
- Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead.
If you disagree with the substance of the article, say you prefer Markdown to rST, then address that. Explain why despite the author's preference for rST, you prefer MD. It's silly to argue over this single sentence about exactly what Markdown is or isn't.
Yeah, that's what I feel gives markdown an edge over nearly anything else; markdown is adopting conventions that were meant to be readable without any kind of rendering. I can very easily tell what my markdown document is going to look like before I render it with Pandoc, because it's basically just taking stuff people were using with plain text that was meant to be read directly anyway.
I haven't used rST, but looking at the examples it looks considerably less readable out of the box.
* one
* two
* buckle my shoe
| inspiring quote
| - some famous guy
foo
a kind of bar
bar
part of a foo
:red: stop
:yellow: speed up
:green: go
- code:: python
def fun():
pass
Go here_ for more info.
- :: here: https://google.com
My **strong** text is well *emphasized*. Very ``literally``.
I've started using MyST for my more recent projects, which gives me the features I care about from reStructuredText - references and table of contents - but lets me use markdown syntax, which is easier for contributors.
Great points about links, especially external links — it makes a lot more sense in the context of a documentation site where you might have the same external link referenced multiple times in the site, but you only want to have to update it once if it changes.
The real gamechanger is for internal links (rST+Sphinx) and the use of the `:ref:` and `:doc:` directives so that you can reference an anchor or a doc link in the same content without having to type the header manually (which will inevitably go stale).
Lemons vs limes. They're both markup languages, but they have different purposes.
Yeah, using markdown to write a book is very limiting and doesn't offer the control needed for typesetting.
OTOH, using markdown to write is liberating - you only get a few markup options, so you can focus on the content:
[It is] notable that the Feynman lectures (3 volumes) write about all of
physics in 1800 pages, using only 2 levels of hierarchical headings:
chapters and A-level heads in the text. It also uses the methodology of
sentences which then cumulate sequentially into paragraphs, rather than the
grunts of bullet points. Undergraduate Caltech physics is very complicated
material, but it didn’t require an elaborate hierarchy to organize.
Edward Tufte, forum post, ‘Book design: advice and examples’ thread
Any sufficiently complicated lightweight markup format contains an ad-hoc, informally-specified, bug-ridden, and only slightly easier to use implementation of half of either DocBook or TeX.
I just wish it would move along already. I get it, some tough calls need to be made on edge cases that do not have a correct answer. Yet I want a 1.0 release so I can feel confident that the language will have staying power.
Then again, it is already in pandoc, so I suppose you can trivially move out of it.
To write your "exercise extension" in Scroll, you would just need to write a few lines of code definining an "exerciseParser". You could put that in the same file you use it in, or in a file named "exercise.parsers".
I think you should explore Scroll -- https://scroll.pub/ -- might do everything you need in a simpler way!
Yes, you could do that. At some point soon (perhaps this already works today), you could probably just copy/paste the entire Scroll Parsers code (https://try.scroll.pub/scroll.parsers) into a ChatGpt or Claude.ai prompt and ask it to write one.
The problem with RST is the same with everything else: it isn't supported everywhere. I wrote a bunch of nice RST documentation at work, the next month some company wide team announced a new standard for documentation (backstage) which is markdown based, and has all the problems of markdown. (In their defense I'm not aware of any tool that would compete with backstage for what we need in a company wide documentation system)
>The most important difference between rst and markdown is that markdown is a lightweight representation of html, while rst is a midweight representation of an abstract documentation tree.
>Its key design goal was readability, that the language be readable as-is, without looking like it has been marked up with tags or formatting instructions, unlike text formatted with 'heavier' markup languages, such as RTF, HTML, or even wikitext
So, for use cases that don’t make a good match of this chief constraint, Markdown might not be the most relevant option. It’s just as meaningful as saying that we prefer TeX over Markdown because fine typographical tuning is an important concern of our specific usecase.
I admit I already dealt with a screw using a hammer, not having the right toolset at hand. But when the situation don’t call for adhoc heterodox actions, I prefer to match a screwdriver with a screw and a hammer with a nail.
... you can extend Sphinx with new text objects!
Say you that instead of an <image>, you want
a <figure> with a <figcaption> ...
What I would do:
[caption]
And write a renderer which supports this syntax to create a figure with a figcaption. A markdown renderer is simple enough to write your own or extend an existing one.
Exactly - does anyone (well, maybe the author does since he did use markdown in previous works) use "basic" markdown anyway? There are always some extensions (either front-text metadata or syntax extensions like this one) in the "interesting" apps. (The python markdown module API kind of feels like it's entirely about extension and only a little bit about actually parsing, but that did make it easy to add an `ASIN()` token for a review site without needing a preprocessing stage...) That does make it a little worse for interchange, but that's maybe less important for markdown anyway.
(Even with pandoc, https://pandoc.org/MANUAL.html#extensions lists a bunch of syntax for things like this, brought in from github, php, and other places - for the example above, `implicit_figures` and `link_attributes` are a good starting point.)
Honestly I stopped reading article after this example. No, thank you. RST syntax looks confusing and overly explicit. I'd stick with MD thankyourverymuch.
I think commenters are missing that the author is talking about this in the context of typesetting their own book. They aren't claiming that rST is better than Markdown in the general case; in general, Markdown's simplicity is a reason why it is so widely used. But that's not what the author is talking about.
Well, the simple mental hack for yourself is just to prepend all blog content, if not all content in general, with
#include <general_disclaimers>
and you'll find yourself much happier in life.
Speaking as one who actually produces blogs on rare occasions, I have to say starting every post with "This is just my experience and may not apply to you this is not a claim of global applicability this is just applicable to this circumstance you're the one who chose to read this I didn't force you" and so on isn't exactly the most compelling way to open a post.
Though I have to admit now I'm really tempted to add that into my post template in like 2pt font or something, just so when people on HN complain I can point them at it.
But that's all beside the point - we're not talking about a "general disclaimer." (I am intrigued by how blurry the lens you look at the world must be, if you live by that "#include <general_disclaimers>" nonsense) People interested in the blogger's rather specific Markdown use case would be better served by a more specific title, as would the nine out of ten Markdown users who are not.
No, I was just recommending it to you since you seem to be perturbed. I just give people a general measure of grace in their writing and it all seems to work out fairly well. You can try that as well, if you like.
Your comment was quite condescending, and it's disingenuous of you to suggest otherwise. If this is all your idea of grace, I'd hate to see less than graceful.
By contrast, I was careful to criticize the title as clickbait, not the post, which is useful (or the book, which sounds genuinely interesting). I wasn't perturbed by the title. By your comments, sure, but that's your doing, not Hillel's.
If the title "Why I prefer rST to markdown" (which a 100% accurate summary of the content) is somehow "clickbait", then the term "clickbait" has been redefined into meaningless.
Since it's a clearer statement of the author's meaning, and since very few users interested in Markdown are going to be doing any typesetting, it seems like a more meaningful title to me.
But less likely to draw in readers... to "bait" them, if you will.
I use Markdown constantly, every single day, and I have tried multiple times to switch to RST and failed. Ultimately the reason Markdown works the best for me is that it gets out of my way. I can just type, I don't have to invest significant effort in formatting structures and can just write. This makes Markdown ideal for my primary use case, which is note-taking live during meetings, and because I already use it there, I also use it for writing blog posts, documentation, and most other document writing purposes. I will freely acknowledge RST is better for writing documentation and formal structured documentation, but the learning curve/barrier for RST is enough higher than Markdown, that it's not worth it for me to use both systems.
I believe the big selling point is supposed to be the directives syntax of rST. I had a similar need to extend markdown, but pandoc already supports directive syntax -- and so do some libraries, eg., the marked library, which I use to render MD as HTML in-browser rather than with pandoc.
The iirc, syntax is :inline{property} and
::: block-directive {options}
contents
:::
It doesnt nest well, so I applied any document-level styles using the filename; but I'd imagine the yaml-style lead-in tags should be the correct way to do it.
Context: I've been a technical writer for ~12 years. I actually started out my career migrating a startup's docs from Word to Sphinx. Then I did a lot of time on Google's proprietary CMS/platform for developer docs. Then a few years on Eleventy-based sites. For the last two years I've been back on a Sphinx-based site (pigweed.dev). I've also done some odd jobs for readme.com-based startups and have dabbled in Docusaurus, Astro, and Hugo.
reStructuredText (reST) by itself can be pretty rough. reST combined with Sphinx is pretty great. I.e. the strengths of Sphinx far outweigh the weaknesses of reST. For big, professional docs sites (+100 pages, +10 contributors) I have pretty strong opinions that Sphinx is probably the most responsible choice, long-term.
> The most important difference between rst and markdown is that markdown is a lightweight representation of html, while rst is a midweight representation of an abstract documentation tree.
Yes, what this means in practice is that it's easy to customize common aspects of your site (such as how images are displayed). A teammate of mine on Pigweed recently created a shortcut for linking to bugs consistently. You type :bug:`59385981` and the link gets transformed into https://pwbug.dev/59385981. If we ever need to mass-migrate all of our bug links, it's now trivial. Another profoundly important consequence of this architecture: all internal links are guaranteed to always resolve (you get warnings/errors if you link somewhere that doesn't exist). I previously wrote about how baffling it is that this is not the industry standard for docs sites: https://technicalwriting.dev/src/link-text-automation.html
The other thing that Sphinx excels at: well-defined extension and theme APIs. It's not exactly easy to build extensions or themes, but lots of people have figured it out (including me). There's a pretty rich ecosystem of extensions and themes on PyPI.
Lately I've been calling Sphinx the sleeping giant of docs systems. It's already pretty great and with a little concerted effort I think we can collectively make it absolutely phenomenal. The upstream Sphinx repo now has a GitHub Discussions section that gets a little traffic; a lot of us seem to congregrate on the #sphinx channel in the Write The Docs Slack.
Agreed that Sphinx is extraordinary and wildly underrated. Sphinx is the only architecturally sound, extensible, widely used documentation framework that I know of. Its plugin ecosystem is amazing and gives me incredible leverage to improve documentation for teams and projects.
I'm not a fan of reStructuredText, but nowadays it's possible to do most things that previously tightly coupled Sphinx to RST in Markdown, courtesy of MyST-Parser: https://github.com/executablebooks/MyST-Parser
Except that the plugins are not compatible with each other or the latest version. That means I'm seeing some bugs that are fixed in the latest but I can't get the fix because my doxygen plugin doesn't work with the latest, and even if it did is the mermaid plugin updated yet? (please don't ask which versions of each I'm using - I believe there are at least 2 different ones that can do both of the above and they all have different abilities...)
Some drive-by feedback on pigweed.dev: It seems like it’s missing a “What is Pigweed” section? I was wondering what Pigweed does and eventually found the Github README, which is somewhat better. (I’ve used Arduino, but I’m not quite sure what I’m looking at.)
Thanks. It's basically middleware for large, professional embedded development teams creating mass market products, usually consumer electronics. The four pillars in the "What is Pigweed?" section on the homepage [1] really do sum up a lot of what we do, but the reality is that it's a difficult "product" to explain in general. But rest assured that it's definitely on my radar that there is huge room for improvement in our "What is Pigweed?" content.
Apparently I'm bad at skimming. I did see that section, though apparently not the heading :)
The first part didn't really do it for me because it seemed too meta:
"Over 150 libraries" - to do what? (Actually, some good examples are in the next section.)
"Automation" - to build, test, and lint what?
"Environments" - to make builds reproducible, to do what?
"Frameworks" - turnkey solutions for what?
They are clickable, though, so maybe I should have clicked. :)
> We provide libraries (modules) for strings, time, assertions, logging, serial communication, remote procedure calls (RPC), and much more.
Maybe put that up front?
Looks like "frameworks" are still work in progress; maybe I'll check back someday when they get to Raspberry Pi Pico support.
Why do you like Sphinx so much? Have you ever used MkDocs/Material for MkDocs? How do they compare?
At my last job I spent a fair amount of time on documentation, but I don't have a great understanding of what technical writers like/dislike, or what documentation systems are best.
I like sphinx because I can make it do all the weird links between documents that I want. Markdown can't do that. I've never used MkDocs so I cannot comment.
I've used both extensively. (Material for) MkDocs is easy to set up and gives very good UX for users. Sphinx is hard to set up and provides better UX in some ways (internal links are on another level entirely), and worse UX in others (the themes are all worse than Material for MkDocs, every single one).
> reStructuredText (reST) by itself can be pretty rough. reST combined with Sphinx is pretty great.
So, you've made a convincing argument of why we should prefer Markdown to reST, 9 times out of 10. If we're working with Sphinx we'll give it serious consideration.
I heard recently that MyST [1] now has full feature parity with everything that reST provides. If I were starting a new project today I would seriously look into Markdown-based Sphinx site powered through MyST. For Pigweed we collectively don't seem to hate reST enough to justify the huge project of migrating everything over from reST to Markdown.
So yeah, I don't really have strong opinions on sticking with rest per so. I only have strong opinions about Sphinx being probably the best choice out there for large docs sites.
Yes, what this means in practice is that it's easy to customize common aspects of your site (such as how images are displayed).
I found this to be very easy using Markdown+pandoc. I turned image tags containing youtube links to thumbnails with video tag and alt text, hooked up image tags to local video files to ffmpeg to optimize and resize videos to then do the same, etc in a few lines of code.
> Another profoundly important consequence of this architecture: all internal links are guaranteed to always resolve (you get warnings/errors if you link somewhere that doesn't exist). I previously wrote about how baffling it is that this is not the industry standard for docs sites: https://technicalwriting.dev/src/link-text-automation.html
This, so much this.
It's a problem in an even more general form (for you're writing about linking to sections within a page): the number of CMS systems or static site builders that make you insert the final URL when you write is _insane_.
If the slug changes, you reorganise your site - now you have to do a search and replace through the site to update all your links.
Static site generators could enable linking like [Hello](../hello.md) and resolve the link at build time, instead of expecting everyone to type [Hello](/why/hello/). And yet the popular ones I've used or looked at don't do this.
It seems to be a marmite feature (you love it or you don't): I've raised it with static site builder team members and received a "Why would you want that?" response. Explanations got nowhere. I'm not sure if it's a niche feature, you have to have experienced the problem to appreciate the solution, or people are used to writing once and not maintaining for a decade or longer, but it'd be great to see wider support.
I liked it as well. But many years ago I tried to get them to implement parts of a book, and they never did. You can work around somewhat by hardcoding chapter numbers—but I don’t feel sphinx is suitable for book-length docs without that simple feature.
If you want Sphinx to achieve that breakout success, priority #1 should be to get high-quality, beautiful themes.
At least for me, aesthetics are a high-priority factor when deciding which site generator to use. Hugo and Gatsby have great built-in themes, and I have chosen them for projects before based largely on this fact.
The Sphinx themes here [0] and here [1] are various levels of "meh" to bland.
Apple's site loads for me in under 1 second with a clean cache force-reload.
The featured themes, in general, have poor typography and whitespace, poor formatting and alignment, and uninspired colors and design. Furo [0] is the best-designed of them and would make a solid foundation for a proper theme.
I don't disagree that themes are an important part of marketing a tool. The engineer in me doesn't want that to be the case, but I recognise that people like pretty things, it makes a difference.
However I'd push back a little. That Fluent UI doc site looks like every other doc site from an open source project started since ~2020 that I've seen, and this means that I have certain expectations: I expect the project to be immature, I expect the docs to be incomplete or out of date, I expect the search to be an annoying AI search that does a worse job than the basic full text search that I expect elsewhere, I expect that to get anything of substance done I'm going to have to hang out on a Discord server. When I see a ReadTheDocs site though with that default theme, I expect the project to be somewhat mature, I expect it to basically work. I expect the docs to be fairly complete.
Are these expectations fair? Not really, but that's the whole point here. Open Source projects are not immune from marketing, and when marketing, what other similar projects do will affect how people respond to your marketing. Nothing exists in isolation.
In my experience, engineers think marketing doesn't apply to them, but these impressions really do matter.
I agree that more stellar, super-visually-appealing themes will help Sphinx get to the next level.
One nit: I called Sphinx a sleeping giant because it's already majorly successful in the Python ecosystem. Pretty much all big Python docs sites are built on top of Sphinx, including docs.python.org. But I think it's fair to say that when Sphinx starts getting major adoption outside of the Python ecosystem then we can start calling it a breakout success.
I did not know about Sphinx until your comment (I'll get to my 2-day delay in response, briefly). I've been doing technical writing "on the side" as my dev work allows, for 20+ years. I've been a TeX/custom-XSL sort of dude the whole time.
I just ported a 200+ page book documenting an internal language/VM/abstraction-layer to Sphinx ... holy moly, this is a life changing system!
I wish the documentation for Sphinx, itself, had a lower ramp and/or more examples; otherwise, I'm still in a pretty fantastic honeymoon phase.
My two main lines of research are how to build good looking PDF (books); and, some sort of system to let me cut a book into POSIX compliant man pages by chapter/section.
When you call a book v0.2 and say "now with epub, get it here" in the first line of your blog on why not to use the most popular of a thing, I am assuming we are going to be having a above board conversation about pros and cons, and I an receptive to your position.
When the first link is also the second link, and both are to BUY a 15 dollar book with the default "pay anything price" bar set to 20, I am just clicking out and ignoring literally everything in your marketing post.
Others can discuss this RST thing I have never heard about, I am just here to point out you have now alienated one of your potential converts. Put the book link at the bottom, and dont double link to the same purchase page in the same paragraph. FFS.
I keep my journal in a blend of asciidoc and markdown.
In asciidoc, I love xref soooo much, which lets you make cross-document links to .adoc files that still work after being translated to html.
But I'm so sad that asciidoc isn't supported by pandoc.
The thing I like about markdown is that it's very readable as just a raw format. With the bonus that it's easily converted to HTML. But it doesn't need to be converted to HTML.
I guess this sort of falls under the "But I hate the syntax" section of the article, but I don't think it quite gets the point. I can look at a Markdown file and read or write it without even really thinking. Which is not really the case with rST (or even HTML, for that matter).
I can look at a rST file and read it, and I think you will have no problem reading every rST file I've ever looked at. Sure it will take you a few seconds to realize you can ignore various syntax things, but that isn't a big deal and then the whole makes sense. Of course rST gives a lot more options to write documents that are not human readable, but it isn't that hard to make them readable.
426 comments
[ 3.2 ms ] story [ 326 ms ] threadTwo problems I had were: (1) common with a lot of systems, the RST tools don't contain an RST unparser. I really wanted to be able to generate RST files automatically by merging content from other RST files and other sources and working with them via a document API. The RST tools don't support that. (2) The RST tools expect a certain defined set of blocks for a particular document; it would be very possible for a system to represent blocks in a generic way which would make possible tools that can transform documents without necessarily knowing the definitions of blocks inside it.
These are problems w/ the tools, not problems with RST itself but every time I'm forced to strip my code back to the walls is an opportunity to think about some other markup system such as something HTML-based.
In python, rST is just one of many supported input formats for docutils: https://docutils.sourceforge.io/README.html#purpose
The entire point docutils is to parse formats and convert using an API: https://www.docutils.org/docs/index.html#api-reference-mater...
If it had that it would be a 200 foot tall giant robot, as it is it's just another documentation generator.
Instead of HTML you can use XML to write structured documents. With XML you can define whatever custom tags you need with schema validation if desired. The advantage of this approach is you have full control over the input schema and output, but the downside is XML is much more syntactically noisy compared to Markdown or RST and you will need a script to parse & convert the XML to your preferred output format(s).
XSLT will happily output any number of formats from an XML doc. There's also a number of existing and very capable XML schemas for documentation like DocBook. You can even go from a lightweight markup like RST or Markdown to DocBook and then have an XSLT pipeline that delivers that intermediary to any number of formats. There's also very capable graphical editors that will spit out DocBook directly.
I find the aversion to XML very strange. It was definitely applied in some places poorly by products with "Enterprise" somewhere in the name but the language itself is very useful. The aversion to XML has led to a lot of work re-learning of the lessons that led to some XML features. It's also led to implementing XML features poorly in other languages.
Loads of other systems I work with still output XML, and I have fast, powerful tools to work with them pre-installed on any Mac I work on, and it doesn't even matter that Apple hasn't updated them in a while since XML hasn't really been changed either. I can look at an XML doc and have so much information about the data inside just telegraphed into my head, just by the way the things are shaped. I don't have to read the tags like a book, I can just gloss over them and still get enough to understand the strings/numbers inside. Yet, they're still on the page, out of focus, in my periphery, providing anchoring and structure and navigation, I can find my place. Maybe I'm just a freak, or that comes from 20 years of staring at them. Whatever reason, I just can't do that with JSON as easily or as fast.
I don't hate on any of the other formats (save for YAML, not interested), and I really appreciate JSON's brevity sometimes for simple data and small responses, but that brevity quickly turns into wisps of formatting and text seemingly floating out in the middle of nowhere.
Same. Every back end project I work on I recommend XML as the transport and every time I get a reflexive "ugh, XML". Then I spend an inordinate amount of time making Swagger endpoints so a consumer can figure out how the JSON output is supposed to behave.
It really isn't. Both namespaces and schema cause far more problems than they solve. And the theoretical advantages of having a declarative language for expressing transformations (and schema) is outweighed by how awful it actually is to write XSLT (and schema).
If we had namespaces for classes and id's in CSS we'd be able to transclude content from one HTML to another HTML page easily but we don't so we can't. (Shadow DOM helps a bit though) The "hygenic macro problem" is one of the many barriers people face doing metaprogramming. In the RDF world it's refreshing to be able to import facts from multiple systems into multiple namespaces so you can put data you got from different sources into the same database and "it just works".
I don't think namespaces solve the macro hygiene problem, because if you're going to have macros then they need to be able to interact with and manipulate your namespaces, and sooner or later you need your macro to e.g. generate a fresh identifier for use within an existing namespace and then you're right back where you started.
Being able to disambiguate when you have one document including another would be useful. But the cost/benefit on XML-style big namespacing up-front just doesn't stack up.
(I kid, mostly. It was a mild pain in the projects I used it for, and I've heard much worse horror stories than mine.)
That is, XSLT is based on pattern matching rules that most people find strange and unfamiliar. It wouldn't be so strange to people if this kind of system was more widespread, but as an island that's different from everything else I think people struggle to wrap their heads around it.
- is a superset of XML (XML is derived from SGML as a simplified, proper subset)
- can parse HTML precisely with all bells and whistles such as tag omission, enumerated attributes and other shortforms (which are based on SGML after all)
- can define markdown syntax as an SGML SHORTREF customization (note that SGML can not cover all of markdown syntax; for example, defining reference links somewhere at the end of a document and expecting SGML to pull a href URL into the place where the link is referenced won't work with SGML SHORTREF)
The combination of markdown-as-sgml and inline HTML or HTML blocks is particularly nice and predictable, and also informs further customization such as using entities (text variables), custom elements, and other advanced SGML stuff with markdown that comes up frequently as a requirement.
I'm reminded of all of the alternate worlds that didn't quite come to fruition, but could have (Lisp, Smalltalk, BeOS, etc.).
For Lisp and Smalltalk, image-based development is a genuine hurdle in the modern world. It complicates many aspects of modern highly-collaborative software development, including version control and continuous integration. I think that the language's respective hacker ethoses might also be a challenge in light of how often people switch jobs nowadays. I haven't used Smalltalk for pay money, but my experience has been that initially getting settled into a large pre-existing Lisp project does take more brain effort than it seems to with a language like Java. (In the long run I'd rather inherit a Lisp project, but I also acknowledge that first impressions are important.)
And BeOS was not a multiuser OS and didn't seem to have a clear path to becoming one. I'm pretty sure the main thing preventing it from becoming as much of a cybersecurity disaster as Windows 95 was is the simple fact that there wasn't any honey in that pot.
That doesn't mean we stopped doing it.
So every language has a markdown implementation, but if you want to use rst in your Rust based static site generator or your PHP based CMS or whatever, it's a lot more work.
And then people don't create any new format for its use case because it already exists.
Almost everything good we have only exists because some person irrationally decided to do it, against all common sense and logic.
To me, the whole point of markdown is to allow you to do simple things faster than writing raw HTML, but with the ability to intersperse raw HTML when needed.
In projects where I've needed the power of RST over markdown, I've felt more comfortable just writing raw HTML directly.
> This means you can extend Sphinx with new text objects! Say you that instead of an <image>, you want a <figure> with a <figcaption>. In basic markdown you have to manually insert the html
I can’t help but wonder what’s wrong with just writing HTML if you want those features? Why the extra layer?
1. HTML is designed for representing webpages, not books. Your intermediate representation will become HTML's DOM, which is likely not well suited to the book use case. You'll either need to use an existing HTML-to-DOM parser or build your own. One big limitation of the DOM, as I see it, is that it doesn't have the concept of a page number.
2. HTML is not extensible in the way the author talks about. This is the "semantic linking" thing I meant. In rST, you can define entirely new top-level constructs: he defines an exercise which has a solution and will be a part of a solution list. The semantic linking part is that other parts of the document will refer back to these solutions. Some output formats will have all solutions inline, with the exercises, while other output formats will collect all solutions in one place at the end. HTML is not flexible in that manner.
What is extensible is XML. It's entirely feasible for XML to be the input, but for better-or-worse, most people are even more turned off by XML than they are by something like rST.
Yep!
> 1. HTML is designed for representing webpages, not books. Your intermediate representation will become HTML's DOM, which is likely not well suited to the book use case. You'll either need to use an existing HTML-to-DOM parser or build your own. One big limitation of the DOM, as I see it, is that it doesn't have the concept of a page number.
I guess my point is that while the main use of HTML indeed to render webpage, it seems pretty well suited at rendering most if not all types of documents. The page number issue you mention is typically only part of the presentation and is handled by the CSS paged media module (though admittedly, support of this isn't universal in my experience).
> 2. HTML is not extensible in the way the author talks about. [...]
Except it is! This is what HTML Custom Elements are all about. Combined with CSS, it can handled to different rendering formats you mention pretty well.
> What is extensible is XML. It's entirely feasible for XML to be the input [...]
I'd actually argue that raw XML is the wrong approach for most things, as the goal of XML is to be extended (e.g.: SVG, MathML, SAML, etc.). But I feel like that's a whole other discussion.
Anyways, I think this is a good conversation and wouldn't mind continuing over email (see my profile). It's fine if some people like rST, but HTML seems better suited for these use case than people make it out to be.
I think that bringing in CSS is a bridge too far for the case of writing a book. Authors are no longer just writing HTML, they're writing HTML and CSS. And CSS is intimately tied to actual web browsers - you're now going to need a browser display engine to be an integral part of how you render your book. (Or you're going to need to implement just enough of that yourself, which is a big undertaking.)
The point of comparison is Markdown, which is relatively straight-forward and simple. HTML and CSS is not that.
The point is that it's objectively better when you want to separate the concerns of writing content and rendering it. It makes writing much easier and keeping the layout consistent as well.
This is my main problem with the article, as this is decidedly incorrect.
Markdown was designed as a tool of conversion of de facto standard of formatting text in email messages and Usenet posts of early nineties. The basic 7-bit ASCII nature of those technologies inspired people to informally annotate the text with special signs to denote formatting like emphasis, section titles and the like.
True, HTML shares a lot of similarities with that nameless standard, so John Gruber in 2004 wrote a basic script [0] to convert the latter to the former, never expecting it to become such a universally used actual standard.
[0] https://daringfireball.net/projects/markdown/
The fact that Markdown was inspired by email conventions doesn't make the statement "markdown is a lightweight representation of html" any less correct.
You can read a Markdown document pretty easily using "less" or any other character-based interface.
The same is not really true of reST.
Maybe I'm confusing that with some other plain-text markup convention?
this is clear from the fact that HTML can be inlined in a markdown document; this is part of the spec (in so much as there even is a "spec" for markdown, anyway).
Some Markdown parsers support some HTML. You cannot, for example, add a YouTube video embed to a GitHub comment. Webpages that allow Markdown quite reasonably limit what’s accepted.
On this question, it isn't.
Markdown qua Markdown, unambiguously supports HTML tags. You can verify that yourself: https://daringfireball.net/projects/markdown/
The introduction being:
> Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML).
It's not possible to read the original specification without agreeing that, yes, Markdown is specifically a way to generate HTML. It also has a nice syntax which has become popular in contexts where embedding HTML isn't a good idea, and there are some tools which allow Markdown to be converted to other formats as well (usually, but not always, these disallow most HTML tags, meaning that they aren't Markdown sensu stricto either).
I would say that all of this supports Hillel's basic point here: Markdown is in fact tied to HTML semantics, in a way which makes it painful to do things with it which aren't tied to those semantics in the same way. Dialects in the Markdown family exist which make this less painful, and they do so by diverging from the original Markdown spec.
The section I quoted (and responded to) didn’t say “Markdown supports HTML tags”, it said (emphasis added) “Markdown parsers support (…)”.
So unless you’re arguing that a tool cannot be considered a true Markdown parser¹ unless it parses HTML, the distinction is irrelevant. I wasn’t making a general argument, I was responding to a specific claim.
> You can verify that yourself
Yes, I am quite familiar with that subpar piece of code and specification. I have spent some time implementing a Markdown parser myself (the output wasn’t HTML) and Gruber’s resources were by far the worst. For various reasons he has been an awful steward of Markdown; its ubiquity is despite him, not because of. Forgive me if I don’t find the author to be the authoritative source anymore.
¹ https://en.wikipedia.org/wiki/No_true_Scotsman
> Markdown was designed…
Surely Gruber is an authority on that.
And anyway, “ markdown is a lightweight representation of html” is true enough that calling it “decidedly incorrect” is wrong. Certainly it’s not a point that’s supported by going to author intent.
Yes, I completely agree with that.
And that wasn’t the point I quoted or responded to.
> From the top comment
Again, I wasn’t replying to the top comment, I was replying to what I quoted, from the comment I replied to.
> Markdown’s syntax is intended for one purpose: to be used as a format for writing for the web.
Both Gruber's original and CommonMark (the closest thing to a standard) support inline HTML.
Both Gruber and CommonMark specs are defined in terms of its HTML output.
Gruber did not take a "de facto standard" from Usenet and simply write an HTML converter for it. He designed his own markup, borrowing from Usenet and other conventions. The “Acknowledgements” section at the bottom of your link alludes to this fact.
Markdown was from the beginning intended as a markup syntax for web CMSs. To say it is a lightweight representation of HTML is correct. The whole point was that every part of the syntax produce a direct HTML equivalent.
> To say it is a lightweight representation of HTML is correct.
It's correct in the context of (2) but not (1). In the context of (1), Markdown is a lightweight markup language for rich text. For example, Pandoc can convert Markdown to formats like PDF, LaTeX, and ePub.
This is made abundantly clear on the Syntax page[0]:
> Markdown’s syntax is intended for one purpose: to be used as a format for writing for the web.
> Markdown is not a replacement for HTML, or even close to it. Its syntax is very small, corresponding only to a very small subset of HTML tags. The idea is not to create a syntax that makes it easier to insert HTML tags. In my opinion, HTML tags are already easy to insert. The idea for Markdown is to make it easy to read, write, and edit prose. HTML is a publishing format; Markdown is a writing format. Thus, Markdown’s formatting syntax only addresses issues that can be conveyed in plain text.
> For any markup that is not covered by Markdown’s syntax, you simply use HTML itself. There’s no need to preface it or delimit it to indicate that you’re switching from Markdown to HTML; you just use the tags.
Markdown is very coupled to HTML, and is hobbled as a result compared to formats like Tiddlywiki and Org that not only have sophisticated syntax for all kinds of content like tables, but also directives to the parser itself to render particular blocks using an external tool. I wrote a bit about this in 2021 in response to query about why Org mode doesn't use Markdown[1].
[0]: https://daringfireball.net/projects/markdown/syntax
[1]: https://rpdillon.net/why-doesnt-emacs-org-mode-just-use-mark...
As a matter of history you're right, just as JSON was intended as nothing more than the literal syntax for JavaScript programs. But it turned out to be a lot more widely applicable.
> To say it is a lightweight representation of HTML is correct.
No, it's a representation of writing. It represents the input not the output.
The first few years after I released Markdown, it was far less popular than I expected and hoped it would be. But it grew, and then for a years it was as popular as I’d expected. But then it kept growing, and now it’s far more popular and widely used than I’d ever thought possible. I genuinely now think it’s used in many cases where it shouldn’t be, and exposed to non-technical users who should be using something WYSIWYG, not markup.
> the single biggest source of inspiration for Markdown’s syntax is the format of plain text email
I also express things that strongly because I’ve observed a distinct tendency to interpret Markdown as the successor of those things or the inventor of such syntax ideas. Whereas in fact, Markdown was not the first lightweight markup language, and is not the only one, and isn’t even the LML unquestionably closest to plaintext email conventions (reStructuredText is an obvious contender; overall, I’d consider reStructuredText a little closer than Markdown, but it’s subjective).
Perhaps my intent would be more clearly expressed by inverting it: email formatting has nothing to do with Markdown.
You seem to have a very peculiar interpretation of the word "connection"
> and is quite incompatible with them.
It can't be incompatible with ad-hoc unspecified and informal formatting habits.
> Perhaps my intent would be more clearly expressed by inverting it: email formatting has nothing to do with Markdown.
That's just as inane for the same reasons.
- Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
- Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead.
https://news.ycombinator.com/newsguidelines.html
If you disagree with the substance of the article, say you prefer Markdown to rST, then address that. Explain why despite the author's preference for rST, you prefer MD. It's silly to argue over this single sentence about exactly what Markdown is or isn't.
I haven't used rST, but looking at the examples it looks considerably less readable out of the box.
I've started using MyST for my more recent projects, which gives me the features I care about from reStructuredText - references and table of contents - but lets me use markdown syntax, which is easier for contributors.
The real gamechanger is for internal links (rST+Sphinx) and the use of the `:ref:` and `:doc:` directives so that you can reference an anchor or a doc link in the same content without having to type the header manually (which will inevitably go stale).
https://www.sphinx-doc.org/en/master/usage/referencing.html#...
that is one of the things I miss most about writing in rST, for sure.
Yeah, using markdown to write a book is very limiting and doesn't offer the control needed for typesetting.
OTOH, using markdown to write is liberating - you only get a few markup options, so you can focus on the content:
> I think Restructured Text is better for writing books than Markdown.
That would be a better title.
https://djot.net/
Designed by one of the authors of CommonMark, an attempt to specify a standardized flavor of Markdown.
Then again, it is already in pandoc, so I suppose you can trivially move out of it.
EDIT: By the way, recent comment from the primary developer: https://github.com/jgm/djot/discussions/310#discussioncommen...
rst:
scroll: "Directives" in rST are "Parsers" in Scroll. The item above would be parsed by the `imageParser`, which at ~50 lines of code (https://github.com/breck7/scroll/blob/b8fd72aa38742cc6cd575f...), is actually more code than the average parser.To write your "exercise extension" in Scroll, you would just need to write a few lines of code definining an "exerciseParser". You could put that in the same file you use it in, or in a file named "exercise.parsers".
I think you should explore Scroll -- https://scroll.pub/ -- might do everything you need in a simpler way!
https://pandoc.org/
Or “how I stopped caring about doc formats”
For hobby usage, use whatever. But I would strongly oppose anyone who would try to make RST standard at my workplace.
A scrum-master, PO or marketing can write markdown. RST would be a hard sell.
Never before on an article I've felt the vibes of the "I stopped reading there"-meme harder than here.
Not really. Quoting https://en.wikipedia.org/wiki/Markdown
>Its key design goal was readability, that the language be readable as-is, without looking like it has been marked up with tags or formatting instructions, unlike text formatted with 'heavier' markup languages, such as RTF, HTML, or even wikitext
So, for use cases that don’t make a good match of this chief constraint, Markdown might not be the most relevant option. It’s just as meaningful as saying that we prefer TeX over Markdown because fine typographical tuning is an important concern of our specific usecase.
I admit I already dealt with a screw using a hammer, not having the right toolset at hand. But when the situation don’t call for adhoc heterodox actions, I prefer to match a screwdriver with a screw and a hammer with a nail.
RIP
Speaking as one who actually produces blogs on rare occasions, I have to say starting every post with "This is just my experience and may not apply to you this is not a claim of global applicability this is just applicable to this circumstance you're the one who chose to read this I didn't force you" and so on isn't exactly the most compelling way to open a post.
Though I have to admit now I'm really tempted to add that into my post template in like 2pt font or something, just so when people on HN complain I can point them at it.
By contrast, I was careful to criticize the title as clickbait, not the post, which is useful (or the book, which sounds genuinely interesting). I wasn't perturbed by the title. By your comments, sure, but that's your doing, not Hillel's.
"Why I prefer rST to Markdown"
or
"Why I prefer rST to Markdown for Typesetting"
Since it's a clearer statement of the author's meaning, and since very few users interested in Markdown are going to be doing any typesetting, it seems like a more meaningful title to me.
But less likely to draw in readers... to "bait" them, if you will.
The iirc, syntax is :inline{property} and
It doesnt nest well, so I applied any document-level styles using the filename; but I'd imagine the yaml-style lead-in tags should be the correct way to do it.reStructuredText (reST) by itself can be pretty rough. reST combined with Sphinx is pretty great. I.e. the strengths of Sphinx far outweigh the weaknesses of reST. For big, professional docs sites (+100 pages, +10 contributors) I have pretty strong opinions that Sphinx is probably the most responsible choice, long-term.
> The most important difference between rst and markdown is that markdown is a lightweight representation of html, while rst is a midweight representation of an abstract documentation tree.
Yes, what this means in practice is that it's easy to customize common aspects of your site (such as how images are displayed). A teammate of mine on Pigweed recently created a shortcut for linking to bugs consistently. You type :bug:`59385981` and the link gets transformed into https://pwbug.dev/59385981. If we ever need to mass-migrate all of our bug links, it's now trivial. Another profoundly important consequence of this architecture: all internal links are guaranteed to always resolve (you get warnings/errors if you link somewhere that doesn't exist). I previously wrote about how baffling it is that this is not the industry standard for docs sites: https://technicalwriting.dev/src/link-text-automation.html
The other thing that Sphinx excels at: well-defined extension and theme APIs. It's not exactly easy to build extensions or themes, but lots of people have figured it out (including me). There's a pretty rich ecosystem of extensions and themes on PyPI.
Lately I've been calling Sphinx the sleeping giant of docs systems. It's already pretty great and with a little concerted effort I think we can collectively make it absolutely phenomenal. The upstream Sphinx repo now has a GitHub Discussions section that gets a little traffic; a lot of us seem to congregrate on the #sphinx channel in the Write The Docs Slack.
I'm not a fan of reStructuredText, but nowadays it's possible to do most things that previously tightly coupled Sphinx to RST in Markdown, courtesy of MyST-Parser: https://github.com/executablebooks/MyST-Parser
[1] https://pigweed.dev/#what-is-pigweed
The first part didn't really do it for me because it seemed too meta:
"Over 150 libraries" - to do what? (Actually, some good examples are in the next section.) "Automation" - to build, test, and lint what? "Environments" - to make builds reproducible, to do what? "Frameworks" - turnkey solutions for what?
They are clickable, though, so maybe I should have clicked. :)
> We provide libraries (modules) for strings, time, assertions, logging, serial communication, remote procedure calls (RPC), and much more.
Maybe put that up front?
Looks like "frameworks" are still work in progress; maybe I'll check back someday when they get to Raspberry Pi Pico support.
At my last job I spent a fair amount of time on documentation, but I don't have a great understanding of what technical writers like/dislike, or what documentation systems are best.
So, you've made a convincing argument of why we should prefer Markdown to reST, 9 times out of 10. If we're working with Sphinx we'll give it serious consideration.
So yeah, I don't really have strong opinions on sticking with rest per so. I only have strong opinions about Sphinx being probably the best choice out there for large docs sites.
[1] https://myst-parser.readthedocs.io/en/latest/
I found this to be very easy using Markdown+pandoc. I turned image tags containing youtube links to thumbnails with video tag and alt text, hooked up image tags to local video files to ffmpeg to optimize and resize videos to then do the same, etc in a few lines of code.
This, so much this.
It's a problem in an even more general form (for you're writing about linking to sections within a page): the number of CMS systems or static site builders that make you insert the final URL when you write is _insane_.
If the slug changes, you reorganise your site - now you have to do a search and replace through the site to update all your links.
Static site generators could enable linking like [Hello](../hello.md) and resolve the link at build time, instead of expecting everyone to type [Hello](/why/hello/). And yet the popular ones I've used or looked at don't do this.
It seems to be a marmite feature (you love it or you don't): I've raised it with static site builder team members and received a "Why would you want that?" response. Explanations got nowhere. I'm not sure if it's a niche feature, you have to have experienced the problem to appreciate the solution, or people are used to writing once and not maintaining for a decade or longer, but it'd be great to see wider support.
At least for me, aesthetics are a high-priority factor when deciding which site generator to use. Hugo and Gatsby have great built-in themes, and I have chosen them for projects before based largely on this fact.
The Sphinx themes here [0] and here [1] are various levels of "meh" to bland.
Look at the standard Sphinx RTD theme: https://sphinx-rtd-theme.readthedocs.io/en/stable/
Now, compare it to documentation sites like:
- Apple's: https://developer.apple.com/documentation/swift/array
- Fluent UI: https://react.fluentui.dev/?path=/docs/concepts-developer-po...
RTD looks dated in comparison.
[0]: https://sphinx-themes.org/
[1]: https://sphinxthemes.com/#featured-themes
I'll ignore Apple because: 1) It loads _extremely_ slowly (10+ seconds). 2) Just looks like a dark theme and nothing more.
The featured themes, in general, have poor typography and whitespace, poor formatting and alignment, and uninspired colors and design. Furo [0] is the best-designed of them and would make a solid foundation for a proper theme.
[0]: https://sphinxthemes.com/themes/furo
However I'd push back a little. That Fluent UI doc site looks like every other doc site from an open source project started since ~2020 that I've seen, and this means that I have certain expectations: I expect the project to be immature, I expect the docs to be incomplete or out of date, I expect the search to be an annoying AI search that does a worse job than the basic full text search that I expect elsewhere, I expect that to get anything of substance done I'm going to have to hang out on a Discord server. When I see a ReadTheDocs site though with that default theme, I expect the project to be somewhat mature, I expect it to basically work. I expect the docs to be fairly complete.
Are these expectations fair? Not really, but that's the whole point here. Open Source projects are not immune from marketing, and when marketing, what other similar projects do will affect how people respond to your marketing. Nothing exists in isolation.
In my experience, engineers think marketing doesn't apply to them, but these impressions really do matter.
One nit: I called Sphinx a sleeping giant because it's already majorly successful in the Python ecosystem. Pretty much all big Python docs sites are built on top of Sphinx, including docs.python.org. But I think it's fair to say that when Sphinx starts getting major adoption outside of the Python ecosystem then we can start calling it a breakout success.
I just ported a 200+ page book documenting an internal language/VM/abstraction-layer to Sphinx ... holy moly, this is a life changing system!
I wish the documentation for Sphinx, itself, had a lower ramp and/or more examples; otherwise, I'm still in a pretty fantastic honeymoon phase.
My two main lines of research are how to build good looking PDF (books); and, some sort of system to let me cut a book into POSIX compliant man pages by chapter/section.
When the first link is also the second link, and both are to BUY a 15 dollar book with the default "pay anything price" bar set to 20, I am just clicking out and ignoring literally everything in your marketing post.
Others can discuss this RST thing I have never heard about, I am just here to point out you have now alienated one of your potential converts. Put the book link at the bottom, and dont double link to the same purchase page in the same paragraph. FFS.
But I'm so sad that asciidoc isn't supported by pandoc.
I guess this sort of falls under the "But I hate the syntax" section of the article, but I don't think it quite gets the point. I can look at a Markdown file and read or write it without even really thinking. Which is not really the case with rST (or even HTML, for that matter).