69 comments

[ 2.5 ms ] story [ 143 ms ] thread
Re: A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output.

I have to disagree. Often the abstract nature is hard to describe and/or the constructs for it either don't exist, or need cleaning/updating. Often I find myself saying, "I don't know why it's more legible to format this thing such and such way, it just is." Creating a good abstract language or category set for a given domain is not easy. It's good to use abstraction where it's practical, but often you just have to tell them system "just format it like this!"

For example, the difference between a button and hyperlink is often blurry. We could abstract it something like: [Action FormatType="Button" Label="Send email to Mom" ActionType="mailto:mom73@moms.sample"/] so that FormatType could be changed to "Hyperlink", but most find this goofy and perhaps long-winded.

As far as compact wiki-like shortcuts versus XML, the first often has more escaping issues or confusion. It's not a free lunch, but about best fitting intended use.

Seconded. That quote may be a description of a markup language that is easy for programs to work with, not one that's easy for people to work with.
Good point. What's people-friendly may not be machine-friendly, and vice versa. Further, what's human writing friendly may not be human reader friendly. When writing, verbosity reduction matters most; but when reading, clarity often trumps verbosity concerns.
In a conversation elsewhere recently I wrote:

> I think Markdown is popular because it’s a reasonably intuitive bridge between how we format handwritten documents (which is itself a blend of what makes sense on the page, and an approximation of ~spoken rhetoric) and HTML.

> In some cases (like the table example), what’s trivial on the page is painful with a keyboard. The technologies that shaped the old idioms didn’t have editability as a selection pressure. You can make a few edits inline, but at some point you’ll just have to rewrite the document.

> Much of the power and pain of code are byproducts of the machines forcing us to be explicit, but we didn’t have the foresight to spend centuries molding ourselves (human languages, pedagogy, rhetoric) around that kind of precision. Markdown meets us very close to where we are. The other path, I think, entails developing toolchains that do expect that precision and requiring the humans to do the changing.

I guess the split between the ease of a format for reading/writing is an extension of the difference between shorthand or even cursive and print when it comes to these affordances.

> What's people-friendly may not be machine-friendly, and vice versa.

I think we’ve established that to be an axiom, otherwise we wouldn’t have 5 popular configuration formats and 10 popular markup languages.

> As far as compact wiki-like shortcuts versus XML, the first often has more escaping issues or confusion.

It isn't an either-or, and never has been. XML has been created as an SGML subset, and SGML always had short references ie. custom tokens the parser is replacing by something else in a context-dependent way. For example, an asterisk can be replaced by an <em> start-element tag to start emphasized text, and, within emphasized text, asterisks can be replaced by </em> end-element tags.

Since when is LaTeX is a "lightweight markup language"?

* It's not at all lightweight, I'm pretty sure it's Turing complete up the wazoo.

* You can reprogram the f'ing grammar, down to the interpretation of individual input characters...

* In fact, I'm not sure you can even call it a markup language. It's a macro-based programmable system on top of a typesetter.

For the generation of documents and papers, I'm not actually sure that avoidance of Turing complete is really a worthy goal. It can certainly go too far, but I think it is a sympathetic view that the quest for complete separation between content and presentation has given more victims to failed delivery than macros have.

I do think that stability is something that needs some discussion here, for a great markup language. Not stability as in "doesn't crash", but in "doesn't change."

Can you get away with ignoring Turing? sure. But it's not going to be a "great markup language"

Victims? Why would this requirement cause development difficulty? If anything it would make it easier to deliver, and a lot easier to develop alternative parsers for, and easier to extend as well. We figured out all these things a long time ago but then ignored them because we thought we knew better then those who came before us.

I'm not sure I follow. Or agree, all told. org-mode is a great markup language that lets me put in actual program snippets. I technically can make it self modifying such that it reflects on itself. May be useful for some things, but I don't know why I would go out of my way to do that. Just as I don't know why i would go out of my way to make it not possible.

Similarly, I confess that plain TeX is a lot more approachable than I had always assumed it was. Even better, older TeX files still work, today.

Granted, early HTML mostly still works, I think. Though, I think xhtml failed miserably. And as nice as it is that HTML still mostly works, I don't know of anyone that actually uses HTML as their authoring language, which is more than a bit of a shame.

> Markup language which completely falls over this is Markdown. There’s no way to express generic tree structure, conversion to HTML with specific browser tags is hard-coded.

This isn't really a fair criticism. True, the original Markdown.pl did not produce a generic tree structure, but that's a fact about the program, not the syntax it parses. Many Markdown and Commonmark implementations do support creation of an abstract syntax tree. Pandoc has done this for the last 17 years. It also provides nestable, generic containers as a syntax extension.

> It feels like there’s a smaller, simpler language somewhere

Here's my attempt: <https://djot.net>.

Wait a minute, I recognize that GitHub handle! Are you John McFarlane? What a mind-blowing day to come across the creator of pandoc. You've saved many a student from pain. Thanks for everything.
>It also provides nestable, generic containers as a syntax extension.

That’s what I’ve meant: markdown syntax doesn’t allow for this, which requires every extension to be syntax extension. Djot’s span&div do address this problem.

(comment deleted)
See also markdoc.dev for instances where some commonality with Markdown is desired or required.

Markdoc—> ast —> renderable tree —> HTML or React

> we are seriously considering the possibility of drafting a specification for the JSON representation of Markdoc's Abstract Syntax Tree (AST) in order to facilitate interoperability between Markdoc tools

Markdown is not markup. It's in the name.

The point of markdown is to be readable by humans, using similar annotations that people have been using in ASCII text emails for decades.

Kudos for fighting the good fight, but honestly this is a losing battle. As time goes on, fewer and fewer of the people in the world using Markdown are interested in this distinction.
Markdown is markup.

> Markdown is a lightweight markup language for creating formatted text using a plain-text editor. -- Wikipedia

> Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML). -- Gruber

Moreover, Markdown is constrained by the fact that it doesn't have a semantics, strictly speaking. It has a syntax, and that syntax is tied to the HTML canonical form of the document, which any valid Markdown file can be transformed into.

The fact that Markdown is written in extremely simple, easy to read (and brilliant IMO) syntax doesn't make it not markup.

Why doesn't markdown qualify as a markup language?
Personally I've found that I prefer a consistent syntax with few special characters — like HTML, but less verbose — which led to the creation of xidoc: https://xidoc.nim.town/
I have a very simple way of evaluating any sort of markup language. How easy is it to edit tables?

https://htmlpreview.github.io/?https://github.com/jgm/djot/b...

In that example, xidoc requires a human to manually align ascii chars to make it look pretty... That's a job that computers are good at!

I would rather be able to define some sort of data block:

    [let "myvar;" [csv;
    col1,col2,col3
    1,2,3
    4,5,6
    ]]
    [output-table "myvar"];
... or something.

I'm not sure right now how useful is markdown really... It was a nice experiment but I mostly use it for bullet points (*) and headings (#). I would say that's successful enough as a contribution to internet culture! :-)

EDIT: seems like I commented on xidoc+djot at the same time, but the criticism is the same for both languages :-)

https://djot.net/

> In that example, xidoc requires a human to manually align ascii chars to make it look pretty... That's a job that computers are good at!

What do you mean? I'd say that's true of either every or no language, depending on if your editor has an align plugin. How does your proposed CSV-like format solve this?

Hmm. I found myself nodding along with what this has to say about syntax design and the responsibilities of well-abstracted markup languages/converters, but I was also a little surprised by some apparent contradictions.

> Great markup format unambiguously interprets an input string as an abstract tree model of a document. It doesn’t ascribe semantics to particular tag names or attributes.

>

Yes!

> Markup language which nails this perfectly is HTML. It directly expresses this tree structure. Various viewers for HTML can then render the document in a particular fashion. HTML’s syntax itself doesn’t really care about tag names and semantics: you can imagine authoring HTML documents using an alternative set of tag names.

But it's a bit weird to uphold HTML when it seems to really be valuing the underlying syntax and not the big heckin' standard that details the valid elements, attributes, and their semantics. :)

> Great markup language defines the semantics of converting text to a document tree

This seems to conflict with the first one I quoted. Maybe I'm missing something? Maybe the author is using semantic in different ways, here? Markup languages tend to mix different kinds of elements, and it can make thinking and communicating in this space tricky! I started trying to untangle this knot recently in https://t-ravis.com/post/doc/what_color_is_your_markup/.

(I also found banning "semantic" from my writing helped me untangle by making it easier to notice when I was letting it do too much work. https://t-ravis.com/post/doc/semantic_the_8_letter_s-word/)

> But it's a bit weird to uphold HTML when it seems to really be valuing the underlying syntax and not the big heckin' standard that details the valid elements, attributes, and their semantics. :)

SGML?

With apologies to Dr. Knuth…

The most important thing in a markup language is the name. A language will not succeed without a good name. I have recently invented a very good name, and now I am looking for a suitable language.

(comment deleted)
(comment deleted)
I would like to know the very good name you have come up with.
IMO this misses the most important thing needed in a great markup language. It should be readable, as a complete document, in its existing text form without any transformation. The ideal syntax would be so good that you would rarely want to transform the document in any way.

I think inspiration in this area should come from an unusual place: RFC documents (like https://www.rfc-editor.org/rfc/rfc6877.txt). These documents are fully readable in their standard text form. The only things missing are conveniences like clickable links, which could be supplied by a viewer application without transforming the text of the document.

Markdown comes the closest to being directly readable, but it makes a number of sacrifices to syntax in view of its intended transformation to HTML. It's also limited in scope, patterned after ASCII email syntax. So you miss document-oriented conveniences like standardized headers, proper citations, and pages - all of which RFC documents have!

That's in part the reason that a bunch of people have come behind and written extensions to Markdown. IMO someone needs to come back and gather the best of these into a consistent and simple syntax that is oriented around the needs of plain text readers.

> It should be readable, as a complete document, in its existing text form without any transformation.

Is this _needed_, or is this nice to have?

A markup language annotates text and describes _how it should be rendered_. It feels redundant to describe how a document should be rendered (presumable for final consumption) _and_ have the document be readable as-is.

Case in point: I'd argue that HTML is a great markup language. I wouldn't call it the most readable in its current form.

I agree with the spirit here, but it ultimately feels more "nice to have" than truly required.

>> It should be readable, as a complete document, in its existing text form without any transformation.

> Is this _needed_, or is this nice to have?

It's kind of the ur usecase of markup languages. Without this property WYSIWYG is significantly better.

I like markup over WYSIWYG because I can use my preferred text editor, can be version controlled, reused in multiple places (reddit, github, etc).

I like that the markup I use (Markdown) is readable as is, but I wouldn't mind losing a bit of readability for more features.

Very much disagree. Think about Markdown readmes. It's great, and very useful, that when looking through a git repo that I can just look at the readme in a shell or text editor, and also as important, diffs when changing a markdown file are a lot cleaner/easier to understand. Simultaneously, it's also great that I get a nice formatted readme page when I'm browsing a GitHub repo, for example.

As the sibling commenter said, if all you care about is the output why wouldn't you just write it in Word?

> Case in point: I'd argue that HTML is a great markup language.

By what measure? I'd consider HTML an awful, awful markup language.

Note, however, that the plain text form has for a long time typically I believe been the output of a machine conversion from an XML vocabulary, and now always is. I’m not deep enough in the weeds to know all the whens and the distribution of formats, but RFC 2629 (June 1999) defined the first XML vocabulary, and RFC 7990 (December 2016) completed the process of declaring an XML format the canonical source, rather than plain text. To learn more, start at https://www.rfc-editor.org/rse/format-faq/.

Consequently, new RFCs are now published most obviously as full regular HTML, like https://www.rfc-editor.org/rfc/rfc9110. The plain text form is still available (https://www.rfc-editor.org/rfc/rfc9110.txt), but it’s no longer paginated and may be incomplete, as specs may include SVG graphics. (RFC 9110 uses ASCII art in some places, e.g. sections 3.6 and 3.7. I am no connoisseur of recent RFCs, but don’t know of any using SVG yet to give an example. I don’t know what the text form gets in this case.)

Most germane to this discussion now: older RFCs got converted to lightly-marked-up-but-mostly-plain-text HTML via rfc2html, and exceptions and tweaks and whatnot had to be made regularly, because in the past the RFC document format was not designed to be machine-readable, and it showed in irregularities that are not acceptable in a markup language.

So in summary: yes, you can get some inspiration from RFC documents, but don’t mistake that textual format for an acceptable markup language, because it wouldn’t actually work.

A subject very near and dear to my heart. Asciidoc, flawed as it is, is still the best game in town, with conditional content, transclusion, and a particularly robust table model. Combined with S1000D architecture, it makes a passable method of writing publications for heavy industry in aerospace and defense. On the double cheap. If your team has some small amount of tech chops.

The `include` is a problem, though, and the devs have known this since at least 2017. Two directives under work here. The `ainclude` is an extension that works in the subdoc direction, and `subdoc` is a directive under development that's on the milestones to go into core. Generic blocks is also on the roadmap, and it's available via extension.

However.

There's deeper problems, I feel, inherent in the DNA of component content systems themselves, as a basic concept. It plagues every CCMS (component content management system), whether it's DITA, S1000D, DocBook (with xincludes), ReST, or otherwise. It's not a markup problem. In S1000D, I call it "the applicability trap".

To put this very briefly, you incur risk when you replace any component of natural language with a constructed language. In the case of CCMSs, you have an implied content architecture doing the work of NL in between content "chunks", but not everyone architects content, or, hell, even their product. The writers won't know this until they start, and by then it's too late.

Without architecture, you have a bunch of chunks that used to be structured linguistically, and now they're not doing much of anything at all.

I can't tell you how many S1000D systems I've seen built around a product that existed as little more than a twinkle in a salesman's eye. You build out an SNS and an applicability model[1], all from a bunch of emails or a spotty CAD diagram, and tell the writers to get cracking. Then, N months and N millions of dollars later, scrap everything because, lo and behold, the actual possible product completely invalidates the content architecture. "How did the doc set cost 19 million dollars?". Well, it's not the markup, it's not the vendor, and whatever miracle editor your salesmen buddies are pushing won't do jack to fix it, either.

What the writers need is a quantifiable test for architecture before they start, or else, sooner or later, the Applicability Trap will get them. Either that, or write individual BDMs[2] for every single product variant that spews out of the pieholes of business development.

I feel a little dumb getting worked up here. All of this crap is going to be glitter in the river come 2030 anyway, because we'll be training our pocket AIs for most of this.

[1](sort of a structure of conditions to filter content)

[2] Big Dumb Manuals, like mamma used to make

Content reuse (like in a CCMS) seems to complicate everything pretty nastily.

For example, if you transclude one markdown file into another markdown file, are the transcluded file's headings rendered as absolute levels - or as relative levels, based on the indentation level at the point of transclusion ? Of course the same issue arises when transcluding HTML, what with <h1>..<h6>.

This issue could arise with (among others?) Lightweight DITA, but I don't think its spec is fully developed enough to even ask the question.

The main problem continues to be, in my view, that we don't have the equivalent of plain text for formatted documents or trees. Plain text can be copy-pasted, read by any program, edited anywhere, and that's great. Binary formats are inaccessible and hard to work with by comparison, but this is only because of mundane technical reasons. We could easily make new binary formats that would be terrific for lightly marked up text, and the editors that go along with them.

Today bold, italic, tables don't fit in plain text, but carriage returns and tab characters do, as well as Halloween pumpkin emojis. There is no good reason for this, this is just where we kind of ended up. You can't put a floating point number in text, or a price, or a phone number, or date. And when you don't have standards that support this you can't have any meaningful kind of data exchange. So forget about copying a table from a spreadsheet anywhere. Everybody ends up shoehorning this necessary functionality in text editors and it won't ever work.

How can computing still be at the stage where if you want a table you have to conjure up a bunch of pipe and dash characters that hopefully compiles into a table that looks presentable? And then you have to switch between "code view" and "presentation view" to check if you did it right. It would be funny if this wasn't so tragic.

I'm not convinced.

What you're talking about sounds an awful lot like MS Word. It has a document model. It has a renderer for said document model. The document model even has first-class abstractions (Word templates). You can directly edit the rendered representation. And it has a mode to show (at least certain kinds of) markup directly in the rendered document.

But the thing that all binary formats share is a lack of transparency. When I have a text file with, say, Markdown formatting, I know I'm seeing 100% of the formatting. Because the irony of not having a "code view" / "presentation view" split is that I'm never 100% sure what I'm actually looking at. Even in "regular" documents, I find that this can matter (especially when you're using Word templates and you want to do some sort of a global format change—it really matters what formatting you've applied to the document and not just what the current rendered output looks like). In documents like HTML that need to be processed in one of a variety of ways, this matters even more. I frankly cannot imagine a solution in which you do the sorts of sophisticated transformations people routinely do with HTML without caring very deeply about what "code" gets generated.

If I'm not mistaken, predecessors to MS Word actually had a code mode, with a corresponding input language to control it. All of those solutions died off, at least in terms of being popular in the long term.

The biggest properties of HTML/Word I prefer over Markdown is that pasting/typing special characters like [] or backslashes into the graphical editor won't break formatting, and there's a standardized format with unambiguous rules to parse content so every receiving application interprets it identically. The biggest properties of Markdown I prefer is that (like HTML) it has first-class semantic code and block and blockquote sections (Google Docs only allows changing the font, unsure if MS Word has a semantic system), copying text between documents preserves meaning but not mismatched fonts/sizes/colors, and you can universally copy-paste it between apps, read it in a text editor if necessary, and diff/merge it textually.

I think there's a local optimum of markup editors, with semantic HTML as a syntax, and a graphical editor which offers code blocks and tables, but does not inject presentational tags like fonts and sizes (and optionally colors) into the document being edited and saved. I think this should satisfy most of the points I want, though it's worse than Markdown at universal semantic copy-paste, reading as plaintext, and diffing/merging. Unfortunately Qt's QTextDocument converts semantic HTML into presentational. On the other hand CKEditor (sadly JS, an example at https://pinouts.ru/edit_page.php?file=mini_pcie&lang=eng) looks promising since the editor exposes semantic tags directly on the UI, though I haven't spent much time researching.

The problem is that we have too many such formats. Html, rtf, md, and a variety of others would claim to fill that role. Each of them is workable on their on, but fail to be universal due to all the others.
> More or less, what I want from markup is to convert a text string into a document tree:

  enum Element {
    Text(String),
    Node {
      tag: String,
      attributes: Map<String, String>
      children: Vec<Element>,
    }
  }

  fn parse_markup(input: &str) -> Element { ... }
> Markup language which nails this perfectly is HTML.

The reason HTML nails it perfectly is because this is modeled after HTML.

If I were to make up a markup language, I wouldn't follow that model.

In particular I would get rid of attributes which to me are a restricted kind of children with a specialized syntax.

This is both unnecessary and undesirable in many cases.

The major problem with attributes is the <String, String> mapping. Once something is defined to be an attribute, it cannot be sensibly extended without creating an unnecessary problem.

For example the `class` attribute in HTML looks like it was originally designed to hold a single class name. Then people realized that it would be desirable to have multiple classes per element.

So, to keep things backwards-compatible, the value of the attribute was extended to hold a space-separated list of classes instead. Essentially creating a little DSL inside of the attribute's value.

If `class` was instead a kind of child, initially limited to a single instance per element, extending to multiple instances in a backward-compatible manner would not require introducing the DSL. It would be natural. Just allow many `class` children.

A valid argument in HTML in favor of attributes is conciseness. But that's an artifact of the syntax.

We could make up a language with syntax for nodes as concise as HTML attributes, eliminating that argument.

> It feels like there’s a smaller, simpler language somewhere

Certainly.

I have experimented with many different designs for such a markup language on top of Jevko[0]. One interesting design that trivially maps to HTML looks like this:

  h1 [Title]

  p [
    paragraph
  ]

  p [
    [paragraph with a ]
    a [
      href=[...]
      [link]
    ]
  ]
Another one looks like this:

  [h1][Title]

  [p][paragraph]

  [p][
    paragraph with a
    [href[...] a][link]
  ]
Both are extremely simple and minimal, but also extensible and lend themselves to writing by hand.

[0] https://news.ycombinator.com/item?id=33287620

You just described Lisp :)
Surely you mean S-expressions. They're great, but not as a markup language.

What I show here is in fact even simpler and more flexible than S-exps[0].

[0] For some details and polemic see this thread: https://news.ycombinator.com/item?id=33334789 | TL;DR: Jevko is well-defined, basically just unicode text + escapeable brackets for making trees; it doesn't treat whitespace as a separator/atmosphere (particularly important in markup); it takes advantage of natural name-value pairing tendencies (like tag-children); and it's closed under concatenation by design

(comment deleted)
> If `class` was instead a kind of child, initially limited to a single instance per element, extending to multiple instances in a backward-compatible manner would not require introducing the DSL. It would be natural. Just allow many `class` children.

That's not the operative difference between children and attributes in HTML.

In HTML, if an element is unsupported, its contents are shown, while its attributes are ignored. This means, if a browser saw something like this:

    <p>
        <class>literature</class>
        <class>english</class>
        Billions of years ago, the Universe was created. This made a lot of people very angry, and has been widely considered a bad idea.
    </p>
In browsers that don't support, or even predate classes, you would want them to be ignored. This only works if they're attributes.
I was talking about is making up a new markup language, which doesn't have to inherit the distinctions and behaviors of HTML.

If, in this language, you wanted to have this feature in combination with what I proposed, you could simply mark the attribute-like children, to inform the engine that it should apply different defaults for them and unmarked children that it doesn't recognize.

This is what I do in the first variant of my language:

  a [
    href=[...]
    [...]
  ]
the `=` appended to `href` marks it as an attribute.

This certainly simplifies things when translating to HTML.

But if we were not constrained by the legacy of HTML then perhaps bothering with this when designing a new language would be unnecessary. Maybe not having this default behavior does not matter in practice and you can have a simpler language without it.

To determine whether that's the case it would help to answer: when is the feature you described useful in HTML? And also: when is this feature harmful?

If you just want to have your language throw up an error whenever it sees an unrecognized element, then you’ll probably be able to simplify it a lot. It’s probably fine, since

* as long as you use a build tool the errors will be seen by the author (who will know how to fix them) and not the reader

* graceful degradation and format extensions are a crapshoot due to Hyrum’s Law [1]

[1]: https://news.ycombinator.com/item?id=30726668

Yes, this is probably the most sensible approach.

Other options would be to ignore everything you don't recognize or ignore all nodes of the form:

  unrecognized tag [value]
i.e. where you don't recognize the key and the value is not complex.

You could still display stuff like:

  unrecognized tag [
    [text]
    ...
  ]
i.e. text nodes which are children of unrecognized elements.

But approaches like these would need a good justification.

The element/attribute distinction is because HTML is a markup language, where the primary content is text and markup is used to add structure and metadata.

You can eliminate attributes to get a simpler format, but it isn't really a markup language anymore, since the distinction between text and markup is removed. It is just a generic data structure, like JSON or s-expressions.

The text/markup distinction is one I’ve been thinking a lot about recently as I’ve been both working on my own lightweight markup language, and implementing a component-based website using Django/Jinja/Nunjucks/whatever-style templates. There are too many places where you end up with attribute-like syntax, but want to pass markup, and it sometimes becomes unclear whether you’re working with a string or markup, yet the two are definitely different—in HTML serialisation, one needs escaping, the other doesn’t; or in something like DOM, one’s a string and the other’s a fragment. I’ve become progressively more and more convinced that these kinds of template languages are quite bad at writing this sort of thing.

Take this example of unfortunate mixing:

  {% somecomponent attr="value", title="<strong>A:</strong> B" %}
      <strong>C:</strong> D
  {% endsomeecomponent %}
Trouble is that you basically have at most one markup slot, but many things semantically want more than one. The closest you can generally get in such languages as these is the likes of this:

  {% somecomponent attr="value" %}
      <strong>A:</strong> B
  {% body %}
      <strong>C:</strong> D
  {% endsomecomponent %}
This can be done in some such languages, but not others—and can’t be done in anything like XML. In XML, the solution would be extra nesting, which gets messy quickly:

  <somecomponent attr="value">
      <title>
          <strong>A:</strong> B
      </title>
      <body>
          <strong>C:</strong> D
      </body>
  </somecomponent>
In JSX I suppose you could do something like this, for better or for worse:

  <SomeComponent attr="value" title={<><strong>A:</strong> B</>}>
      <strong>C:</strong> D
  </SomeComponent>
If you use a fragment/child nodes for what is logically a string value, you run the risk of people putting non-text in there; things like an href should clearly be strings, not text nodes—or else you have to decide what to do with <a><href>/<b>exam</b>ple</href>…</a>, and poorly-written DOM manipulators will surely occasionally assume the href element contains only one text node at times. But there’s then also the question of whether you want strings to be usable in text node context (probably the more pragmatic approach), or if you want to maintain a strong distinction between the two (probably the more correct approach).

For my own language, in the more macroy part (when you’re extending it beyond the basic elements), I’m maintaining a distinction between strings and markup fragments, using different delimiters for the two, with "…" for strings and {…} for fragments. I’m also treating attributes and children equivalently, having all as possibly-named children. The basic example could be something like this in my current syntax:

  @somecomponent(attr: "value", title: {**A:** B}, {
      **C:** D
  })
> The element/attribute distinction is because HTML is a markup language, where the primary content is text and markup is used to add structure and metadata.

That may have been the idea, but it falls apart in implementation. Lines between content and metadata get blurred.

Consider attributes like: input/value, input/placeholder, optgroup/label, img/alt, title.

And then elements like: script, colgroup, col, title, noscript, noframes, applet, object, the void elements[0].

Then consider the ability of CSS to add content to elements, including lifting it from attributes[1], and, on the other side, completely hiding element contents.

The line between metadata and data is conventional. It's all data in the end.

But let's suppose it isn't and there is a hard line. Why should the metadata in attributes not be allowed to have structure and be extensible in the same way data is?

> You can eliminate attributes to get a simpler format, but it isn't really a markup language anymore, since the distinction between text and markup is removed. It is just a generic data structure, like JSON or s-expressions.

What I'm saying is that HTML-like attributes are not necessary to keep the distinction. You can keep the functionality, simplify things and at the same time make them more flexible.

HTML-like attributes also don't stop people from using the language to describe generic data structures (XML).

On the other side, if JSON or S-expressions had syntax and semantics that made them suitable as markup languages, I don't see a problem with that.

[0] https://developer.mozilla.org/en-US/docs/Glossary/Void_eleme...

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/attr

The difference is not between tags and attributes, the difference is between text and markup.

A markup language is human-readable text with added markup. A plain text file is basically a valid HTML file (I believe <title> is the only required element). You can then add markup to provide structure and metadata.

This sets it apart from generic data structure formats. You can express the same information in JSON, but in JSON there is no distinction between text content and other information, it is all just structured data.

Something like s-expression is clearly simpler than HTML syntax, and it is endlessly flexible, but it is just not optimized towards writing structured text content the way markup languages are.

That's more like it!

Indeed, markup languages are built around text whereas in JSON/S-expressions/syntaxes designed for expressing data, text is secondary.

The syntax I created is uniquely flexible in that it can be used for both data and markup.

The first variant of the two in my original comment is more data-like, but minimal enough that it works sufficiently well as markup too. Text is a little harder to mark up in this variant than in HTML, as individual text nodes have to be wrapped in []. However this gives precise control over the content of the text nodes and separates indentation and other whitespace of the document tree from whitespace which is intended as actual content:

    [paragraph with a ]
    a [
      href=[...]
      [link]
    ]
The second variant is very much like HTML: text here is primary and interleaved with markup in []:

    paragraph with a [href[...] a][link]
This variant also separates the attributes and tag from the children, much like HTML. However the attributes are technically not restricted to being flat strings. If we weren't translating to HTML, we could have attributes with tree structure.

So we have something simpler and more flexible than both HTML and S-expressions. This is the beauty and value of it.

I hope I can succeed in communicating this to potential users, so they can benefit from it.

As much as I like matklad's work here I disagree. You can't have docs is AST and stuff like list element start with minuses.

First requirement want to nest and organize everything, second one relies on some "natural" (highly contextual) way to do it.

What looks natural for list of strings will not be natural for lists of tables of lists of maping between location snippit and some text.

> Markup language which completely falls over this is Markdown. There’s no way to express generic tree structure

Isn’t using a basic list structure for documents a feature, not a bug? For example, the article itself does not need or use any hierarchy. Surely keeping documents simple is part of the raison d'être for markdown?

Hierarchical structures for documents usually need more software support, which does provide some nice features such as nested formatting, table cells containing complex elements (including other tables), and auto-numbering of headings/outlines.

However a hierarchy is more fragile when text documents are edited as plain text or using text based tools. Surely a prime purpose of markdown and asciidoc is to be simple plain text.

A hierarchy also seems to lead to either (a) hard formatting requirements like strict XHTML, or (b) loose requirements that cause many corner case bugs between implementations (like HTML used to be when XHTML was created).

> For example, the article itself does not need or use any hierarchy.

This two-column layout makes use of hierarchical capabilities of asciidoctor:

https://matklad.github.io/2022/10/28/elements-of-a-great-mar...

Ha - thanks for the correction! That example with the table on the right is sure to be relatively fragile markdown? I am considering implementing a database representation of simple documents. Hierarchical nesting (even with NoSQL) seems like a complexity worth avoiding if possible, so I do really appreciate your example. You don’t seem to use tables in your previous ten articles. The benefit versus cost of the two examples within your article are good to think about, thanks.
Completely agree. I just don't understand why the author cares about some of his "axioms":

> A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output.

Who cares? It's trivial to generate all these other formats from HTML. The thing that's great about Markdown is it is readable in plain-text source, can be styled in any number of nice HTML presentations, and then is easy to convert to things like nice, document structured PDFs. I just don't understand the problem.

The author also doesn't really make any argument as to why a hierarchical document structure is so important to him in the first place. He just kinda states it as being of critical importance, but I can't see why, at all.

I think this is missing a little justification.

They go from this, which I agree with:

> A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output.

> Great markup format unambiguously interprets an input string as an abstract tree model of a document.

To this:

> [In Markdown] there's no way to express generic tree structure, conversion to HTML with specific browser tags is hard-coded.

I understand why you'd want the markup to be turned into a tree, but that's different from saying you should be able to build any tree with the markup. Isn't the point to encode text documents? The fact that you can't build the entire layout of your HTML result in the markup language doesn't seem like a sensible requirement. Why do you want or need infinitely-nested structure in documents?

Something like parinfer is what’s needed here. Thread the needle between syntactical structures and direct manipulation thereof. We need to move past believing that a 1.5 dimensional editing interface is a good way to work with n-dimensional structures.
XML has all the elements. Plus attributes ;)

And schemas and schema validation on top!

I like the idea of markdown but there is just too much missing with most implementations and I think the primary use case is to move away from XML due to verbosity and readability.

I'd really like to see a new API to allow creators to create their own markup languages directly in the browser with native support to map the markup to the underlying renderer.

Article misses one element of great markup language: to reduce complexity, there should be only one mechanism for expressing things. HTML-style attributes are unnecessary. Anything you can express with an element, you can also express with attributes. Attributes are there probably because the overall model of XML/SGML is so complex.
Attributes exist because HTML is a markup language where text is the primary content and markup adds structure and metadata.

Eliminating attributes would mean metadata should be expressed as text content, which in turn means it is not really a markup language anymore, just a generic data structure syntax like JSON or s-expressions.

> A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output.

This is why I like FB2 and dislike ePUB.