Thank you for pandoc. I made the (at the time perhaps not transparently wise) choice to go all in on it when I started my PhD, and that decision has paid nothing but dividends since. I owe my career as a scientist to it.
Pandoc is amazing. I have HTML and want a PDF? One command and it just works painlessly. That was just my last use of pandoc this weekend. I do not use it that often, but when I do its perfect.
> by writing N parsers (“readers”) and M renderers (“writers”), one could support N × M conversions.
Beautiful writeup for a wonderful project. In an age of vibe-coding hype it's also so nice to see how things can be extended and snowball in usefulness when things are built correctly, by hand, from basic principles.
> Perhaps, then, in the future, people will no longer have a need for tools like pandoc.
I think we will need wonderful things like pandoc more and more. As mentioned there is a huge ecological and practical difference. Even if LLMs could get infintisamally close to deterministic-level reliability, it's still so many more orders of magnitude better in efficiency, especially with big batch jobs etc.
which strips out styling, wrapper divs, spans, inline attributes, etc from (for instance) HTML copied from a google or word doc. Just the semantic goodness!
To top it all off Pandoc has a great experience for contributors. Over the past few years I've opened several bug reports related to Typst and docx and all of them got responses that were kind and helpful. I even had a few PRs merged in despite knowing almost nothing of Haskell.
Pandoc is a fantastic piece of software. I have never had any issues with it, and it's the tool I reach for anytime I need to covert documents. I'm super grateful to John MacFarlane for creating it and maintaining it for all these years!
Pandoc is awesome. My fav usage is configuring git to use it to normalize binary docs to markdown (like a .docx) so they can be diffed. Works amazing for redlining contacts.
I do this too. If you use e.g. GitHub you can build this into your PR checks so that any PRs containing docx file changes get a CI bot comment with the diff from the docx.
Thank you for Pandoc! I used it for many things over the years, and it was always there as a good option. I mostly use it for Markdown->Typst these days and it does this job very well.
"The choice of Haskell has also led to a high quality and low volume of contributors"
I feel this influence of choosing a tech stack and its impact on self selected and auto-reenforced culture is most often underestimated.
From my own experience, at a time I was (involuntarily) working in Java, and when .Net was released, from a pure technical point of view it was like a breath of fresh air. Java was suffering from overengineering, archtecture astronauts galore and no sensible UX framework. .Net, the new kid, came in lean and clean with a UX library that 'just worked'.
Problem later was that for all its flaws and being overly 'academic', in teams (the real thing, not the awfull app), you could have indepth discussions about non trivial aspects of SWE topics in the Java world, whereas for all its technical prowess, in .Net land you were mostly dwelling amongst the 2 week CRUD app bootcamp folks. This ofc is a gross oversimplication.
You had brilliant engineers and challanged codemonkeys on both sides. But the skew was more than a little biased.
A few years ago, I gobbled together some bash scripts around pandoc to build a site generator for my personal website. Works great. I use html templates, markdown for the content, etc. Mostly the bash scripts just serve to list files and process them one by one. I actually process them concurrently by forking processes so it's reasonably fast. A bit wonky but it works fine for my use case.
As for Haskell, I guess tree transformations and parsing are the perfect use case for functional programming. I studied in Utrecht in the nineties when Erik Meijer was still teaching there (later went to work at Microsoft Research where he contributed to things like F# and Linq). In short, my compiler course was taught using functional programming. We were toying around with writing our own parser generators to implement a subset of Modula 3 or our own toy languages. Lots of monads and other esoteric abstractions.
I haven't really done much professionally with any of that since except having a really easy time when languages like Kotlin, Javascript, etc. started borrowing liberally from functional programming. These days, if you have a list, calling map or forEach on it with another function is perfectly normal in many languages. Very nice alternative to a for or while loop.
35 comments
[ 0.22 ms ] story [ 34.5 ms ] threadI never heard about djot [0], is anyone using it?
[0]: https://djot.net/
Beautiful writeup for a wonderful project. In an age of vibe-coding hype it's also so nice to see how things can be extended and snowball in usefulness when things are built correctly, by hand, from basic principles.
> Perhaps, then, in the future, people will no longer have a need for tools like pandoc.
I think we will need wonderful things like pandoc more and more. As mentioned there is a huge ecological and practical difference. Even if LLMs could get infintisamally close to deterministic-level reliability, it's still so many more orders of magnitude better in efficiency, especially with big batch jobs etc.
https://gist.github.com/rahimnathwani/210b1f9cb6ce731a304322...
tidyhtml () { pandoc -f html-native_divs-native_spans -t markdown-raw_html-raw_attribute | pandoc -f markdown -t html }
which strips out styling, wrapper divs, spans, inline attributes, etc from (for instance) HTML copied from a google or word doc. Just the semantic goodness!
find . -name '*.md' -type f -exec sh -c '
' sh {} +Pandoc is my go-to tool. Thank you, Sir!
[1] https://johnmacfarlane.net/index.html
[2] https://johnmacfarlane.net/tools.html
If you compare this with the HTML produced by typst or hevea, this is super useful, as we can then roll out our own styles.
I feel this influence of choosing a tech stack and its impact on self selected and auto-reenforced culture is most often underestimated.
From my own experience, at a time I was (involuntarily) working in Java, and when .Net was released, from a pure technical point of view it was like a breath of fresh air. Java was suffering from overengineering, archtecture astronauts galore and no sensible UX framework. .Net, the new kid, came in lean and clean with a UX library that 'just worked'.
Problem later was that for all its flaws and being overly 'academic', in teams (the real thing, not the awfull app), you could have indepth discussions about non trivial aspects of SWE topics in the Java world, whereas for all its technical prowess, in .Net land you were mostly dwelling amongst the 2 week CRUD app bootcamp folks. This ofc is a gross oversimplication. You had brilliant engineers and challanged codemonkeys on both sides. But the skew was more than a little biased.
As for Haskell, I guess tree transformations and parsing are the perfect use case for functional programming. I studied in Utrecht in the nineties when Erik Meijer was still teaching there (later went to work at Microsoft Research where he contributed to things like F# and Linq). In short, my compiler course was taught using functional programming. We were toying around with writing our own parser generators to implement a subset of Modula 3 or our own toy languages. Lots of monads and other esoteric abstractions.
I haven't really done much professionally with any of that since except having a really easy time when languages like Kotlin, Javascript, etc. started borrowing liberally from functional programming. These days, if you have a list, calling map or forEach on it with another function is perfectly normal in many languages. Very nice alternative to a for or while loop.
I use it to create pdfs for my blog posts with that code :-)