Tectonic is forked from the XeTeX TeX engine. However, the build process for XeTeX (and all other mainstream TeX engines) is extremely baroque:
1. The original Knuth sources are written in a language called WEB (which is basically used by nobody else)
2. Those sources are then patched with a series of “change files” due to restrictions on distributing modified sources
3. These patched files are then converted to the Pascal language using some custom processing scripts
4. That Pascal code is then converted to C code using additional custom tools
5. Those C files are then compiled against a set of extension files and libraries written natively in C and C++
One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process. Tectonic is based on the C/C++ files that emerge from the XeTeX build process, and is gradually translating that code to modern Rust.
While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time. This repository recreates the final XeTeX C/C++ files from their source, so that modifications can be ported into the main Tectonic codebase.
> However, the build process for XeTeX (and all other mainstream TeX engines) is extremely baroque
(Disclaimer: I'm on the TeX Live team)
Yes, the build process is rather unusual internally, but this is pretty well insulated from the user. The "standard" build commands used by lots of other open source projects
git clone […] && ./configure && make
should mostly just work, and give you a functioning TeX engine at the end. The full steps are listed at [0] if you're interested in more details though. And TeX Live is regularly built on essentially every platform imaginable [1], so it's pretty unlikely that you'll ever need to work with the low-level build system.
And only some of the engines require this complex build process; LuaTeX is the recommended engine these days [2], and it's written entirely in C (with a few C++ libraries), and it uses a standard autotools build process [3].
> One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process.
TeX development is pretty welcoming in my personal experience: the first time that I built TL (~5 years ago), I was able to follow the official instructions without any problems, and I got a working TeX engine by the end of it. And it was only 2 or 3 years after that that I became an "official" member of the TL team, so I like to think that the development process is fairly welcoming.
But if you have any suggestions on what we can do better, please let me know, either by replying to this comment or to the email address linked in my profile.
> Tectonic is based on the C/C++ files that emerge from the XeTeX build process
I haven't looked at the Tectonic source, but the problem is that the C code generated by Web2C is fairly unreadable, so it's not really very usable as source code.
> While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time.
XeTeX is essentially frozen for the time being [4], unless someone steps up to maintain it. We are cautiously optimistic regarding LLMs though: they seem to be fairly decent at writing WEB code, and since even the TL development team only has 2 developers who are proficient in WEB, we'll take any help that we can get. (We don't have any non-trivial AI-written code yet, but it's definitely something that we're looking into)
A tool like this is sorely needed for LaTeX, and Tectonic is especially intuitive to embed into other applications, but the divergence of XeTeX from pdflatex makes it incompatible with most Overleaf projects. This is just an unfortunate ecosystem gripe, but for most workflows I end up having to reach for latexmk instead for this reason.
I have been using TeX/LaTeX for ages and today same issues hinder the user experience like multiple decades ago - cryptic error messages, complex pipeline, lack of the proper Unicode symbols support out of the box, and so on.
Nowadays, with Typst existing, it's vital for TeX ecosystem to solve these issues, since none of them are present in Typst. Projects like Tectonic would solve this for TeX, but they lack enough hands and (maybe) financial support.
Otherwise, using TeX only makes sense nowadays only if 1) you already have some templates 2) some features are still missing in Typst 3) you are just forced to use TeX/LaTeX for whatever reason.
Tectonic is a cool project, but hasn't seen any significant changes in a few years---and likely won't anytime soon. It seems we maintainers don't have the time and motivation to put serious work into Tectonic.
I haven't looked at the code in years (and thus may be wrong), but here's a quick overview:
Tectonic's code consists of thin bindings to /harfbuzz/graphite/etc and a vendored XeTeX (in C, with some tweaks to make the build easier), driven by Rust that tries to keep the TeX environment predictable and sane. A few components have been fully ported to Rust (bibtex, spx2html), but the project is very unfinished.
I've looked into the dark corners of TeX when I worked on Tectonic, and it is not pretty. TeX relies on a stack of evil hacks and esoteric behavior that is very hard to replicate, and very difficult to expose in an ergonomic way. This is true of the core system, and of many packages on CTAN.
A quick example: code highlighting does not work in Tectonic. The canonical solution is https://ctan.org/pkg/minted, which spawns a python process to style your code. Reproducibility is one of Tectonic's selling points, so we cannot replicate this behavior.
With https://typst.app/ as good as it is, there's little motivation to modernize TeX---especially considering the effort required. Typst _is_ modern TeX, and I'd rather spend my time there.
Just wanted to say big thanks to the maintainers, I've been using tectonic the last couple years as my only LaTeX distribution, works everywhere (including macOS), it's available in conda-forge, so I can just have it as a dependency in my projects. Everything "just works", that's the best way to describe it.
In my project https://mdview.io I support Latex, but sometimes people share some weird types of syntax which looks like Latex but not rendered correctly. Haven't' found a good way to fix it, probably I will use some hybrid approach which I adopted for broken Mermaid diagrams (LLM + heuristics)
Other solutions get a lot right, but I don't know if I will ever use something other than LaTeX. Nothing gets math typesetting feeling so right. The justification and bin packing feels right. I might be esoteric for it. Maybe I care too much about things looking excellent. Maybe Claude is just good enough at debugging anything I'm facing. But I'm not interested in anything realtime/live if it means giving up on output quality. Overleaf has a live preview mode right there. I think a lot of the "problems with latex" are actually just problems with people developing packages in isolation in a decentralized manner. I don't care for LaTeX the language, or its implementation, but I do care for the output a whole lot, in a way that not the font alone or any approximation can reach. Maybe it's just inside baseball and people don't care about advanced typography. If I'm working on a piece of research that will carry my name on it, waiting for that compile is worth it. I still remember Scholarly Markdown. I might genuinely one day implement some other transpiler so I can always do my last pass in LaTeX. As long as your thing emits some decent IR, I'm happy.
Typst math output looks nearly identical to TeX math output. I have a simple comparison at https://lwn.net/Articles/1037577/. This is to be expected, as they began with the same algorithm.
13 comments
[ 5.7 ms ] story [ 43.5 ms ] threadTectonic is forked from the XeTeX TeX engine. However, the build process for XeTeX (and all other mainstream TeX engines) is extremely baroque:
1. The original Knuth sources are written in a language called WEB (which is basically used by nobody else)
2. Those sources are then patched with a series of “change files” due to restrictions on distributing modified sources
3. These patched files are then converted to the Pascal language using some custom processing scripts
4. That Pascal code is then converted to C code using additional custom tools
5. Those C files are then compiled against a set of extension files and libraries written natively in C and C++
One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process. Tectonic is based on the C/C++ files that emerge from the XeTeX build process, and is gradually translating that code to modern Rust.
While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time. This repository recreates the final XeTeX C/C++ files from their source, so that modifications can be ported into the main Tectonic codebase.
https://github.com/tectonic-typesetting/tectonic-staging/blo...
(Disclaimer: I'm on the TeX Live team)
Yes, the build process is rather unusual internally, but this is pretty well insulated from the user. The "standard" build commands used by lots of other open source projects
should mostly just work, and give you a functioning TeX engine at the end. The full steps are listed at [0] if you're interested in more details though. And TeX Live is regularly built on essentially every platform imaginable [1], so it's pretty unlikely that you'll ever need to work with the low-level build system.And only some of the engines require this complex build process; LuaTeX is the recommended engine these days [2], and it's written entirely in C (with a few C++ libraries), and it uses a standard autotools build process [3].
> One of the big motivators for the launch of the Tectonic project was to break out of this ridiculously unwelcoming development process.
TeX development is pretty welcoming in my personal experience: the first time that I built TL (~5 years ago), I was able to follow the official instructions without any problems, and I got a working TeX engine by the end of it. And it was only 2 or 3 years after that that I became an "official" member of the TL team, so I like to think that the development process is fairly welcoming.
But if you have any suggestions on what we can do better, please let me know, either by replying to this comment or to the email address linked in my profile.
> Tectonic is based on the C/C++ files that emerge from the XeTeX build process
I haven't looked at the Tectonic source, but the problem is that the C code generated by Web2C is fairly unreadable, so it's not really very usable as source code.
> While the core TeX architecture and XeTeX are largely stable, they do, however, evolve with time.
XeTeX is essentially frozen for the time being [4], unless someone steps up to maintain it. We are cautiously optimistic regarding LLMs though: they seem to be fairly decent at writing WEB code, and since even the TL development team only has 2 developers who are proficient in WEB, we'll take any help that we can get. (We don't have any non-trivial AI-written code yet, but it's definitely something that we're looking into)
[0]: https://tug.org/texlive/build.html
[1]: https://ftp.math.utah.edu/pub/texlive-utah/
[2]: https://www.latex-project.org/news/2024/11/01/issue40-of-lat...
[3]: https://gitlab.lisn.upsaclay.fr/texlive/luatex
[4]: https://tex.stackexchange.com/a/755239/270600
Nowadays, with Typst existing, it's vital for TeX ecosystem to solve these issues, since none of them are present in Typst. Projects like Tectonic would solve this for TeX, but they lack enough hands and (maybe) financial support.
Otherwise, using TeX only makes sense nowadays only if 1) you already have some templates 2) some features are still missing in Typst 3) you are just forced to use TeX/LaTeX for whatever reason.
Tectonic is a cool project, but hasn't seen any significant changes in a few years---and likely won't anytime soon. It seems we maintainers don't have the time and motivation to put serious work into Tectonic.
I haven't looked at the code in years (and thus may be wrong), but here's a quick overview:
Tectonic's code consists of thin bindings to /harfbuzz/graphite/etc and a vendored XeTeX (in C, with some tweaks to make the build easier), driven by Rust that tries to keep the TeX environment predictable and sane. A few components have been fully ported to Rust (bibtex, spx2html), but the project is very unfinished.
I've looked into the dark corners of TeX when I worked on Tectonic, and it is not pretty. TeX relies on a stack of evil hacks and esoteric behavior that is very hard to replicate, and very difficult to expose in an ergonomic way. This is true of the core system, and of many packages on CTAN.
A quick example: code highlighting does not work in Tectonic. The canonical solution is https://ctan.org/pkg/minted, which spawns a python process to style your code. Reproducibility is one of Tectonic's selling points, so we cannot replicate this behavior.
With https://typst.app/ as good as it is, there's little motivation to modernize TeX---especially considering the effort required. Typst _is_ modern TeX, and I'd rather spend my time there.
Thank you for fully fixing LaTeX for me.
It reminds me of that very embarrassing dance we used to do around 2008 :)
https://it.wikipedia.org/wiki/Tecktonik