The definition of compiler i learned was “takes some code, translate it to semantically equivalent code in a different language (which might be machine language, bytecode…)”. This is also used in PLaI, a respected learning resource: https://www.plai.org/
I think this is a pretty acceptable definition, and yes, it does make the term transpiler a little useless.
> Compilers already do things that “transpilers” are supposed to do. And they do it better because they are built on the foundation of language semantics instead of syntactic manipulation.
IMO: Transpilers are compilers, but not all compilers are transpilers.
In my book, transpilers are compilers that consume a programming language and target human-readable code, to be consumed by another compiler or interpreter (either by itself, or to be integrated in other projects).
i.e. the TypeScript compiler is a transpiler from TS to JS, the Nim compiler is a transpiler from Nim to C, and so on.
I guess if you really want to be pedantic, one can argue (with the above definition) that `clang -S` might be seen as a transpiler from C to ASM, but at that point, do words mean anything to you?
Whenever someone argues the uselessness or redundancy of a particular word, a helpful framework to understand their perspective is "Lumpers vs Splitters" : https://en.wikipedia.org/wiki/Lumpers_and_splitters
An extreme caricature example of a "lumper" would just use the word "computer" to label all Turing Complete devices with logic gates. In that mindset, having a bunch of different words like "mainframe", "pc", "smartphone", "game console", "FPGA", etc are all redundant because they're all "computers" which makes the various other words pointless.
On the other hand, the Splitters focus on the differences and I previously commented why "transpiler" keeps being used even though it's "redundant" for the Lumpers : https://news.ycombinator.com/item?id=28602355
We're all Lumpers vs Splitters to different degrees for different topics. A casual music listener who thinks of orchestral music as background sounds for the elevator would be "lump" both Mozart and Bach together as "classical music". But an enthusiast would get irritated and argue "Bach is not classical music, it's Baroque music. Mozart is classical music."
The latest example of this I saw was someone complaining about the word "embedding" used in LLMs. They were asking ... if an embedding is a vector, why didn't they just re-use the word "vector"?!? Why is there an extra different word?!? Lumpers-vs-splitters.
in addition to that, some people just seem to have an extreme aversion to neologisms. I remember being surprised by that when ajax (the web technology) first came out and there was a huge "why does this thing which is just <I honestly forget what it was 'just'> need its own name?" faction.
Transpilers are compilers that translate from one programming language to the other. I am not 100% sure where these "lies" come from, but it's literally in the name, it's clearly a portmanteau of translating compiler... Where exactly are people thinking the "-piler" suffix comes from?
Yes, I know. You could argue that a C compiler is a transpiler, because assembly language is generally considered a programming language. If this is you, you have discovered that there are sometimes concepts that are not easy to rigorously define but are easy for people to understand. This is not a rare phenomenon. For me, the difference is that a transpiler is intending to target a programming language that will be later compiled by another compiler, and not just an assembler. But, it is ultimately true that this definition is still likely not 100% rigorous, nor is it likely going to have 100% consensus. Yet, people somehow know a transpiler when they see one. The word will continue to be used because it ultimately serves a useful purpose in communication.
so is the thing that generates bytecode from java or c# source code a compiler or a transpiler? Because the generated code will be compiled by the JIT at runtime.
I don't really understand the reasoning in the article. Nobody argues that orange is a meaningless word just because it's not wrong to call an orange a fruit.
Sure, a transpiler is a specialized form of compiler. However that doesn't mean it's not much clearer to describe a transpiler using the more specific name. As such recommending someone replace "compiler" with "transpiler" (when appropriate) does not mean using compiler is wrong. It simply means that, outside of some very niche-interest poetry, using transpiler is better!
I don't understand what the issue is: a transpiler is a compiler that outputs in a language that human programmers use.
It's good to be aware of that from an engineering standpoint, because the host language will have significantly different limitations, interoperability and ecosystem, compared to regular binary or some VM byte-code.
Also, I believe that they are meaningfully different in terms of compiler architecture. Outputting an assembly-like is quite different from generating an AST of a high-level programming language. Yes of course it's fuzzy because some compilers use intermediate representations that in some cases are fairly high-level, but still they are not meant for human use and there are many practical differences.
It's a clearly delineated concept, why not have a word for it.
Meaningless word + list of "lies"...
Nice read anyways.
"BabelJS is arguably one of the first “transpilers” that was developed so that people could experiment with JavaScript’s new language features that did not yet have browser implementations"
Just my two cents. Haxe was created long time ago, and BabelJS is arguably not one of the first "transpilers" people can play with.
Language interoperability is a material question. Outputting Javascript, Python, C++ vs assembler/machine code have very different implications for calls to/from other languages
Is JIT also meaningless?
But ultimately if you don’t want to use a word, don’t use it. Not wanting to hear a word says more about the listener than the speaker
I think the distinction is meaningful - for example many compilers used to have C backends (GCC for example did) - so you code went through almost the entire compiler pipeline - from frontend to IR to backend where the backend did almost everything a compiler does, it only skipped target machine specific stuff like register allocation (possibly even that was done), arch specific optimizations and assembly generation.
A transpiler to me focuses on having to change or understand the code as little as possible - perhaps it can operate on the syntax level without having to understand scopes, variable types, the workings of the language. It does AST->AST transforms (or something even less sophisticated, like string manipulation).
In my mind, you could have a C++ to C transpiler (which removes C++ constructs and turns them into C ones, although C++ is impossible to compile without a rich understanding of the code), and you could have a C++ to C compiler, which would be a fully featured compiler, architected in the way I described in the start of the post, and these would be two entirely different pieces of software.
So I'd say the term is meaningful, even if not strictly well defined.
I think the note about generators may be a good definition for when one language is "more powerful" than another; at least it's a good heuristic:
> The input and output languages have the syntax of JavaScript but the fact that compiling one feature [generators] requires a whole program transformation gives away the fact that these are not the same language. If we’re to get beyond the vagaries of syntax and actually talk about what the expressive power of languages is, we need to talk about semantics.
If a given program change is local in language X but global in language Y, that is a way in which language X has more expressive power.
This is kind of fuzzy because you can virtually always avoid this by implementing an interpreter, or its moral equivalent, for language X in language Y, and writing your system in that DSL (embedded or otherwise), rather than directly in language Y. Then, that anything that would be a local change in language X is still a local change. But this sort of requires knowing ahead of time that you're going to want to make that kind of change.
> Lie #3: Transpilers Target the Same Level of Abstraction
> This is pretty much the same as (2). The input and output languages have the syntax of JavaScript but the fact that compiling one feature requires a whole program transformation gives away the fact that these are not the same language
It is not really the same as (2), you can't cherry pick the example of Babel and generalise it to every transpiler ever. There are several transpilers which transpile from one high-level language to another high-level language such as kotlin to swift. i.e; targeting the same level of abstraction.
Wonder what this person would say about macro expansions in scheme, maybe that should also be considered a compiler as per their definition.
"Programming languages are not just syntax; they have semantics too. Pretending that you can get away with just manipulating the former is delusional and results in bad tools."
So eloquently put, what starts off as just simple syntactic conversion usually snowballs into semantics very quickly.
I think it's pretty clear to anyone with experience in the field that the notion of compilers, interpreters, transpilers are porous and even more so when you had the concept of VM ("Let's interpret a compiled artefact") and JIT ("I put a compiler in your interpreter. Don't worry, it all runs on the same VM in the end.")
These things live on a continuum. Still, I think the different worlds are useful. They put forward different concepts and ideas. It helps framing things.
A compiler takes in one language and outputs some other language. E.g. C to LLVM IR or LLVM IR to x86_64 assembly.
An assembler is a type of compiler that takes in an assembly language and outputs machine code.
A transpiler is a type of compiler that takes in a language commonly used by humans to directly write programs and outputs another language commonly used by humans to directly write programs. E.g. c2rust is a C to unsafe Rust compiler, and since both are human-used languages it's a transpiler. Assembly language isn't commonly written by humans though it used to be, so arguably compilers to assembly language are no longer transpilers even though they used to be.
The existence of a transpiler implies a cispiler, a compiler that takes in code in one language and outputs code in that same language. Autoformatters are cispilers.
It's basically a flowchart showing all of the different things that we mean when we say compiler/interpreter/transpiler, and which bits they have in common.
Funny, but it has two paths for transpiler - the kind that parses and outputs source from an AST, and the asm.js kind, that actually just uses a high-level language as an assembly-ish target.
I think the term transpiler is ok. It’s not pedagogical or anything but most engineering jargon is like that, and this defiantly isn’t the worst one I’ve seen.
I always understood transpilers to be defined by what they do, not how they work. Whether it's implemented as an AST transformation or something more complex would be irrelevant.
34 comments
[ 2.6 ms ] story [ 47.2 ms ] threadI think this is a pretty acceptable definition, and yes, it does make the term transpiler a little useless.
So you do know the difference.
In my book, transpilers are compilers that consume a programming language and target human-readable code, to be consumed by another compiler or interpreter (either by itself, or to be integrated in other projects).
i.e. the TypeScript compiler is a transpiler from TS to JS, the Nim compiler is a transpiler from Nim to C, and so on.
I guess if you really want to be pedantic, one can argue (with the above definition) that `clang -S` might be seen as a transpiler from C to ASM, but at that point, do words mean anything to you?
An extreme caricature example of a "lumper" would just use the word "computer" to label all Turing Complete devices with logic gates. In that mindset, having a bunch of different words like "mainframe", "pc", "smartphone", "game console", "FPGA", etc are all redundant because they're all "computers" which makes the various other words pointless.
On the other hand, the Splitters focus on the differences and I previously commented why "transpiler" keeps being used even though it's "redundant" for the Lumpers : https://news.ycombinator.com/item?id=28602355
We're all Lumpers vs Splitters to different degrees for different topics. A casual music listener who thinks of orchestral music as background sounds for the elevator would be "lump" both Mozart and Bach together as "classical music". But an enthusiast would get irritated and argue "Bach is not classical music, it's Baroque music. Mozart is classical music."
The latest example of this I saw was someone complaining about the word "embedding" used in LLMs. They were asking ... if an embedding is a vector, why didn't they just re-use the word "vector"?!? Why is there an extra different word?!? Lumpers-vs-splitters.
Yes, I know. You could argue that a C compiler is a transpiler, because assembly language is generally considered a programming language. If this is you, you have discovered that there are sometimes concepts that are not easy to rigorously define but are easy for people to understand. This is not a rare phenomenon. For me, the difference is that a transpiler is intending to target a programming language that will be later compiled by another compiler, and not just an assembler. But, it is ultimately true that this definition is still likely not 100% rigorous, nor is it likely going to have 100% consensus. Yet, people somehow know a transpiler when they see one. The word will continue to be used because it ultimately serves a useful purpose in communication.
Sure, a transpiler is a specialized form of compiler. However that doesn't mean it's not much clearer to describe a transpiler using the more specific name. As such recommending someone replace "compiler" with "transpiler" (when appropriate) does not mean using compiler is wrong. It simply means that, outside of some very niche-interest poetry, using transpiler is better!
It's good to be aware of that from an engineering standpoint, because the host language will have significantly different limitations, interoperability and ecosystem, compared to regular binary or some VM byte-code.
Also, I believe that they are meaningfully different in terms of compiler architecture. Outputting an assembly-like is quite different from generating an AST of a high-level programming language. Yes of course it's fuzzy because some compilers use intermediate representations that in some cases are fairly high-level, but still they are not meant for human use and there are many practical differences.
It's a clearly delineated concept, why not have a word for it.
"BabelJS is arguably one of the first “transpilers” that was developed so that people could experiment with JavaScript’s new language features that did not yet have browser implementations"
Just my two cents. Haxe was created long time ago, and BabelJS is arguably not one of the first "transpilers" people can play with.
[1] https://en.wikipedia.org/wiki/Haxe
[2] https://haxe.org
Is JIT also meaningless?
But ultimately if you don’t want to use a word, don’t use it. Not wanting to hear a word says more about the listener than the speaker
A transpiler to me focuses on having to change or understand the code as little as possible - perhaps it can operate on the syntax level without having to understand scopes, variable types, the workings of the language. It does AST->AST transforms (or something even less sophisticated, like string manipulation).
In my mind, you could have a C++ to C transpiler (which removes C++ constructs and turns them into C ones, although C++ is impossible to compile without a rich understanding of the code), and you could have a C++ to C compiler, which would be a fully featured compiler, architected in the way I described in the start of the post, and these would be two entirely different pieces of software.
So I'd say the term is meaningful, even if not strictly well defined.
I think the note about generators may be a good definition for when one language is "more powerful" than another; at least it's a good heuristic:
> The input and output languages have the syntax of JavaScript but the fact that compiling one feature [generators] requires a whole program transformation gives away the fact that these are not the same language. If we’re to get beyond the vagaries of syntax and actually talk about what the expressive power of languages is, we need to talk about semantics.
If a given program change is local in language X but global in language Y, that is a way in which language X has more expressive power.
This is kind of fuzzy because you can virtually always avoid this by implementing an interpreter, or its moral equivalent, for language X in language Y, and writing your system in that DSL (embedded or otherwise), rather than directly in language Y. Then, that anything that would be a local change in language X is still a local change. But this sort of requires knowing ahead of time that you're going to want to make that kind of change.
Sadly https://people.csail.mit.edu/files/pubs/stopify-pldi18.pdf is 403. But possibly https://people.csail.mit.edu/rachit/files/pubs/stopify-pldi1... is the right link.
When used, it has often been implied that a compiler that outputs to a human-readable programming language wouldn't be a "real compiler".
> This is pretty much the same as (2). The input and output languages have the syntax of JavaScript but the fact that compiling one feature requires a whole program transformation gives away the fact that these are not the same language
It is not really the same as (2), you can't cherry pick the example of Babel and generalise it to every transpiler ever. There are several transpilers which transpile from one high-level language to another high-level language such as kotlin to swift. i.e; targeting the same level of abstraction.
Wonder what this person would say about macro expansions in scheme, maybe that should also be considered a compiler as per their definition.
So eloquently put, what starts off as just simple syntactic conversion usually snowballs into semantics very quickly.
Poly-transpiler? It will also trigger more people.
These things live on a continuum. Still, I think the different worlds are useful. They put forward different concepts and ideas. It helps framing things.
An assembler is a type of compiler that takes in an assembly language and outputs machine code.
A transpiler is a type of compiler that takes in a language commonly used by humans to directly write programs and outputs another language commonly used by humans to directly write programs. E.g. c2rust is a C to unsafe Rust compiler, and since both are human-used languages it's a transpiler. Assembly language isn't commonly written by humans though it used to be, so arguably compilers to assembly language are no longer transpilers even though they used to be.
The existence of a transpiler implies a cispiler, a compiler that takes in code in one language and outputs code in that same language. Autoformatters are cispilers.
It's basically a flowchart showing all of the different things that we mean when we say compiler/interpreter/transpiler, and which bits they have in common.
Funny, but it has two paths for transpiler - the kind that parses and outputs source from an AST, and the asm.js kind, that actually just uses a high-level language as an assembly-ish target.
For example, Google Scholar search for "transpiler" yields just 3200 results, compared to ~1.4M for "compiler".