pgrust sounds very interesting, but with the deep changes there’s no viable path to upstream it - is the end goal to be robust enough that it’ll get wide adoption?
Is it really interesting though? It's essentially just vibe-coded by people who are unqualified for this kind of work. One of the authors claimed that what qualified them was having worked on a large-scale postgres cluster; they never actually worked on databases or compilers.
Sure, but it still means that the OS has to decide who is allowed to do it and to what extent. Sophisticated worms like Stuxnet would be much harder with strict W^X for example, since CVE-2010-2568 and the like would be much harder to execute.
Capabilities are a way to control that, and the point being that only responsible proven applications get the certificate, hence how it all goes on iOS.
You're making the assumption that "responsible" is something provable, but that is not the case, it is specially not easy to prove software is secure from tampering its behaviour.
Nonsense, there's no "system wide implications". Mappings are per process, and W^X is just a strategy to help harden individual processes, not the entire system. There's no herd immunity here.
JITs do not grant the ability to bypass any OS/system sandboxes. The lack of W^X doesn't do that, either. If a process opts out of W^X, such as to enable a JIT, it's voluntarily making itself less hardened, but at the end of the day this isn't any more meaningful than the program being allowed to be written in, say, C, which also voluntarily reduces the processes security hardening.
No, you're not understanding mine. Nobody builds an OS/system expecting that every executable is perfectly well behaved with zero bugs and zero ill intent. Applications are allowed to run code. JITs just run code in that same process. They are already limited to what the process was already allowed to do in the first place.
And my point about C is literally that even without a JIT, applications can still have arbitrary execution vulnerabilities.
A JIT intended to run untrusted code as part of a sandbox, like a browser, is a big risk. But that's because of the untrusted code part, not the JIT. By comparison, something like a Python or Java JIT is as near as makes no difference completely risk free. The JIT is working on exclusively "trusted" code. Same basic concept applies here with this database usage.
Yes, but your claim is that applications can still have arbitrary execution vulnerabilities only on operating systems that allow dynamic executable code.
I am refuting that statement by showing that not allowing dynamic executable code is not enough. It is irrelevant to that end that you can mitigate ROP by other means.
If a process is allowed to execute code, it's always allowed to execute arbitrary code as well. Those two are fully intertwined, be it via dynamic executable code, ROP chains, because it has an interpreter, or because the initial binary itself already is the malicious payload in the first place.
To the OS those are all identical scenarios, it's irrelevant what caused the arbitrary code execution to happen.
You're entirely correct because JIT requires violating Write xor Execute security policy. This is the reason on iOS, it is limited to Apple shipped software.
> - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support.
> - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […]
> - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […]
As someone who has written a jit compiler, I am puzzled by the claim that jitting requires write/execute permissions. When I have written a jit, I loaded some memory with read/write permissions using mmap. Once I filled in the generated code, I mprotected the region to read/execute before executing.
The drawback to this approach is there can be some bloat because you can only mprotect at page granulariy so a jitted function that only takes say 10 bytes to represent would take up a full page in memory, but this is extreme and in practice, the overhead is unlikely to be worth worrying about.
Read the code in the post. Everything is written in unsafe Rust. The assembly itself knows no memory safety at all and is completely up to the programmer skill whether it can be trusted to not mess up.
But then we'd have to admit that it's not the JIT that's a problem, it's the lack of guardrails and analysis features in the machine code interfaces that higher level languages expose!
So what, are you willing to go away from von-neumann architecture where instructions are data and data are instructions, i.e. the instruction-data hominocity that underpins JIT compilation? Are you willing to go to a pseudo-Harvard architecture where the ability of JIT compiling is soft locked by other means like VM or strong code authentication or policy protection, which is what Apple is doing.
Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT
There's more to it than that though. Defects in JIT logic can lead to nasty low-level bugs. Plain old interpreters, especially if written in a safe language, are unlikely to have similar issues. This is important when the input code is untrusted. JIT bugs are a major source of browser vulnerabilities.
JITs are an attack surface in that process. They are still restricted to things that process was already allowed to do, no matter how badly implemented the JIT is.
In this example usage, one would hope that authentication already happened before the JIT processed the command. So an authorized user can attack themselves is the only realistic risk, which is hardly significant
There's plenty of scope for harm just within the process, even ignoring the possibility of escaping the process. In the case of a database server, essentially everything of value takes place within the database process (or processes). That process presumably has both access to the raw database data, and network access. We wouldn't want it sending data to an attacker's server.
> one would hope that authentication already happened before the JIT processed the command
We'd hope, yes, but SQL injection issues are still somewhat common. Also, an organisation might trust their DBMS to enforce permissions, and a JIT bug is the kind of thing that might allow non-permissioned data access. A DBMS should be hardened against malicious queries, just as a browser should be hardened against malicious JavaScript.
In web browsers, the numbers show JIT compilers are a major cause of security issues. I don't know if there are hard numbers on JIT engines causing security issues in DBMSs though.
> There's plenty of scope for harm just within the process,
Of course, but the process has every right to decide for itself if it wants to take that risk. Just like it decides if it wants to take the risk of a memory unsafe language, forgoing fuzzing, or going all out with formal verification.
Uhm, Common Lisp, where JIT is not only available but is also manageable: the programmer can decide what deserves to be compiled and what does not.
Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
Compiling to bytecode is still compilation. An interpreter would operate directly on the S-expressions. Obviously, this makes some constructs (such as TAGBODY) very inefficient.
You can turn that off by setting or binding sb-ext:*evaluator-mode* to :interpret.
Also, on my machine, compiling the identity lambda form takes about 200 usec with (optimize (compilation-speed 3) (debug 0)). SBCL could use a faster JIT mode, perhaps at compilation-speed 3/speed 0. Perhaps there are some other internal special variables that could be tweaked to reduce compile time.
Reminds me of the 2024 blog post Look ma, I wrote a new JIT compiler for PostgreSQL [0]. Both articles lament that Postgres's LLVM-based JIT [1] takes a while to generate code.
> The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile.
That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM.
Thanks, sljit looks somewhat similar to GNU lightning.
On reflection I wonder if I overstated the widespread use of JIT and of JIT compiler frameworks. All the 'major' well-resourced high-profile JIT-based interpreters I can think of don't use an off-the-shelf JIT framework for their backend, which makes sense as they want to carefully tune the code-generation. OpenJDK, OpenJ9, .Net, V8, SpiderMonkey, JavaScriptCore. LuaJIT and Python's new JIT don't use one either, nor does the Linux kernel's BPF engine.
The Guile Scheme interpreter uses a fork of the GNU Lightning JIT library. [0] Julia and (as mentioned) Postgres use LLVM for their JITs. I'm trying to think of other projects that use a JIT framework/library.
Similarly, I can't think of many problem domains where it makes sense to use JIT. The ones that spring to mind are interpreters (of course), regex engines, and DBMSs. JIT can also help in high-performance computing, to tailor the code to the particular problem and the particular CPU. [1] I don't think there are many other contexts where it makes sense to use JIT though.
JIT compilation brings its own drawbacks in portability (both between hardware platforms and operating systems), complexity, and perhaps cybersecurity, which might also limit its adoption, even if a good JIT framework could help with all three.
I think that the "manual JIT compilation" that Common Lisp provide is the most practical compromise here. Sure, you don't have the automated switching between bytecode execution and progressively optimized compilation and you need to manually track runtime typing information to feed to the compiler, but the machinery is so much simpler and builtin!
LLVM isn't the only game in town, nobody using it (or libgccjit) for JIT should be surprised to see relatively long compile times. I wonder if the postgres project will try a different backend.
There's a strong 'diminishing returns' effect in striking a balance between compile time and the performance of the generated code. I'd expect a more lightweight (less optimising) JIT engine to be able to produce code with pretty respectable performance while taking only a fraction of the time that LLVM takes. There's a follow-up to the blog post I linked above, which bears this out. [0] (I don't know if that JIT solution is production-ready or viable for merging into postgres, mind.)
The blog post [0] gives this performance comparison table:
Not sure where the idea comes from that Cranelift is much faster than LLVM -O0, at least in our experiments in 2024 it wasn't, see [1] Fig. 6.
Template-based code generators suffer from bad code quality due to missing register allocation.
Our TPDE-based compilers compile a bit slower than template-based code generation but the generated code is much smaller and faster ([2] Fig. 2). Also for database workloads ([2] Fig. 6).
All that said, Postgres' main limitation is that it (IIRC) only compiles single expressions from operators, not pipelines. This fundamentally limits the achievable performance improvement compared to databases that perform more extensive query compilation.
Always good to have proper researchers in the thread.
> Not sure where the idea comes from that Cranelift is much faster than LLVM -O0
Cranelift describes itself as a fast, secure, relatively simple and innovative compiler backend. [0] Interesting that LLVM can compete there, with its optimisations dialed down.
> Postgres' main limitation is that it (IIRC) only compiles single expressions from operators, not pipelines. This fundamentally limits the achievable performance improvement compared to databases that perform more extensive query compilation.
That sounds pretty limiting. That's separate from query optimisation though, right? The query optimiser is presumably able to reason 'broadly' and not just at the level of individual expressions? High-level query-plan optimisation must be much more consequential than effective use of JIT compilation.
> That's separate from query optimisation though, right? The query optimiser is presumably able to reason 'broadly' and not just at the level of individual expressions? High-level query-plan optimisation must be much more consequential than effective use of JIT compilation.
Yes, yes, and yes. For databases, query optimization (esp. join ordering for larger queries, which heavily depends on estimates) is fundamental. Query optimization happens at the level of the query plan, JIT compilation is only relevant afterwards. A bad query plan leads to asymptotically worse performance (e.g., bad join ordering with huge intermediate results).
On query plan execution: The "classical" model as used in e.g. Postgres is a pull-based iterator model, where operators implement a next() method yielding the next tuple and in there recursively call next() on their child operators (e.g., a next() of a select operator calls next() on its child operator, then applies the predicate [what Postgres JIT-compiles], and returns the tuple if the predicate was true). This can happen one tuple at a time (Postgres) or "vectorized" where multiple tuples are processed at once (e.g. DuckDB). A query-compiling database will split the tree into pipelines and compile each pipeline as one function (e.g., a pipeline will iterate over all the tuples from a source (e.g. tablescan) and a select operator then becomes an if statement inside that loop). This results in pretty tight loops, avoids per-tuple dispatch overhead, and enables more optimizations inside the JIT-ted code (e.g., tuple values don't need to be reloaded from memory all the time). (I find the original paper on query compilation [1] to be well readable.)
There’s been a meme circulating about how AI doesn’t help because “code was never the hard part.” I think that’s true in some domains, but in others, writing the code absolutely was the hard part. JIT compilers are a great example of that.
This is absolutely a JIT-compiler. It compiles code into machine code. This is a surprisingly efficient way to get noticeable speedup relative to interpretation. Also it is much safer than proper optimising compiler. Say ebpf jit-compiler functions very similarly, because it is fast and _secure_ way to jit. (well, there's a bit of cheating because before emitting bpf bytecode it goes through gcc/clang pipeline).
LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
To back up the "surprisingly efficient way to get a speed-up" thing: I once wrote a toy compiler, without an optimizer and without even a register allocator (so all variables lived in stack memory). In my test benchmarks it was roughly 4x slower than Clang at -O3, IIRC.
That's not exactly blazing fast for a low level C-like language, but it's not bad. It's infinitely faster than what I've ever gotten a toy interpreter to be.
By choosing LLVM you're also taking a serious latency hit, and potentially burning a ton of CPU cycles on optimizations that will never apply. JSC, for instance, implemented LLVM for FTLJIT (which is where many of the JS bits jangling around in LLVM originated from), but it was only useful for code that was highly likely to benefit from the optimizations because of the high cost of compilation. The webkit folks have since ripped out LLVM and replaced it with their own specialized JIT, which is essentially just what's demonstrated here (with some optimization passes).
> By not using LLVM, you're missing all the optimizations it does
And yet, a good portion of software that runs today's world is written in scripting languages & executed using interpreters.
Which is okay! Imho: multiply [# of users] with [how often each user sees that software's effect] and [how much that contributes to the overall user experience], then you get a ballpark idea of how much $$/effort is worth spending on optimization.
In other words: for a one-off, don't bother. But as usercount, frequency of use by individual users, poor UX or RAM/CPU consumption goes up, progress from script -> compiled -> optimizing compiler -> (if necessary) hand-optimized assembly as needed. And of course consider high-level design, data structures, algorithms etc in that process. A change there might be more effective than a switch from interpreted -> optimizing compiler.
"Developer time" should not factor into that much (again: imho) unless users=developers.
Thoughtlessly putting every change through a (slow?) pipeline that does 'random' toolbox-of-optimizations without need, is wasteful. Apply that toolbox as needed while keeping the above in mind.
Sounds like a "no true Scotsman" statement. How are you defining "real"?
Some human, somewhere, has to describe how to turn high-level language constructs into machine code. "When you see this pattern, emit this sequence of bytes." That's just templates and stencils. There's no magic for turning source code into machine code by divining the ISA at compile time.
Anything that's taking source code and, at the time of execution, is compiling it to machine code on-the-fly is JIT compilation. Regardless how long it takes, lack of optimization, or which machine is the target (x86, ARM32/64, RISC-V, JVM, WebAssembly), it's JIT.
"at the time of execution" could mean that your program never runs an interpreter and instead compiles the input code before running any of it, basically like the --run option in the tiny c compiler
This is perhaps a little meta, but this was a pleasant read. It’s refreshing to read an article about using an LLM that doesn’t read like it was also written by that LLM.
I might use this approach to generate the stencils for a JIT firewall I’ve been experimenting with.
It also occurs to me that this could be used to generate eBPF byte code on the fly as well
88 comments
[ 1.7 ms ] story [ 47.9 ms ] threadhttps://en.wikipedia.org/wiki/W%5EX
It has to do that anyway?
Which at this point most companies would rather save money and forbid JIT altogether.
Note that mainframes and micros have JIT environments that aren't at the same safety level as regular desktop PCs.
JITs do not grant the ability to bypass any OS/system sandboxes. The lack of W^X doesn't do that, either. If a process opts out of W^X, such as to enable a JIT, it's voluntarily making itself less hardened, but at the end of the day this isn't any more meaningful than the program being allowed to be written in, say, C, which also voluntarily reduces the processes security hardening.
And my point about C is literally that even without a JIT, applications can still have arbitrary execution vulnerabilities.
A JIT intended to run untrusted code as part of a sandbox, like a browser, is a big risk. But that's because of the untrusted code part, not the JIT. By comparison, something like a Python or Java JIT is as near as makes no difference completely risk free. The JIT is working on exclusively "trusted" code. Same basic concept applies here with this database usage.
Only on operating systems that allow dynamic executable code.
I am refuting that statement by showing that not allowing dynamic executable code is not enough. It is irrelevant to that end that you can mitigate ROP by other means.
To the OS those are all identical scenarios, it's irrelevant what caused the arbitrary code execution to happen.
https://en.wikipedia.org/wiki/W%5EX
> - Android Runtime Just-In-Time (JIT) compilation/profiling is fully disabled and replaced with full ahead-of-time (AOT) compilation. The only JIT compilation in the base OS is the V8 JavaScript JIT which is disabled by default for the Vanadium browser with per-site exception support.
> - Dynamic code loading for both native code or Java/Kotlin classes is blocked for nearly the entire base OS. […]
> - Dynamic code loading for both native code or Java/Kotlin classes can be disabled for user installed apps via 3 exploit protection toggles: […]
https://grapheneos.org/features
The drawback to this approach is there can be some bloat because you can only mprotect at page granulariy so a jitted function that only takes say 10 bytes to represent would take up a full page in memory, but this is extreme and in practice, the overhead is unlikely to be worth worrying about.
Alternatively, only allow for the execution of cryptographly signed static linked binaries, this naturally includes the interpreter above.
Fun fact: even Apple themselves have JIT. JavaScriptCore on iOS has JIT, it's just that the App Store policies forbid any application submissions with JIT or trying to mmap/mprotect an executable region. There used to be apps on TrollStore that runs JIT
Allowing code execution allows code execution, that's it, that's the entirety of it.
In this example usage, one would hope that authentication already happened before the JIT processed the command. So an authorized user can attack themselves is the only realistic risk, which is hardly significant
There's plenty of scope for harm just within the process, even ignoring the possibility of escaping the process. In the case of a database server, essentially everything of value takes place within the database process (or processes). That process presumably has both access to the raw database data, and network access. We wouldn't want it sending data to an attacker's server.
> one would hope that authentication already happened before the JIT processed the command
We'd hope, yes, but SQL injection issues are still somewhat common. Also, an organisation might trust their DBMS to enforce permissions, and a JIT bug is the kind of thing that might allow non-permissioned data access. A DBMS should be hardened against malicious queries, just as a browser should be hardened against malicious JavaScript.
In web browsers, the numbers show JIT compilers are a major cause of security issues. I don't know if there are hard numbers on JIT engines causing security issues in DBMSs though.
Of course, but the process has every right to decide for itself if it wants to take that risk. Just like it decides if it wants to take the risk of a memory unsafe language, forgoing fuzzing, or going all out with formal verification.
Besides run time, JIT is available also when the code is compiled or loaded for execution (i.e., do you have a compilation or loading speed-up in mind? no problem, you can also compile that speed-up into native machine code, and so ad infinitum...).
[0] https://www.sbcl.org/manual/#compiler-only-implementation
Also, on my machine, compiling the identity lambda form takes about 200 usec with (optimize (compilation-speed 3) (debug 0)). SBCL could use a faster JIT mode, perhaps at compilation-speed 3/speed 0. Perhaps there are some other internal special variables that could be tweaked to reduce compile time.
Mine doesn't have the SB-INTERPRETER package, so I doubt binding the variable has an effect.
> The rarity of JIT compilers makes me believe that implementing a JIT compiler historically was too difficult for it to be worthwhile.
That's only true of writing a JIT from scratch. There's no rarity of JITs, it's just that LLVM (and other frameworks) are often used. Every major interpreter has a JIT compiler. PCRE2 has a JIT compiler. There are JIT frameworks out there with much faster code-generation than LLVM: Cranelift, GNU Lightning, Mir. I doubt they could do code-generation faster than a custom copy-and-patch JIT, but they'd be much faster than LLVM.
[0] https://www.pinaraf.info/2024/03/look-ma-i-wrote-a-new-jit-c... , discussed: https://news.ycombinator.com/item?id=39742916
[1] https://www.postgresql.org/docs/current/jit-reason.html
On reflection I wonder if I overstated the widespread use of JIT and of JIT compiler frameworks. All the 'major' well-resourced high-profile JIT-based interpreters I can think of don't use an off-the-shelf JIT framework for their backend, which makes sense as they want to carefully tune the code-generation. OpenJDK, OpenJ9, .Net, V8, SpiderMonkey, JavaScriptCore. LuaJIT and Python's new JIT don't use one either, nor does the Linux kernel's BPF engine.
The Guile Scheme interpreter uses a fork of the GNU Lightning JIT library. [0] Julia and (as mentioned) Postgres use LLVM for their JITs. I'm trying to think of other projects that use a JIT framework/library.
Similarly, I can't think of many problem domains where it makes sense to use JIT. The ones that spring to mind are interpreters (of course), regex engines, and DBMSs. JIT can also help in high-performance computing, to tailor the code to the particular problem and the particular CPU. [1] I don't think there are many other contexts where it makes sense to use JIT though.
JIT compilation brings its own drawbacks in portability (both between hardware platforms and operating systems), complexity, and perhaps cybersecurity, which might also limit its adoption, even if a good JIT framework could help with all three.
[0] https://doc.guix.gnu.org/guile/latest/en/html_node/Just_002d...
[1] https://www.intel.com/content/www/us/en/developer/articles/t...
See https://github.com/marcoheisig/Petalisp#why-is-petalisp-writ...
It was the limits of 8 bit home computers hardware that made the interpreter version be more widely known.
Same to Lisp, Smalltalk, and many other languages.
Fully agree with you.
Except that using LLVM has high latency limitting it's applicability. Postgres just disabled LLVM by default because of this[0].
[0] https://www.postgresql.org/message-id/E1w8GWU-002bSL-31%40ge...
There's a strong 'diminishing returns' effect in striking a balance between compile time and the performance of the generated code. I'd expect a more lightweight (less optimising) JIT engine to be able to produce code with pretty respectable performance while taking only a fraction of the time that LLVM takes. There's a follow-up to the blog post I linked above, which bears this out. [0] (I don't know if that JIT solution is production-ready or viable for merging into postgres, mind.)
The blog post [0] gives this performance comparison table:
[0] https://www.pinaraf.info/2025/12/jit-episode-iii-warp-speed-...Template-based code generators suffer from bad code quality due to missing register allocation.
Our TPDE-based compilers compile a bit slower than template-based code generation but the generated code is much smaller and faster ([2] Fig. 2). Also for database workloads ([2] Fig. 6).
All that said, Postgres' main limitation is that it (IIRC) only compiles single expressions from operators, not pipelines. This fundamentally limits the achievable performance improvement compared to databases that perform more extensive query compilation.
[1]: https://aengelke.net/pubs/2403-cgo.pdf [2]: https://aengelke.net/pubs/2602-cgo1.pdf
PS: sorry for the promotion of my own research here, just couldn't resist.
> Not sure where the idea comes from that Cranelift is much faster than LLVM -O0
Cranelift describes itself as a fast, secure, relatively simple and innovative compiler backend. [0] Interesting that LLVM can compete there, with its optimisations dialed down.
> Postgres' main limitation is that it (IIRC) only compiles single expressions from operators, not pipelines. This fundamentally limits the achievable performance improvement compared to databases that perform more extensive query compilation.
That sounds pretty limiting. That's separate from query optimisation though, right? The query optimiser is presumably able to reason 'broadly' and not just at the level of individual expressions? High-level query-plan optimisation must be much more consequential than effective use of JIT compilation.
[0] https://cranelift.dev/
Yes, yes, and yes. For databases, query optimization (esp. join ordering for larger queries, which heavily depends on estimates) is fundamental. Query optimization happens at the level of the query plan, JIT compilation is only relevant afterwards. A bad query plan leads to asymptotically worse performance (e.g., bad join ordering with huge intermediate results).
On query plan execution: The "classical" model as used in e.g. Postgres is a pull-based iterator model, where operators implement a next() method yielding the next tuple and in there recursively call next() on their child operators (e.g., a next() of a select operator calls next() on its child operator, then applies the predicate [what Postgres JIT-compiles], and returns the tuple if the predicate was true). This can happen one tuple at a time (Postgres) or "vectorized" where multiple tuples are processed at once (e.g. DuckDB). A query-compiling database will split the tree into pipelines and compile each pipeline as one function (e.g., a pipeline will iterate over all the tuples from a source (e.g. tablescan) and a select operator then becomes an if statement inside that loop). This results in pretty tight loops, avoids per-tuple dispatch overhead, and enables more optimizations inside the JIT-ted code (e.g., tuple values don't need to be reloaded from memory all the time). (I find the original paper on query compilation [1] to be well readable.)
[1]: https://www.vldb.org/pvldb/vol4/p539-neumann.pdf
By not using LLVM, you're missing all the optimizations it does.
LLVM is a large dependency if you need to JIT. There are plenty of smaller (and much faster) alternatives which are much better fit for smaller projects. Larger projects usually roll out their own jit-pipeline because they can integrate better with the source language/interpreter and apply tricks LLVM is not well suited to (say, LLVM is not great at deoptimisation). I think only Julia is really a heavy user of LLVM JIT, also it is known for extremely slow repl from time to time.
That's not exactly blazing fast for a low level C-like language, but it's not bad. It's infinitely faster than what I've ever gotten a toy interpreter to be.
And yet, a good portion of software that runs today's world is written in scripting languages & executed using interpreters.
Which is okay! Imho: multiply [# of users] with [how often each user sees that software's effect] and [how much that contributes to the overall user experience], then you get a ballpark idea of how much $$/effort is worth spending on optimization.
In other words: for a one-off, don't bother. But as usercount, frequency of use by individual users, poor UX or RAM/CPU consumption goes up, progress from script -> compiled -> optimizing compiler -> (if necessary) hand-optimized assembly as needed. And of course consider high-level design, data structures, algorithms etc in that process. A change there might be more effective than a switch from interpreted -> optimizing compiler.
"Developer time" should not factor into that much (again: imho) unless users=developers.
Thoughtlessly putting every change through a (slow?) pipeline that does 'random' toolbox-of-optimizations without need, is wasteful. Apply that toolbox as needed while keeping the above in mind.
Some human, somewhere, has to describe how to turn high-level language constructs into machine code. "When you see this pattern, emit this sequence of bytes." That's just templates and stencils. There's no magic for turning source code into machine code by divining the ISA at compile time.
Anything that's taking source code and, at the time of execution, is compiling it to machine code on-the-fly is JIT compilation. Regardless how long it takes, lack of optimization, or which machine is the target (x86, ARM32/64, RISC-V, JVM, WebAssembly), it's JIT.
It is very relevant
I might use this approach to generate the stencils for a JIT firewall I’ve been experimenting with.
It also occurs to me that this could be used to generate eBPF byte code on the fly as well