Interesting, this is very similar to llvmlite.Builder which is a wrapper over llvm. I am probably going to create something similar for my Python -> C -> assembly JIT.
The LLVM ORC and Clang-REPL projects would be worth checking out if you haven't already: there's a healthy community of high performance computing folks working in this space over at https://compiler-research.org.
In particular, this talk might be interesting:
"Unlocking the Power of C++ as a Service:
Uniting Python's Usability with C++'s Performance"
how deterministic is the emit really. if i feed same expression tree twice,same node layout same captures. do i get exact same bytes out every time (ignoring reloc) or not. if output produced is byte stable across runs for same input graph ,that opens up memoized JIT paths.worth checking if current impl already does this or needs a pass to normalise alloc order
Hotspot (TM) JIT compiles java code to machine code when it detects hot code paths, speeding up execution during runtime, exactly the use case described in the article.
Having written small compilers or other code-generators targeting both the JVM and .NET runtimes, i can say that the .NET equivalents have some extra simple options for scenarios like this.
Both have always supported building libraries/assemblies and loading them (the ASM library+custom classloaders for Java and AssemblyBuilder in .NET are about equally capable).
However .NET also has DynamicMethod that is specifically built to quickly build just small functions that aren't tied to larger contexts (similar API to asm/assemblybuilder).
But an even easier option for stuff exactly like in the article that people don't widely really seem to know about is that Linq (yes that "sql-like" stuff) actually contains parts for deferred method building that can be easily leveraged to quickly produce customized native code methods.
The neat part about the Linq-code generator is that you can just drop in plain C# code-blocks to be translataed by the C# compiler into Linq snippets and then with some helpers transform everything to Linq-tree's that can then be compiled.
The benefit over Asm/AssemblyBuilder/DynamicMethod is that Linq nodes are basically an built-in AST that can be directly leveraged whereas the other API's requires some mapping of your own AST's to the respective stack-machines.
This line is part of the code that creates an AST-like structure that is then fed into the compiler. The actual multiplication is done by calling the function handle returned from the Compile method.
They built this to translate a search query that is only known at runtime. Presumably they already have an AST or similar, so calling methods as it is being walked isn't any harder than operators.
this looks convenient to use from c++, but the example code it generates is rather suboptimal (see https://godbolt.org/z/3rWceeYoW in which no normal compiler would set up and tear down a stack frame for that) so i'm guessing there isn't any support for optimisations? what's the advantage of this over just compiling + calling dlopen/LoadLibrary on the result?
For simple functions an C compiler will generate code that is perhaps 50% faster than this standard prologue/epilogue (modern CPU's eat up most of the "bloat" whereas the branch to _any_ function will cause some branch predictor pressure), as soon as the function grows the gains will be smaller as long as the code runs somewhat in a straight line and isn't subject to cache misses.
Compared to even an optimized interpreter this will be somewhere between 4x to 20x faster (mainly due to having far far smaller branch predictor costs), so even if it doesn't generate optimal code it will still be within an magnitude of optimal native code whereas an interpreter will be much further behind.
dlopen/LoadLibrary,etc comes with far more memory pressure and OS bookkeeping.
FWIW for C only I've used libtcc repo.or.cz/w/tinycc.git with great success. The API is a joy, as we all expect from a Bellard project. It focuses on compilation speed, the generated code is not at all optimized.
It's interesting to see C++ expressions being used to create what is I think an AST that then gets compiled. I would love to see some syntactic sugar, though. For example, `expression.Mul(rsquared, expression.Immediate(PI))` could be `rsquared * expression.Immediate(PI)`. With overloading, anything that is not a recognized type could be converted to an immediate, so it could simply be `rsquared * PI`. Simple control structures could be even implemented with lambdas.
25 comments
[ 0.17 ms ] story [ 44.9 ms ] threadIn particular, this talk might be interesting:
"Unlocking the Power of C++ as a Service: Uniting Python's Usability with C++'s Performance"
Video: https://www.youtube.com/watch?v=rdfBnGjyFrc Slides: https://llvm.org/devmtg/2023-10/slides/techtalks/Vassilev-Un...
https://github.com/llvm/llvm-project/tree/main/clang/tools/c...
Why?
Both have always supported building libraries/assemblies and loading them (the ASM library+custom classloaders for Java and AssemblyBuilder in .NET are about equally capable).
However .NET also has DynamicMethod that is specifically built to quickly build just small functions that aren't tied to larger contexts (similar API to asm/assemblybuilder).
But an even easier option for stuff exactly like in the article that people don't widely really seem to know about is that Linq (yes that "sql-like" stuff) actually contains parts for deferred method building that can be easily leveraged to quickly produce customized native code methods.
The neat part about the Linq-code generator is that you can just drop in plain C# code-blocks to be translataed by the C# compiler into Linq snippets and then with some helpers transform everything to Linq-tree's that can then be compiled.
The benefit over Asm/AssemblyBuilder/DynamicMethod is that Linq nodes are basically an built-in AST that can be directly leveraged whereas the other API's requires some mapping of your own AST's to the respective stack-machines.
https://asm.ow2.io/
https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...
https://learn.microsoft.com/en-us/dotnet/api/system.reflecti...
https://learn.microsoft.com/en-us/dotnet/api/system.linq.exp...
This is C++, no? Why not use operator overloading for the project?
They built this to translate a search query that is only known at runtime. Presumably they already have an AST or similar, so calling methods as it is being walked isn't any harder than operators.
Compared to even an optimized interpreter this will be somewhere between 4x to 20x faster (mainly due to having far far smaller branch predictor costs), so even if it doesn't generate optimal code it will still be within an magnitude of optimal native code whereas an interpreter will be much further behind.
dlopen/LoadLibrary,etc comes with far more memory pressure and OS bookkeeping.
Bing uses internally a better version, but improvements are not merged back to github. See https://github.com/BitFunnel/NativeJIT/issues/84#issuecommen...
I did this for ruby-libjit, and it made writing a JIT compiler much easier to read. Here's an example: https://github.com/cout/ruby-libjit/blob/master/sample/fib.r...
And a real-world example (one of the very earliest JIT compilers for ruby, written in ruby, dates back to the ruby 1.8.x days): https://github.com/cout/ludicrous/blob/master/lib/ludicrous/...