Show HN: Spall – WASM-based profiler for JavaScript and C and Odin (gravitymoth.com)
[I'm the author] Spall is a web-accessible profiler that I made to help my web-dev friends load gigabyte+ JSON traces without lunch-break-long load times. Recently, Spall got experimental support for auto-tracing with binary traces (along with an in-progress native-port, to give it more memory headroom), which was used to help track down and fix some hard-to-spot lock contention issues in the Odin-language compiler.
I demoed it at the Handmade Seattle conference in October, https://guide.handmade-seattle.com/c/2022/spall/, with a head-to-head against Perfetto, another big web profiler.
I'll be around to answer any questions. Thanks for looking at my project! If you like Spall, you can catch my other projects over at https://colrdavidson.github.io/.
If you happen to be hiring I'd love to hear from you. Looking for something fun and new to do after a long sabbatical, working on cool open-source projects!
21 comments
[ 2.8 ms ] story [ 33.3 ms ] threadYou mention in the "For Everyone Else" section that you have an auto-tracer system to instrument all your tracing. Aside from being very handy, how does it compare to [1] which you mention right after?
[1] https://github.com/bvisness/dtrace2spall
Auto-tracing is far more comprehensive, but it's a big datadump, and it does impact runtimes a bit more.
So far I've mostly used auto-tracing to figure out how bigger pieces of code work, rather than strictly for profiling. It was super helpful trying to walk through the more complicated parts of the Odin compiler to fix things, being able to see the order of functions called on the timeline.
In the section of the interview where you talk about the difficulties you had, you discuss the challenges of making Odin work properly on WASM, and how you needed custom memory-management and allocation strategies.
Could you talk some more about that experience and the general process of working with that sort of low-level bit in Odin? I've been experimenting with using Odin to write a DB from scratch.
Outside of the WASM context where Odin runs into some difficult friction, systems-level work has been reasonable. I wrote a good chunk of a C/DWARF debugger in both C and Odin to compare, and Odin was by far the more pleasant language to write it in, my Odin code was much smaller because I didn't need xmacros, and I had proper namespaced enums, tagged unions, and slices, but still looked/read pretty similarly to well-written C or Go. I did have to wrap a few syscalls myself though, because the core library doesn't quite cover things like ptrace or networking yet.
I've put a fair bit of work into getting cross-platform networking into Odin's core library, hopefully we'll see it show up soon enough, and get some of those common nice-to-haves crossed off.
https://odin-lang.org/docs/overview/
It's a systems programming language that syntactically feels similar to Go and Jonathan Blow's unreleased language, Jai.
It is meant to be a C replacement (or, on the same level as C), similar to Zig.
[0] https://vimeo.com/776796857
For everyone doing new things, there's a binary format which is pretty straightforward.
The binary side supports begin and end events, which are defined here: https://github.com/colrdavidson/spall/blob/master/formats/sp...
and the header library mirrors those structs, and chalks out the interfaces around here: https://github.com/colrdavidson/spall/blob/master/spall.h#L2...
Basically, it's a small header, and then events, plopped one after another into the file. Each thread gets a buffer to store events as they come in, and they get written as atomic chunks with fwrite/append, so threads never have to interact directly.
In practice, the overhead for tracepoints with the binary format is super low, depending on the machine on x86, we've seen somewhere between 12-20ns for begin+end trace overhead when compiled with clang (+ occasional disk flush overhead, if you're generating a ton of data). I believe I can get that overhead even smaller to get things nice and lightweight, it's a major goal of mine to be able to trace 10+ million events per second without impacting runtimes significantly, so that bigger projects can be comfortably fully auto-traced.