The new Zig I/O idea seems like a pretty ingenious idea, if you write mostly applications and don't need stackless coroutines. I suspect that writing libraries using this style will be quite error-prone, because library…
I also don't know all the details, but the Rust parser tokens contain horrible crimes, primarily because of macros. All I wanted to say was that applying DoD to the parser in Rust would (IMO) be much more difficult than…
I don't think we hit a bedrock. As I wrote, we have a lot of ideas for massive improvements. But we need more people to work on them.
I'm not sure how that works. You either let the compiler compile your whole program with AVX (which duplicates the binary) or you manually use AVX with runtime detection on selected places (which requires writing manual…
You have a fair point, I agree that while compiler performance is a priority, is is one of many priorities, and not currently super high on the list for many Rust Project developers. I wish it was different, but the…
For many use-cases yes, but there are crates bottlenecked on different things than the codegen backend. But I don't think that's the point. We could get rid of LLVM and use other backends, same as we could do other…
I think a reasonable comparison would have to be DoD Rust parser vs current Rust parser. Comparing across languages isn't very useful, because Zig has very different syntax rules, and doesn't provide diagnostics near…
Great post! I have some random guesses as to why the 40% vs 60-70% memory issues percentage: - 180k is not that much code. The 60-70% number comes from Google and Microsoft, and they are dealing with way larger…
That didn't even occur to me, tbh :) But it doesn't have to be SQL linting, I just wanted to appreciate the mindset of not being lazy/afraid to write an unorthodox test.
If I only insert data into the DB once, I could miss important states. Like, I could add non-NULL data to a NOT NULL column, then make it NULL, and then make it NOT NULL again. If I don't insert NULL into the column…
If you're writing applications or tests, it's mostly the simpler kind of code. If you're writing reusable (or perf. critical code), you will start seeing generics and lifetimes much more often.
(author of the post) Just to clarify a bit, the test ofc isn't a fully general solution to solving issues with database migrations (I hinted what that might be in the blog post), although it's still useful to provide a…
I don't really see an attack surface for a dev dependency.
Yeah by user space I just meant without root, sorry. HQ runs on supercomputers where the environment is heavily locked up, even Docker doesn't work. I think that PID namespaces aren't really possible, but I haven't…
Could be done, yeah, but 20s isn't that much, and I'd like to avoid adding more test-only magic environment variables zo configure this (our end-to-end tests are in Python and they use HQ as a binary).
It is sadly not propagated to grandchildren. I tries the subreaper approach, but it doesn't help. The children are reparented to the worker, but when the worker dies, they are then just reparented to init, like normally.
I do use setsid when spawning the children (I omitted it from the post, but I set it in the dsme pre_exec call where I configure DEATHSIG) but they don't receive any signal, IIRC. Or if they do, it does not seem to be…
The stdlib already mostly does all of that :) Check out https://kobzol.github.io/rust/2024/01/28/process-spawning-pe....
That's a very good point! But yeah, we use the single threaded runtime, so this shouldn't be a concern.
The Rust bhild config defaults are pretty fine for the general case. It's just that not everyone has the general case :) In Rust the normal distribution will be quite flattened.
Yeah, I actually generated these small charts out of a flamegraph, because it contains too much information and isn't easily split into three distinct parts. And once you condense the information into just 3 blocks,…
There have been some recent improvements to this, but yeah, it can be still quite large. There is a WIP development of a garbage collector in Cargo that could help with this.
It didn't really require establishing a working group :) That is just an effort to improve Rust binary sizes in general. In the end, changes in Rust can take a very long time (years) sometimes. It's maintained and…
There is some work underway to enable removing the backtrace generation/parsing from Rust binaries. It's hardcoded for now though.
You can't really use dead code elimination on debug symbols, because you don't know which symbols will you need. You would need to know where and how will your program crash or if the user will want to use a debugger on…
The new Zig I/O idea seems like a pretty ingenious idea, if you write mostly applications and don't need stackless coroutines. I suspect that writing libraries using this style will be quite error-prone, because library…
I also don't know all the details, but the Rust parser tokens contain horrible crimes, primarily because of macros. All I wanted to say was that applying DoD to the parser in Rust would (IMO) be much more difficult than…
I don't think we hit a bedrock. As I wrote, we have a lot of ideas for massive improvements. But we need more people to work on them.
I'm not sure how that works. You either let the compiler compile your whole program with AVX (which duplicates the binary) or you manually use AVX with runtime detection on selected places (which requires writing manual…
You have a fair point, I agree that while compiler performance is a priority, is is one of many priorities, and not currently super high on the list for many Rust Project developers. I wish it was different, but the…
For many use-cases yes, but there are crates bottlenecked on different things than the codegen backend. But I don't think that's the point. We could get rid of LLVM and use other backends, same as we could do other…
I think a reasonable comparison would have to be DoD Rust parser vs current Rust parser. Comparing across languages isn't very useful, because Zig has very different syntax rules, and doesn't provide diagnostics near…
Great post! I have some random guesses as to why the 40% vs 60-70% memory issues percentage: - 180k is not that much code. The 60-70% number comes from Google and Microsoft, and they are dealing with way larger…
That didn't even occur to me, tbh :) But it doesn't have to be SQL linting, I just wanted to appreciate the mindset of not being lazy/afraid to write an unorthodox test.
If I only insert data into the DB once, I could miss important states. Like, I could add non-NULL data to a NOT NULL column, then make it NULL, and then make it NOT NULL again. If I don't insert NULL into the column…
If you're writing applications or tests, it's mostly the simpler kind of code. If you're writing reusable (or perf. critical code), you will start seeing generics and lifetimes much more often.
(author of the post) Just to clarify a bit, the test ofc isn't a fully general solution to solving issues with database migrations (I hinted what that might be in the blog post), although it's still useful to provide a…
I don't really see an attack surface for a dev dependency.
Yeah by user space I just meant without root, sorry. HQ runs on supercomputers where the environment is heavily locked up, even Docker doesn't work. I think that PID namespaces aren't really possible, but I haven't…
Could be done, yeah, but 20s isn't that much, and I'd like to avoid adding more test-only magic environment variables zo configure this (our end-to-end tests are in Python and they use HQ as a binary).
It is sadly not propagated to grandchildren. I tries the subreaper approach, but it doesn't help. The children are reparented to the worker, but when the worker dies, they are then just reparented to init, like normally.
I do use setsid when spawning the children (I omitted it from the post, but I set it in the dsme pre_exec call where I configure DEATHSIG) but they don't receive any signal, IIRC. Or if they do, it does not seem to be…
The stdlib already mostly does all of that :) Check out https://kobzol.github.io/rust/2024/01/28/process-spawning-pe....
That's a very good point! But yeah, we use the single threaded runtime, so this shouldn't be a concern.
The Rust bhild config defaults are pretty fine for the general case. It's just that not everyone has the general case :) In Rust the normal distribution will be quite flattened.
Yeah, I actually generated these small charts out of a flamegraph, because it contains too much information and isn't easily split into three distinct parts. And once you condense the information into just 3 blocks,…
There have been some recent improvements to this, but yeah, it can be still quite large. There is a WIP development of a garbage collector in Cargo that could help with this.
It didn't really require establishing a working group :) That is just an effort to improve Rust binary sizes in general. In the end, changes in Rust can take a very long time (years) sometimes. It's maintained and…
There is some work underway to enable removing the backtrace generation/parsing from Rust binaries. It's hardcoded for now though.
You can't really use dead code elimination on debug symbols, because you don't know which symbols will you need. You would need to know where and how will your program crash or if the user will want to use a debugger on…