mwc360
No user record in our sample, but mwc360 has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but mwc360 has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
OSS Delta support deletion vectors. The problem is that the OSS Deltalake (based on Delta-rs) python library does not and this prevents engines like DuckDB and Polars from writing to suck tables. I'm pretty sure DV is…
Very cool! This seems like fantastic functionality and would make it super easy to migrate small Spark workloads to DuckDB :)
Ritchie - thx for graciously correcting some things I got wrong, will get it corrected!
FYI I had V-Order and Optimzed Write disabled in the benchmark. The only wrote diff was that I enabled deletion vectors in Spark since it’s supported which the other two don’t.
Author here: that’s exactly what I was trying to communicate but you said it better :)
Miles Cole here: I’d love to see Daft on Ray become more widely used. Same Dataframe API and run it in either single or multi-machine mode. The only thing I don’t love about it today is that their marketing is a bit…
Miles Cole here… thx for the correction, another reader just notes this as well. I’ll get this corrected tomorrow and possibly retest after verifying I have spill set up. Thx!
Hi - Miles Cole here… I used lazy APIs where available. I.e. everything up to write_delta() is lazy in the Polars (Mod) variant. Yeah I was debating whether to share all of the source code. I may share a portion of it…