Apple container is more akin to a replacement for docker or colima (although patterned more like Kata containers where each container is a separate vm as opposed to a bunch of containers in a single vm). It's a…
cool to see ctenv. definitely a similar vibe. thanks for sharing! will look at more closely. Interesting to see how you incorporated some dockerfile patterns. devcontainer feature-esque. I'm curious to know if you are…
I'm one of the creators of shai. Thanks for the callout! Interesting to see the work on Yolobox and in this space generally. The pattern we've seen as agent use grows is being thoughtful about what different agents get…
Arrow supports a union type for heterogeneous columns (we use it for random json in Dremio) and a 128-bit decimal.
PrestoDB is similar to Impala, Hive and other SQL Engines. Each is designed to do distributed SQL processing. Dremio does embed an OSS distributed SQL processing engine (Sabot, built natively on Arrow) as well but we…
Arrow is all about in-memory, not long-term persistence. Systems can write it to disk but it is the one in-memory representation to rule them all, not storage/disk. Disk has its own requirements and challenges outside…
For those level of operations, the Arrow library itself will probably have something fairly soon.
It's more than an implementation detail because we're also targeting interoperability between multiple separate technologies. One of the key things that the article didn't fully cover is that Arrow serves two purposes:…
Dremio is focused on a combination of data access, acceleration and a self-service analyst experience (Tableau/Qlik one-click integration, data curation and data management). We also invest heavily in our pushdowns. For…
People are already using Arrow for high performance GPU processing. FPGA is possible but I'm not sure if anyone is actually doing this today.
One quick note to make on this. Kudu is a storage implementation, (similar to Parquet in some ways). Arrow isn't about persistence and is actually built to be complementary to both Kudu and Parquet. Also note: Kudu is a…
The Arrow project itself is a set of libraries. One of the things we'll do is try to add more algorithms over time to it so if you want say, a fast arrow sort or arrow predicate application. Full SQL is always far more…
You nailed it. The most exciting part about all of this is being able to move between a "data science" context and a "database" context (and back again) without pain or penalty.
Possible, yes. Performant or pleasant? Maybe not :)
We built Dremio (github.com/dremio/dremio-oss, apache licensed) entirely on top of Apache Arrow specifically for the purposes of creating a high speed analytical capabilities including MOLAP like work as well as other…
I'm also a developer on Arrow (https://github.com/jacques-n/), similar to WesM. It is always rewarding (and also sometimes challenging) to hear how people understand or value something you're working on. I think Dan's…
Apple container is more akin to a replacement for docker or colima (although patterned more like Kata containers where each container is a separate vm as opposed to a bunch of containers in a single vm). It's a…
cool to see ctenv. definitely a similar vibe. thanks for sharing! will look at more closely. Interesting to see how you incorporated some dockerfile patterns. devcontainer feature-esque. I'm curious to know if you are…
I'm one of the creators of shai. Thanks for the callout! Interesting to see the work on Yolobox and in this space generally. The pattern we've seen as agent use grows is being thoughtful about what different agents get…
Arrow supports a union type for heterogeneous columns (we use it for random json in Dremio) and a 128-bit decimal.
PrestoDB is similar to Impala, Hive and other SQL Engines. Each is designed to do distributed SQL processing. Dremio does embed an OSS distributed SQL processing engine (Sabot, built natively on Arrow) as well but we…
Arrow is all about in-memory, not long-term persistence. Systems can write it to disk but it is the one in-memory representation to rule them all, not storage/disk. Disk has its own requirements and challenges outside…
For those level of operations, the Arrow library itself will probably have something fairly soon.
It's more than an implementation detail because we're also targeting interoperability between multiple separate technologies. One of the key things that the article didn't fully cover is that Arrow serves two purposes:…
Dremio is focused on a combination of data access, acceleration and a self-service analyst experience (Tableau/Qlik one-click integration, data curation and data management). We also invest heavily in our pushdowns. For…
People are already using Arrow for high performance GPU processing. FPGA is possible but I'm not sure if anyone is actually doing this today.
One quick note to make on this. Kudu is a storage implementation, (similar to Parquet in some ways). Arrow isn't about persistence and is actually built to be complementary to both Kudu and Parquet. Also note: Kudu is a…
The Arrow project itself is a set of libraries. One of the things we'll do is try to add more algorithms over time to it so if you want say, a fast arrow sort or arrow predicate application. Full SQL is always far more…
You nailed it. The most exciting part about all of this is being able to move between a "data science" context and a "database" context (and back again) without pain or penalty.
Possible, yes. Performant or pleasant? Maybe not :)
We built Dremio (github.com/dremio/dremio-oss, apache licensed) entirely on top of Apache Arrow specifically for the purposes of creating a high speed analytical capabilities including MOLAP like work as well as other…
I'm also a developer on Arrow (https://github.com/jacques-n/), similar to WesM. It is always rewarding (and also sometimes challenging) to hear how people understand or value something you're working on. I think Dan's…