I guess you want to say MPI and C++ are better for distributed computation?
Yes we do. We try to leverage Arrow as much as we can. And different from many other computing frameworks, Fugue doesn't invent new data types, it just uses Arrow data types.
Yes, we plan to integrate with Snowflake and Snowpark.
Hello whinvik, I totally agree the debugging experience of Spark is suboptimal. But why? I don't think the main reason is too much 'magic', I think it is because of the coupling of user's logic and the complicated…
Fugue SQL extends standard SQL so you can use SQL-ish language end to end. For example, let's assume you use duckdb or mysql or other standard sql for data processing, how do you deal with intermediate tables? You may…
No, we use antlr, we have no dependency on Java.
That is very true. Thank you. Fugue SQL is one way, and it also has functional API. They both can be translated into the underlying runtime. You can choose based your preference and real need.
Hey, I am the author of Fugue. Fugue is a higher level abstraction compared to Ray. It provides unified and non-invasive interfaces for people to use Spark, Dask and Pandas. Ray/Modin is also on our roadmap. It provides…
I guess you want to say MPI and C++ are better for distributed computation?
Yes we do. We try to leverage Arrow as much as we can. And different from many other computing frameworks, Fugue doesn't invent new data types, it just uses Arrow data types.
Yes, we plan to integrate with Snowflake and Snowpark.
Hello whinvik, I totally agree the debugging experience of Spark is suboptimal. But why? I don't think the main reason is too much 'magic', I think it is because of the coupling of user's logic and the complicated…
Fugue SQL extends standard SQL so you can use SQL-ish language end to end. For example, let's assume you use duckdb or mysql or other standard sql for data processing, how do you deal with intermediate tables? You may…
No, we use antlr, we have no dependency on Java.
That is very true. Thank you. Fugue SQL is one way, and it also has functional API. They both can be translated into the underlying runtime. You can choose based your preference and real need.
Hey, I am the author of Fugue. Fugue is a higher level abstraction compared to Ray. It provides unified and non-invasive interfaces for people to use Spark, Dask and Pandas. Ray/Modin is also on our roadmap. It provides…