This has to be ruled out first: https://github.com/whatwg/fs/issues/7#issuecomment-116176851... ...but then the OPFS will be a quite decent fit. We (DuckDB-Wasm) are also looking closely at OPFS. IMHO the requirement…
We're still on a journey to explore what APIs work best with JavaScript but the differences between WASM and Node are on purpose. DuckDB-Wasm has isolated wasm heap memory and runs in a separate web worker. That's why…
The author outlines many problems that you'll run into when implementing a persistent storage backend using the current browser APIs. We faced many of them ourselves but paused any further work on an IndexedDB-backend…
I agree! DuckDB-Wasm can already open DuckDB database files in the browser the very same way.
It depends. Querying CSV files is particularly painful over the network since we still have to read everything for a full scan. With Parquet, you would at least only have to read the columns of group by keys and…
DuckDB-Wasm uses a traditional buffer manager and evicts pages using a combination of FIFO + LRU (to distinguish sequential scans from hot pages like the Parquet metadata).
Yes we do! DuckDB-Wasm can read files using HTTP range requests very similar to the sql.js-httpvfs from phiresky. The blog post contains a few examples how this can be used, for example, to partially query Parquet files…
DuckDB-wasm is targeting the browser so it's not directly competing with Pandas (that's the job of native DuckDB). It's targeting use cases where you want to push analytical computation away from servers into the client…
The WebAssembly module is 1.6 - 1.8 MB brotli-compressed depending on the Wasm feature set. We're currently investigating ways to reduce this to around 1 MB. We further use streaming instantiation which means that the…
This has to be ruled out first: https://github.com/whatwg/fs/issues/7#issuecomment-116176851... ...but then the OPFS will be a quite decent fit. We (DuckDB-Wasm) are also looking closely at OPFS. IMHO the requirement…
We're still on a journey to explore what APIs work best with JavaScript but the differences between WASM and Node are on purpose. DuckDB-Wasm has isolated wasm heap memory and runs in a separate web worker. That's why…
The author outlines many problems that you'll run into when implementing a persistent storage backend using the current browser APIs. We faced many of them ourselves but paused any further work on an IndexedDB-backend…
I agree! DuckDB-Wasm can already open DuckDB database files in the browser the very same way.
It depends. Querying CSV files is particularly painful over the network since we still have to read everything for a full scan. With Parquet, you would at least only have to read the columns of group by keys and…
DuckDB-Wasm uses a traditional buffer manager and evicts pages using a combination of FIFO + LRU (to distinguish sequential scans from hot pages like the Parquet metadata).
Yes we do! DuckDB-Wasm can read files using HTTP range requests very similar to the sql.js-httpvfs from phiresky. The blog post contains a few examples how this can be used, for example, to partially query Parquet files…
DuckDB-wasm is targeting the browser so it's not directly competing with Pandas (that's the job of native DuckDB). It's targeting use cases where you want to push analytical computation away from servers into the client…
The WebAssembly module is 1.6 - 1.8 MB brotli-compressed depending on the Wasm feature set. We're currently investigating ways to reduce this to around 1 MB. We further use streaming instantiation which means that the…