I don't have an opinion, but this article gives a fairly unbiased comparison of DuckDB vs Clickhouse-local, which I imagine exhibits similar performance characteristics as ChDB, just without the embedded part: https://www.vantage.sh/blog/clickhouse-local-vs-duckdb
I suppose if you had data in a format that DuckDb doesn't work with, like Protobuf, Avro, ORC, Arrow, etc. ClickHouse reads and writes data in over 70 formats
One reason would be if you're already fluent in ClickHouse's SQL dialect. Although they maintain great standard SQL compatibility, they also have a great deal of special functions/aggregates/etc that are ClickHouse specific.
Other reasons include their wide range of input formats, special table functions (e.g. query a URL).
I think it makes sense if you are just considering whether or not to use ClickHouse, it's a very easy place to start. If you then outgrow embedded you won't have to move to another database afterwards, you can probably just attach the tables to a "real" ClickHouse instance and continue using it without having to do lengthy data migration.
Disclaimer: I am a chdb maintainer! duckdb is currently thinner and has lots of active contributors and mature integrations, while chdb is still in its early stages BUT if you already love ClickHouse (like we do) chdb is a great choice as it inherits all the ClickHouse stability, performance and more importantly, all the 70+ supported formats for the embedded use case without any of the server/client requirements, making it perfect for fast in-process and serverless OLAP executions.
Note chdb is based on ClickHouse codebase but completely community powered so there's no feud with DuckDB (I'm a quackhead, too!) which actually offers lots of great inspiration and many integration opportunities with ClickHouse/chdb for combined compute and processing of datasets. I personally love both and use them together all the time in my colab "OLAPps"
I absolutely love this trend of "pip install x" as a distribution mechanism for this kind of thing.
People badmouth Python packaging a lot, but once you've learned how to use a virtual environment (admittedly a steeper learning curve than I'd like it to be) having binary wheels for so much of this kind of stuff is a huge win.
What packaging systems are there that don't encourage different environments for different projects?
I think Python's biggest weakness on this front is mainly that virtual environments came along too late - if they'd been designed into how pip etc worked from the beginning we would be in much better shape.
Lots of programming languages or build systems will look for dependencies in globally-known locations.
Other than cargo, python's virtualenv, node_modules, and some other modern languages/systems I would imagine most are not project-scoped. Go had a single global dependency tree until go.mod, etc...
A good language-specific package manager should know how to resolve its dependencies and maybe install them locally, but most importantly provide this information to the system package manager (apt, rpm, etc.) to handle.
Having "pip install x" is a poor man's ad-hoc system package manager.
Something like https://github.com/finos/perspective ? We use an OLAP(-y) WASM engine to provide query-ability to our data visualization tool, and doing the calculations in the browser is cheaper and simpler than a server-side database for datasets that fit in browser memory.
ChDB is the in-process version for Python. You can try clickhouse-local if you want a CLI experience, or clickhouse-client against a clickhouse-server for the server experience
34 comments
[ 2.6 ms ] story [ 81.6 ms ] threadOther reasons include their wide range of input formats, special table functions (e.g. query a URL).
Disclaimer: I work at ClickHouse
Note chdb is based on ClickHouse codebase but completely community powered so there's no feud with DuckDB (I'm a quackhead, too!) which actually offers lots of great inspiration and many integration opportunities with ClickHouse/chdb for combined compute and processing of datasets. I personally love both and use them together all the time in my colab "OLAPps"
It and the ability to do `from 'https://example.com/.csv` is why i love duckdb
SELECT * FROM url('https://example.com/*.csv')
We recently worked with the chDB team to add support for Jupyter notebooks: https://jupysql.ploomber.io/en/latest/integrations/chdb.html
People badmouth Python packaging a lot, but once you've learned how to use a virtual environment (admittedly a steeper learning curve than I'd like it to be) having binary wheels for so much of this kind of stuff is a huge win.
See also Playwright, DuckDB, ziglang... I wrote a bit about those here https://simonwillison.net/2022/May/23/bundling-binary-tools-...
And the fact that it's not easy to use doesn't really make things better.
I think Python's biggest weakness on this front is mainly that virtual environments came along too late - if they'd been designed into how pip etc worked from the beginning we would be in much better shape.
Lots of programming languages or build systems will look for dependencies in globally-known locations.
Other than cargo, python's virtualenv, node_modules, and some other modern languages/systems I would imagine most are not project-scoped. Go had a single global dependency tree until go.mod, etc...
https://hpc.guix.info/blog/2021/09/whats-in-a-package/
A good language-specific package manager should know how to resolve its dependencies and maybe install them locally, but most importantly provide this information to the system package manager (apt, rpm, etc.) to handle.
Having "pip install x" is a poor man's ad-hoc system package manager.
https://clickhouse.com/docs/en/operations/utilities/clickhou... https://clickhouse.com/docs/en/getting-started/quick-start