Ask HN: Modern OLAP Cube?

5 points by mrits 11y ago ↗ HN

What are startups using as alternatives to Oracle and Microsoft for olap data?

The only decent open source project I can find is Mondrian on top of Postgresql.

I'm investigating cubes because I believe it fits my current problem. We have a star schema that we currently just run tons of different reports on. We take advantage of Postgresql materialized views and abuse the crap out of functions.

One of the larger problems we have is dealing with non-additive metrics. E.g, we might want to see how many distinct ip addresses are in a time frame. And since "distinct" is not additive we currently keep large sets around in the rollups to make them additive. I've investigated HyperLogLog data structures to solve this problem but I couldn't get product management on board with probabilistic data structures.

I'm looking forward to hearing what other people are experimenting with.

3 comments

[ 3.0 ms ] story [ 29.0 ms ] thread

akbar501 11y ago ↗

Apache Kylin, created by eBay, is a modern cubes on HBase solution. However, the overhead of HBase is likely not worth it unless you have a data size problem.

http://www.kylin.io/

skrowl 11y ago ↗

I'm not sure what language you're targeting, but you might look at http://cubes.databrewery.org/

greggyb 11y ago ↗

Unfortunately for you, my work causes me to focus on the Microsoft stack, so I can't give any specific product recommendations.

That being said, technology-wise I can suggest a columnstore database to address some of your needs, particularly the need to use the distinct operator. The storage format of a columnstore lends itself to aggregation and distinct operations, with multiple orders of magnitude improvements possible compared to SQL.

Additionally, traditional OLAP does not handle distinct operations all that wonderfully to begin with, so it likely wouldn't help you too much for that option.

Further, you've not provided much detail about your dimensional model or your use case. It's worth exploring different fact table structures. The Kimball Data Warehouse books are always worth reading.