> Superset is fast, lightweight, intuitive, and loaded with options that make it easy for users of all skill sets to explore and visualize their data, from simple line charts to highly detailed geospatial charts.
I tried Superset a few years back, and maybe it's changed since then, but intuitive is about the last thing I'd use to describe it. Things which I could figure out in a few minutes on any other BI tool literally took me hours of searching. It didn't help that they decided to rename core concepts at some point so half the online documentation made no sense anymore. Others at those companies who tried it at the time said similar things.
I also found Superset unintuitive to use and setup as well. I settled on standing up Metabase because it was so simple to get started with trying it since it can be launched as a single jar. The business users loved it and so did I and administration with a Postgres backend instead of the internal h2 database was a breeze.
Metabase is great. It truly is a BI tool. Superset is more of a visualization platform, which works great if you have engineers building reports. Less good if you expect more junior analysts to be super productive.
We ran into the same exact issue with Superset not being intuitive, just for a different audience that is more technical. Also went with Metabase which is good, easy to use, lacks some a few chart types but overall the past year has seen quite a few changes and bug fixes consistently happening.
I had the same experience. Featurewise Superset looked better, but after wasting a couple of hours trying to install it, I just gave up.
Instead I installed Metabase in 5 minutes tops: spin ec2 instance, whether and java -jar . I've never looked back.
The only thing that turns me off I'd that it's implemented in an obscure language. At one time I wanted to add some custom postprocessing to an api (given an sql query, get some python/pandas postproc command from a sql comment and execute it in the returned table), but the used language is just not for me (some lisp dialect)
My experience with Superset was the opposite. It's easy to install using containers. You can have it up and running and connected to ClickHouse in a few minutes. I also found the internal design pretty intuitive--the SQL query lab is much easier than Grafana's editor.
I like Grafana too, but there's basically no isolation between your query and the SQL database at least in the Altinity Grafana plugin for ClickHouse which is the main one I use.
Metabase is more intuitive. Also, being unintuitive isn't great but not the worst thing. A project not even realizing that (and thinking the exact opposite) is much much worse. Unintuitive can be fixed with PRs over time. Delusional project leadership cannot.
They are doing pretty well that it is even clear what the project is really even about. Good luck figuring that out within 30 seconds of hitting the average Apache project homepage.
Had a similar experience with Superset. A few others have mentioned Metabase and I agree it's better, but if you're looking for a different approach to data, check out Definite (https://www.definite.app/). It's a "data stack in a box". A few things we're doing differently:
1. Built-in data warehouse - We spin up a duckdb database for you to load data to
2. 500+ connectors - You don't need to buy a separate ETL and you can pull in all your data (e.g. Postgres, Stripe, HubSpot, Zendesk, etc.) automatically
3. Semantic layer - Define dimensions, measures, and joins in one place. We have pre-built models for all the sources we support (e.g. the Stripe model already has measures for MRR, churn, etc.)
4. Simple BI - Build a table with the data you want and generate visuals off that table
The fundamental difference is that Grafana isn't great at cross referencing data in different data sources. (I love Grafana and I pay for the Cloud version.)
I found that running TrinoDB in a docker container and adding the trino plugin to grafana was very straightforward. TrinoDB feels magical sometimes, except that the SQL syntax they use seemed awkward IIRC. Also, there are inexplicable performance problems with certain queries that require trying subtlety different SQL queries until it snaps out of it.
grafana is built more for operational and timeseries data, but not so optimal for complex analytical queries. Ex: up-to-second data on cpu load on a host.
superset is the flip side of grafana; not good for up-to-second updates, but good for complex queries. Also, non-time series stuff. Ex: Which customer groups bought which products for all time? <— that type of BI stuff.
Why is that, though? I'd think that there'd be some plugins/extensions for Grafana that could do this. Grafana could then become the next PowerBI/Tableau/Superset killer eventually.
Different audience / use case. I've noticed that products often lean towards speaking to app builders (full stack swe's) or data builders (data analysts / scientists / data engineers). They require different mental models I feel.
Grafana I sense is culturally focused on observability visualization (aka needs of full stack devs). Culture is very hard to change!
They're both washboarding apps, and while I'm sure they each have panel types the other doesn't yet support, I don't think that's intrinsic. The differentiation as I see it, is that Superset is designed to craft SQL queries and visualize the results. The query builder is probably where this shows the most.
To make it more concrete -- coworkers tell me Grafana doesn't work so well with Apache Druid, while Superset supports it quite well.
Yes, I am at a company using Metabase, but I have a decent amount of experience with Superset (albeit from many years ago).
The reason we chose Metabase was that it had table joins, while Superset doesn't (unless it has added them since I used it). It also looks a bit sleeker. But I strongly prefer Superset; I found that with Metabase I had to turn a lot of things off to make it usable (Let me see "the_table" not "The Table"!), I was constantly annoyed at the opacity around models vs "questions", etc. and every time I wanted to change a question Metabase insisted on creating a new one instead. The real issue here was when we wanted to swap out the data source for a lot of questions but there was no clean way to do so without MB just creating new questions.
The "model" vs "question" thing is really annoying as there's no real difference from the user's perspective, and it's easy to accidentally convert a model back to a question without noticing when you publish something. You notice when you try to drill into the chart. There's a lot of annoying manual labor in metabase, e.g. I want to filter something into 10 different charts and I need to duplicate it 10 times and change a filter on each one. Still yeah joins are nice. A non-bugged aggregate count/sum as a window function would be nicer.
>I was constantly annoyed at the opacity around models vs "questions"
Yeah, somewhere along the line Metabase decided to get opinionated on "self-serve". I imagine it works well for some teams and companies, but for the tech-oriented, it's annoying.
I prefer my BI tools to be platforms that make for easy charting and cross-filters, while I build and control the models behind the scenes with a tool like dbt.
Superset lets you join tables within the same database. If you want to do cross-DB joins, we have a new (beta) in-memory meta-DB that lets you do this, but we generally see and recommend people using things like Trino for this.
Metabase is a bit more user-friendly to be honest than Superset. Superset has a WAY more liberal license, so it's ideal for people who want to customize Superset and build data apps.
Reposting from a comment of mine about 60 days ago:
I recently ran a little shootout between Superset, Metabase, and Lightdash — all open source with hosted options. All have nontrivial weaknesses but I ended up picking Lightdash.
Superset is the best of them at data visualization but I honestly found it almost useless for self-serve BI by business users if you have existing star schema. This issue on how to do joins in Superset (with stalebot making a mess XD) is everything difficult about Superset for BI in a nutshell. https://github.com/apache/superset/issues/8645
Metabase is pretty great and it's definitely the right choice for a startup looking to get low cost BI set up. It still has a very table centric view, but feels built for _BI_ rather than visualization alone.
Lightdash has significant warts (YAML, pivoting being done in the frontend, no symmetric aggregates) but the Looker inspiration is obvious and it makes it easy to present _groups of tables_ to business users ready to rock. I liked Looker before Google acquired it. My business users are comfortable with star and snowflake schemas (not that they know those words) and it was easy to drop Lightdash on top of our existing data warehouse.
> YAML, pivoting being done in the frontend, no symmetric aggregates
(one of the maintainers of Lightdash) You touched on some of our most interesting problems here! Would be especially interested to hear about what you liked / didn't like about symmetric aggregates in Looker and how you find dev with YAML. If you have an idea of how you'd like these to look in Lightdash, the team would be really open to making that a reality.
Oh that's awesome! Must be awesome to work on that. We've been using Airflow in production for 6 years at this point with various clients and it's been great, and we're trying to sell people on Superset now as well.
This comment is off-topic in this thread - it doesn't belong here and I don't see why you'd quote-tweet a different thread to me from months ago in an unrelated place. It is tantamount to stalking.
If you truly want me to answer, and you're only posting here because you can't reply to old threads, let me know and I'll send you my personal email and we can engage there.
Obviously you can't attack another user like this on HN, so please don't.
There's another problem: you've been posting about nothing but this one topic for months. That's not what HN is for. I know the topic is important and I'm sure your views are sincere, but we still have to preserve this site for its intended purpose. Using HN primarily for political battle and similar conflicts is against HN's rules, not because those conflicts don't matter (of course they matter, more than basically everything on HN), but because if we don't have such a rule, they will take over the site completely, and then HN won't be HN any more.
You're simply breaking HN's rules, as I tried to explain above. The rules have been the same for many years (https://news.ycombinator.com/newsguidelines.html) and have nothing specifically to do with this conflict. The topic itself isn't excluded from HN - there have been several massive threads about it, and nobody is being punished or censored here because they hold one view rather than another. We do, however, have to ban accounts that insist on using the site just to pursue a conflict about one thing, regardless of what the one thing happens to be. I don't want to ban you—that's why I posted my reply to you in the first place. But we do have to apply the rules even-handedly.
The lack of a semantic layer and join limitations are what made me pass on superset, but that was a couple years ago so maybe those features have been added.
I built my own semantic layer instead. I use this in production in my company but obviously use at your own risk as it's a one-man show.
This looks interesting for me, but I'd really like more detail about the architecture and deployment in the docs.
There is this:
> A final SQL query against the combined data from the DataSource Layer
> The Combined Layer is just another SQL database (in-memory SQLite by default) that is used to tie the datasource data together and apply a few additional features such as rollups, row filters, row limits, sorting, pivots, and technical computations.
But it leaves me with questions - how/when does this get populated? What other options are there besides in-memory SQLite? (I presume that's just a convenience for development and would use something else in production?)
Or is it just what Superset calls a 'metastore' i.e. data about the data, and the queries are run against the data source layer?
Or from a comment elsewhere in this thread about Superset:
> Superset lets you join tables within the same database. If you want to do cross-DB joins, we have a new (beta) in-memory meta-DB that lets you do this
It first runs one or more queries against your DataSources in a drill-across query fashion. You can think of DataSources as one or more completely separate databases. You could have one mysql, one postgresql, one duckdb etc all in the same Warehouse (not saying this is common in production, just an example). Within those DataSource queries it's also joining all needed tables together for you, i.e. joining multiple tables in each database to meet your required grain.
It then takes the results of all those queries and combines that data in another layer which is currently an in-memory sqlite database. The purpose of that layer is joining the data for presentation as well as applying some additional features like rollups, technicals, formula fields, etc.
I'm not familiar with what superset does under the hood or exposes as an API so I don't know how to compare it, if there is some similar backend piece. But I suspect no part of superset is quite the same as this, based on what its front end can do.
So it's irritating to me that this is ranking #1 on HN (why is it, btw?) I just pulled the trigger on a large data gathering project using Metabase, and feel a bit hampered by the limitations in terms of charts and plugins... but I considered Superset first, and after a lot of thought I decided that almost everything I've ever worked with that was run by the Apache foundation turned out to be semi-abandoned disasterware over time. In fact I wasn't even sure if Superset was still an active project or if it just looked like one, in the way e.g. no one bothered to pull the OpenOffice website offline.
So now that I picked Metabase, Superset is topping HN for no apparent reason. Why?
The problem really is that some Apache projects are actually alive (Apache itself, apparently Superset, Groovy, etc) and some appear alive at first glance.
More things should move into the Attic, like OpenOffice.
These are all projects that once were (more) relevant, however seem to have become rather niche (Gradle, Jetbrains/VSCode, GoogleDocs/Libreoffice e.g. for the first three are the dominant competitors).
Most of these projects (like the massive commons listings) are either used by some Java library somewhere (meaning their success/relevance is tied to the usage of Java), or are obscure enough that they are no longer used widely and so suffer from lack of interest.
There are gems in this list, to be sure, but if you just run into half-maintained projects all the time you're not likely to associate good things with the Apache name?
Well, OpenOffice as I said. Cordova is/was a hot mess (with some nice pioneering features, just really not well maintained imo and felt like quicksand to build even a small app on) Then the sort of long slow death of Flex (now Royale?) Apache seems like where software no one loves anymore goes to die.
I suppose it depends on projects you're using. For many developers their primary exposure to the Apache Foundation is through projects like Maven and Kafka, and those certainly don't feel dead.
Hmm. I suppose all open source looks that way if it doesn't get regular funding/attention.
Apache does house a lot of abandonware. They had some relevance as recently as 6-7 years ago but they've been largely replaced by nginx I think. That being said, I view them like the local soup-kitchen - important to have and maintain, but not where I want to go for a 5-star meal.
Yes, I agree. However a lot of their forward facing projects seem to be effective abandon-ware (few people interested in contributing, competing more popular solutions based on forks, or just no longer relevant).
These projects don't give the apache foundation an appearance of importance or relevance, rather they make it look rather rundown.
That's how open-source abandonware is supposed to work though: the idea is that whenever a (for-profit) company produces something that it can't afford to run anymore but also can't afford to shut-down and damage their customer relationships, then they'll open-source the project and give it to an open-source foundation for stewardship and repo hosting. Yes, it's where software goes-to-die-a-long-death, but it also gives some people hope, and the possibility of giving it a new life in future. Currently, the Apache Foundation is the go-to place for that, and it benefits everyone considering the alternatives are worse.
Obivously the main "alternative" is for the original company to simply shut down the product/service, which can do irreperable harm to a company when they have high-profile customers who are utterly dependent on a service.
Another alternative is to use an open-source foundation that's directly managed by the original company, which is what Microsoft did with its DotNet Foundation ( https://dotnetfoundation.org/ ) - and while Microsoft's legal team ensures the foundation is "legally" independent, in practice we know all the significant shots are being called from within Microsoft-proper; but it does give us some modest reassurances that .NET won't suddenly return to being closed-source overnight.
Another alternative is to not open-source it and to instead sell it off to another company that can maintain it while still being profitable - this is what Adobe did with Flash: they sold it all off to Samsung because their Harman division wanted to continue using Flash for embedded/automotive UX work. This approach can work, but doesn't benefit the wider ecosystem the way that open-sourcing does - and something something shareholder value and return-on-investment by selling rather than writing-it-off...
What companies won't do is let any of their engs that are passionate about a project split-off from the company to run and maintain it, le sigh.
The Apache Foundation also takes on projects that are literally abandoned. It acts as an umbrella that takes over hosting a project for commercial actors that can no longer develop it, but want to at least give existing users a open source (Apache License) version of the software to continue with/depend on.
Any time I hear "Apache Foundation" my stomach turns as I hesitate to ask my next question. "What we are trying to use from them is built on Java right"
> I think the HN algo is pretty easily manipulated. I worked at a startup that had an effective process to get things to the front page
That sounds (potentially) sleazy. If you think it's a technique that HN could potentially defend against, I encourage you to explain it to hn@ycombinator.com.
I don't think it's semi-abandoned. I had a brief interaction with the project in my previous job, and I found the community and the company to be reasonably engaged and responsive.
I have the opposite experience. Lots of good stuff is hosted by Apache Foundation, such as Kafka, Maven, Cassandra, Camel, the Tika project, Superset, Solr, but I will admit they had more relevance 10 years ago. And I dont think there are many organizations that keep open source projects alive longer than the Apache projects.
I used Metabase at my last gig (CTO @ e-commerce, 30+ users) and it was well-received and dare I say even a bit adored. It was the only self-hosted tool I'd receive after-hours text messages about going down that someone urgently needed back up for some task due tomorrow.
Business users loved the self-serve query builder, and it wasn't uncommon to walk around the office and see Metabase up on someones screen. My CEO absolutely loved it, and used it daily including to put together data for board decks.
None of my users cared about visualizations, and lived in tabular data. This included finance, marketing, merchandising, operations, and executives (CEO/COO/CFO). The only people that lamented the limited visualization were analysts. Power users did all their day-to-day work in Excel or other tools anyway, such as managing marketing spend or inventory allocations.
Metabase was great for dashboards and self-service (ad-hoc). 10/10 would deploy again.
Apache Software Foundation is just an umbrella organization to keep things on life support till someone can apply sufficient motive force to resurrect. I think that's really valuable. Lots of projects there have had that effort applied to them and kept going.
It's part of our Open Source Data Platform and it's one of the few open source BI tools out there and there are not a lot of alternatives in this space. We generally like it.
Superset allowed us to replace Tableau and not looking back
Took me a while figure out how to embed it into my app using Superset Embedded SDK.
Superset Embedded SDK - "Embedded SDK allows you to embed dashboards from Superset into your own app, using your app's authentication. Embedding is done by inserting an iframe, containing a Superset page, into the host application."
Superset source code is very easy to read and understand, and as a result it's possible to implement some advanced caching techniques reduce the load on charts.
No BI is perfect.
Watching Superset for years gives me confidence the project will work as supposed down the road, and eventually some of its packages can be reusable for all kind of visualizations and data hacking.
Our main approach to visualisation is to start with eChart and simple Reactjs wrapping and spin off Superset on subdomain for power users, and later see which one works better. Same look gives a very pleasant experience.
I dont want to start a rant against Tableau. It's a powerhouse. It's a great superior software. But when it comes to optimizing cost and comparing the total cost of ownership and opportunity to stop paying for Tableau server license we voted in favor of Superset and mix of Reactjs+Echarts widgets.
Honestly it's so hard to compare Tableau and Superset. Tableau has every feature and bell / whistle imagine-able. But it's heavy, desktop oriented, and pricey.
Superset is lightweight and open source, but only has 5% of the features. So it really depends what you need!
Bokeh is an option in the frontend-viz space that puts out pretty solid SVG for statically-rendered charts, while also having the option of more Tableau-like interactive functionality with input fields, dynamic filters, etc. Might be a decent option for you?
Their interactive "embedded-mode" avoids iframes too... but it's built with web components, so you wind up in shadow-DOM hell if you want to do anything dynamic on the view's contents.
We use ECharts in our open source BI tool (Evidence) and it's a great library. Has helped us build a declarative syntax for viz which can be version controlled (https://evidence.dev)
We're fans of Obsidian! DataView looks cool - love the ability to define the tables in code inline in the markdown. That's similar to how we inline DuckDB WASM SQL queries in markdown: https://docs.evidence.dev/core-concepts/queries/
Evidence looks cool, and I evaluated sometime back. The docs says the pages are all pre-rendered for all possible combinations. Is that the case still? If so, if I have a date filter, is it going to pre-render all possible dates?
We recently changed our architecture to include interactivity without having to pre-render all combinations. Pages are still pre-rendered with their initial content, but each Evidence app now ships with filter components and an in-browser DuckDB instance so you can build interactive apps. We call this Universal SQL - if you're interested, we wrote up our rationale for doing this here: https://evidence.dev/blog/why-we-built-usql/
How do you deal with data visibility and permissions? I mean, most tables have data that should only be seen by a specific user or group ID, and that layer is usually handled by the application. It would be awesome to expose the power of Superset for users, but I imagine creating the security layer would be a pain.
Had good results with echarts. With Superset not so much: complicated to install, lost all dashboards after an update, cryptic error messages, custom queries meh: we decided to use views in Postgres. The project with Superset was finished successfully, but the time spend is a multiple compared to using something like Power BI.
All in all, not very innovative, but highly needed open source version of a traditional BI tool. Definitely something to follow and to use in temporary, not too demanding use cases. And hopefully a future replacement of Tableau or Power BI.
Redash development slowed down for sure, but it's not looking abandoned. It's just that I've been using it for some time now, I'm wondering if is anything feature-wise that could justify the switch.
Generally what you get when VentureCapital/PrivateEquity buys out Redash.io, messes up end users in the process and spits it out a few years later, leaving users confused as to where it stands in the BI tools landscape.
I wish more projects had guided tour videos that demonstrated the power of the tool in the hands of an expert user. Not "get started" but "why should I care".
Wes McKinney used to have an excellent 5 minute introduction to pandas in this genre.
You can check this out. This is a Preset Demo, but shows quite a bit of Superset within Preset (which offers multiple instances of Superset as "Workspaces") https://www.youtube.com/watch?v=V0HwGnC1rU8
There is a major release on the horizon (4.0) and there were just a couple of patch releases for the 3.x variants. I'm surprised to see it trending too, but I'm happy about it. More people need to know that Open Source BI is here, and here to win.
Superset is powerful, but I wonder why they don't fix "papercuts", e.g., misaligned pixels on a spinner, or inability to copy a value from a table's cell, or non-monospace font for numbers in a table, etc. There are hundreds of small annoyances in the product.
We try! We also accept PRs and Issues if there are things bugging people, of course. It's always a balancing act between building some new feature that people are clamoring for, or fixing those cosmetic issues that always crop up.
Should it? If you really need that, join the different sources with TrinoDB (or any related managed service like AWS Athena) and connect it to Superset.
We use https://cube.dev/ as intermediate layer between data warehouse database and Superset (and other "terminal" apps for BI like report generators). You define your schema (metrics, dimensions, joins, calculated metrics etc) in cube and then access them by any tool that can connect to SQL db
One thing to keep in mind with BI software is that the users are often very different than, well, those individuals that prefer to use mutt as an email client.
Many, or most, users for a BI tool will be operations, product managers, and business management who simply will not find the interface to be intuitive, responsive, or well designed. At least that's my experience.
Wow, those Apache guys have so many projects. Of course, they've been at it for years, starting with the Apache web server, then Tomcat, etc., and also, many projects were first developed outside and then handed over to them, for whatever reasons.
How would you compare Superset with PowerBI for analytics and CSS integration? Trying to develop features and advanced analytics capabilities into an app?
If you want to style the whole application, you can fork the repo and go bananas. If you're looking for theming, there's more to be done yet on that front, and I wrote an article on that too: https://preset.io/blog/theming-superset-progress-update/
Is Superset a decent tool if you're just a single person doing data analysis? Say I have a handful of sqlite databases, and just want to be able to develop some queries / charts. I was looking into Tableau / Power BI / Superset, and all of them seemed pretty heavyweight for a single user, and none of them seemed super easy to get setup locally.
Any recommendations for a good piece of software for the single user case? Or a more convenient way to run the heavyweight tools?
I'll say PowerBI has the potential to be more powerful, but you need to love the whole M and DAX languages eco system. And the integration with python and R it's not that bad.
But if your vis are with the scope of native Tableau capabilities, then Tableau it's more friendly and gets less in the way of you and your work.
If you are doing data analysis I don't think any of the 3 pieces of software you mentioned are going to be that helpful.
I see these products as tools for data visualization and reporting i.e. presenting prepared datasets to users in a visually appealing way. They aren't as well suited for serious analytics.
I can't comment on Superset or Tableau but I am familiar with Power BI (it has been rolled out across my org), the type of statistics you can do with it are fairly rudimentary. If you need to do any thing beyond summarizing (counts, averages, min, max etc). It is not particularly easy.
For data analysis I use SAS or R. This software allows you do things like multivariate regression, timeseries forecasting, PCA, Cluster analysis etc. There is also plotting capability.
Both these products are kind of old school, I've been using them since early 2000's, the "new school" seems to be Python. Pretty much all the recent data science people in my organization use Python. Particularly Pandas and libraries like Seaborn (https://seaborn.pydata.org/).
The "power" users of Power BI in my organization tend to be finance/HR people for use cases like drill down into cost figures or Interactively presenting KPI's and other headline figures to management things like that.
For my last employer, I set up Superset for a number of our clients to show all sorts of heavily customized marketing analytics dashboards, web performance graphs, project management burndown reports, you name it. As with another commenter's experience, we also got a client to replace Tableau with it, and not look back. Such a great product.
In my experience, people with a business related background have an easier time learning how to use BI tools (this is true even if Superset may be less user-friendly than other commercial product like Tableau); Jupyter is an interactive computing platform that is based on notebooks and cells, that's more useful for data scientists/engineers whose needs might exceed the capabilities of a SQL interface.
199 comments
[ 0.25 ms ] story [ 190 ms ] threadI tried Superset a few years back, and maybe it's changed since then, but intuitive is about the last thing I'd use to describe it. Things which I could figure out in a few minutes on any other BI tool literally took me hours of searching. It didn't help that they decided to rename core concepts at some point so half the online documentation made no sense anymore. Others at those companies who tried it at the time said similar things.
https://www.metabase.com/demo
Demo is nice.
Instead I installed Metabase in 5 minutes tops: spin ec2 instance, whether and java -jar . I've never looked back.
The only thing that turns me off I'd that it's implemented in an obscure language. At one time I wanted to add some custom postprocessing to an api (given an sql query, get some python/pandas postproc command from a sql comment and execute it in the returned table), but the used language is just not for me (some lisp dialect)
I like Grafana too, but there's basically no isolation between your query and the SQL database at least in the Altinity Grafana plugin for ClickHouse which is the main one I use.
And the documentation is sparse at best.
1. https://www.apache.org/foundation/how-it-works/#incubator
What it was though, was riddled with dozens of Python runtime errors and innumerable glitches.
Metabase is where it’s at.
1. Built-in data warehouse - We spin up a duckdb database for you to load data to
2. 500+ connectors - You don't need to buy a separate ETL and you can pull in all your data (e.g. Postgres, Stripe, HubSpot, Zendesk, etc.) automatically
3. Semantic layer - Define dimensions, measures, and joins in one place. We have pre-built models for all the sources we support (e.g. the Stripe model already has measures for MRR, churn, etc.)
4. Simple BI - Build a table with the data you want and generate visuals off that table
I'm mike@definite.app if you have any questions.
As such, closer to an open source replacement for PowerBI.
superset is the flip side of grafana; not good for up-to-second updates, but good for complex queries. Also, non-time series stuff. Ex: Which customer groups bought which products for all time? <— that type of BI stuff.
Grafana I sense is culturally focused on observability visualization (aka needs of full stack devs). Culture is very hard to change!
To make it more concrete -- coworkers tell me Grafana doesn't work so well with Apache Druid, while Superset supports it quite well.
Is it worth it for BI on small datasets?
The reason we chose Metabase was that it had table joins, while Superset doesn't (unless it has added them since I used it). It also looks a bit sleeker. But I strongly prefer Superset; I found that with Metabase I had to turn a lot of things off to make it usable (Let me see "the_table" not "The Table"!), I was constantly annoyed at the opacity around models vs "questions", etc. and every time I wanted to change a question Metabase insisted on creating a new one instead. The real issue here was when we wanted to swap out the data source for a lot of questions but there was no clean way to do so without MB just creating new questions.
Also, Metabase doesn't have serialization unless you pay them AND you self-host, (if I'm self hosting then what exactly am I paying for?) and that's pretty annoying. https://www.metabase.com/docs/latest/installation-and-operat....
But it does let you join tables. Sometimes that's enough to make MB worth dealing with.
Yeah, somewhere along the line Metabase decided to get opinionated on "self-serve". I imagine it works well for some teams and companies, but for the tech-oriented, it's annoying.
I prefer my BI tools to be platforms that make for easy charting and cross-filters, while I build and control the models behind the scenes with a tool like dbt.
I've found the weird "make it easy" mindset a bit annoying with Metabase too. The whole questions, nice table names...
I'll give Superset a try in my next project I think.
I recently ran a little shootout between Superset, Metabase, and Lightdash — all open source with hosted options. All have nontrivial weaknesses but I ended up picking Lightdash. Superset is the best of them at data visualization but I honestly found it almost useless for self-serve BI by business users if you have existing star schema. This issue on how to do joins in Superset (with stalebot making a mess XD) is everything difficult about Superset for BI in a nutshell. https://github.com/apache/superset/issues/8645
Metabase is pretty great and it's definitely the right choice for a startup looking to get low cost BI set up. It still has a very table centric view, but feels built for _BI_ rather than visualization alone.
Lightdash has significant warts (YAML, pivoting being done in the frontend, no symmetric aggregates) but the Looker inspiration is obvious and it makes it easy to present _groups of tables_ to business users ready to rock. I liked Looker before Google acquired it. My business users are comfortable with star and snowflake schemas (not that they know those words) and it was easy to drop Lightdash on top of our existing data warehouse.
(one of the maintainers of Lightdash) You touched on some of our most interesting problems here! Would be especially interested to hear about what you liked / didn't like about symmetric aggregates in Looker and how you find dev with YAML. If you have an idea of how you'd like these to look in Lightdash, the team would be really open to making that a reality.
For pivoting in the backend, this is coming! Issue here: https://github.com/lightdash/lightdash/issues/2907
I've been running it in production since 2017, at two jobs, the current one a big corporation.
Best general-purpose, database-backed dashboarding system out there. I would never pay for Tableau or PowerBI.
Same for Airflow.
If you truly want me to answer, and you're only posting here because you can't reply to old threads, let me know and I'll send you my personal email and we can engage there.
There's another problem: you've been posting about nothing but this one topic for months. That's not what HN is for. I know the topic is important and I'm sure your views are sincere, but we still have to preserve this site for its intended purpose. Using HN primarily for political battle and similar conflicts is against HN's rules, not because those conflicts don't matter (of course they matter, more than basically everything on HN), but because if we don't have such a rule, they will take over the site completely, and then HN won't be HN any more.
If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
Open source Business intelligence platform made with Python - https://news.ycombinator.com/item?id=29368664 - Nov 2021 (49 comments)
Apache Superset 1.1 - https://news.ycombinator.com/item?id=27439939 - June 2021 (28 comments)
The Apache Software Foundation Announces Apache Superset as a Top-Level Project - https://news.ycombinator.com/item?id=25905277 - Jan 2021 (1 comment)
Apache Superset is an enterprise-ready business intelligence web application - https://news.ycombinator.com/item?id=21133931 - Oct 2019 (7 comments)
I wrote about Superset's semantic layer here: https://preset.io/blog/understanding-superset-semantic-layer...
One popular option is to use dbt or Cube for the semantic layer and pair with Superset: https://preset.io/blog/announcing-presets-ui-integration-wit... and https://preset.io/blog/open-source-looker-cube-superset/
I built my own semantic layer instead. I use this in production in my company but obviously use at your own risk as it's a one-man show.
https://github.com/totalhack/zillion
There is this:
> A final SQL query against the combined data from the DataSource Layer
> The Combined Layer is just another SQL database (in-memory SQLite by default) that is used to tie the datasource data together and apply a few additional features such as rollups, row filters, row limits, sorting, pivots, and technical computations.
But it leaves me with questions - how/when does this get populated? What other options are there besides in-memory SQLite? (I presume that's just a convenience for development and would use something else in production?)
Or is it just what Superset calls a 'metastore' i.e. data about the data, and the queries are run against the data source layer?
> Superset lets you join tables within the same database. If you want to do cross-DB joins, we have a new (beta) in-memory meta-DB that lets you do this
...is it this?
It first runs one or more queries against your DataSources in a drill-across query fashion. You can think of DataSources as one or more completely separate databases. You could have one mysql, one postgresql, one duckdb etc all in the same Warehouse (not saying this is common in production, just an example). Within those DataSource queries it's also joining all needed tables together for you, i.e. joining multiple tables in each database to meet your required grain.
It then takes the results of all those queries and combines that data in another layer which is currently an in-memory sqlite database. The purpose of that layer is joining the data for presentation as well as applying some additional features like rollups, technicals, formula fields, etc.
I'm not familiar with what superset does under the hood or exposes as an API so I don't know how to compare it, if there is some similar backend piece. But I suspect no part of superset is quite the same as this, based on what its front end can do.
Happy to answer any other questions you have.
So now that I picked Metabase, Superset is topping HN for no apparent reason. Why?
Amen brother.
Can you name a few examples?
OpenOffice is probably the most famous (it still has the name, but it is dead, LibreOffice is the real "active" fork).
And the things in the "Attic" are officially dead - https://projects.apache.org/committee.html?attic and many more projects should be there.
Lots of other projects just die silently and/or you are unsure of the status.
Here you at least have a chance to revive them if you like as there is always an overarching organisation.
More things should move into the Attic, like OpenOffice.
https://projects.apache.org/projects.html?name
These are all projects that once were (more) relevant, however seem to have become rather niche (Gradle, Jetbrains/VSCode, GoogleDocs/Libreoffice e.g. for the first three are the dominant competitors).
Most of these projects (like the massive commons listings) are either used by some Java library somewhere (meaning their success/relevance is tied to the usage of Java), or are obscure enough that they are no longer used widely and so suffer from lack of interest.
There are gems in this list, to be sure, but if you just run into half-maintained projects all the time you're not likely to associate good things with the Apache name?
Hmm. I suppose all open source looks that way if it doesn't get regular funding/attention.
Apache does house a lot of abandonware. They had some relevance as recently as 6-7 years ago but they've been largely replaced by nginx I think. That being said, I view them like the local soup-kitchen - important to have and maintain, but not where I want to go for a 5-star meal.
These projects don't give the apache foundation an appearance of importance or relevance, rather they make it look rather rundown.
Obivously the main "alternative" is for the original company to simply shut down the product/service, which can do irreperable harm to a company when they have high-profile customers who are utterly dependent on a service.
Another alternative is to use an open-source foundation that's directly managed by the original company, which is what Microsoft did with its DotNet Foundation ( https://dotnetfoundation.org/ ) - and while Microsoft's legal team ensures the foundation is "legally" independent, in practice we know all the significant shots are being called from within Microsoft-proper; but it does give us some modest reassurances that .NET won't suddenly return to being closed-source overnight.
Another alternative is to not open-source it and to instead sell it off to another company that can maintain it while still being profitable - this is what Adobe did with Flash: they sold it all off to Samsung because their Harman division wanted to continue using Flash for embedded/automotive UX work. This approach can work, but doesn't benefit the wider ecosystem the way that open-sourcing does - and something something shareholder value and return-on-investment by selling rather than writing-it-off...
What companies won't do is let any of their engs that are passionate about a project split-off from the company to run and maintain it, le sigh.
I think the HN algo is pretty easily manipulated. I worked at a startup that had an effective process to get things to the front page
That sounds (potentially) sleazy. If you think it's a technique that HN could potentially defend against, I encourage you to explain it to hn@ycombinator.com.
Pretty sure it's as simple as posting in your general slack channel "@here we posted a new article to HN, go upvote and write a comment"
https://preset.io/
I don't think it's semi-abandoned. I had a brief interaction with the project in my previous job, and I found the community and the company to be reasonably engaged and responsive.
Business users loved the self-serve query builder, and it wasn't uncommon to walk around the office and see Metabase up on someones screen. My CEO absolutely loved it, and used it daily including to put together data for board decks.
None of my users cared about visualizations, and lived in tabular data. This included finance, marketing, merchandising, operations, and executives (CEO/COO/CFO). The only people that lamented the limited visualization were analysts. Power users did all their day-to-day work in Excel or other tools anyway, such as managing marketing spend or inventory allocations.
Metabase was great for dashboards and self-service (ad-hoc). 10/10 would deploy again.
It's part of our Open Source Data Platform and it's one of the few open source BI tools out there and there are not a lot of alternatives in this space. We generally like it.
Superset allowed us to replace Tableau and not looking back
Took me a while figure out how to embed it into my app using Superset Embedded SDK.
Superset Embedded SDK - "Embedded SDK allows you to embed dashboards from Superset into your own app, using your app's authentication. Embedding is done by inserting an iframe, containing a Superset page, into the host application."
https://github.com/apache/superset/tree/master/superset-embe...
Superset is based on very high quality and well maintained chart library eChart
https://echarts.apache.org/examples/en/#chart-type-linesG
Community Roadmap
https://github.com/apache/superset/projects?query=is%3Aopen
Huge respect to Preset.io and its team for contributing to the project and keep it in a great shape
https://preset.io/blog/
Superset source code is very easy to read and understand, and as a result it's possible to implement some advanced caching techniques reduce the load on charts.
No BI is perfect.
Watching Superset for years gives me confidence the project will work as supposed down the road, and eventually some of its packages can be reusable for all kind of visualizations and data hacking.
Our main approach to visualisation is to start with eChart and simple Reactjs wrapping and spin off Superset on subdomain for power users, and later see which one works better. Same look gives a very pleasant experience.
https://www.tableau.com/products/server
If you have money, dedicated team of data analytics who are already familiar with Tableau - no need to torture them with other tools.
Superset is lightweight and open source, but only has 5% of the features. So it really depends what you need!
Their interactive "embedded-mode" avoids iframes too... but it's built with web components, so you wind up in shadow-DOM hell if you want to do anything dynamic on the view's contents.
Previous HN discussion: https://news.ycombinator.com/item?id=35645464 (97 comments)
Reminds me Obsidian DataView but with charts https://github.com/blacksmithgu/obsidian-dataview
This whole ideas to have data, visualisations and knowledge base in one private offline place is very appealing
The Markdown <-> Markup typing experience is just so good compared to e.g. Slack, Reddit and other markdown-esque tools
Here's an example project with some filter components and custom styling: https://ecommerce.evidence.app/
This is still a static app - the data warehouse was only hit during the app's build process
All in all, not very innovative, but highly needed open source version of a traditional BI tool. Definitely something to follow and to use in temporary, not too demanding use cases. And hopefully a future replacement of Tableau or Power BI.
Wes McKinney used to have an excellent 5 minute introduction to pandas in this genre.
I think Athena can only query data on S3?
Many, or most, users for a BI tool will be operations, product managers, and business management who simply will not find the interface to be intuitive, responsive, or well designed. At least that's my experience.
In what way, any details?
Not been tracking that or using OpenOffice for a while.
If you want to style the whole application, you can fork the repo and go bananas. If you're looking for theming, there's more to be done yet on that front, and I wrote an article on that too: https://preset.io/blog/theming-superset-progress-update/
Any recommendations for a good piece of software for the single user case? Or a more convenient way to run the heavyweight tools?
This is a single user application, unless you make it part of your built application.
K8s installation instructions: https://superset.apache.org/docs/installation/running-on-kub...
RBAC configuration: https://superset.apache.org/docs/security/#rest-api-for-user...
But if your vis are with the scope of native Tableau capabilities, then Tableau it's more friendly and gets less in the way of you and your work.
I see these products as tools for data visualization and reporting i.e. presenting prepared datasets to users in a visually appealing way. They aren't as well suited for serious analytics.
I can't comment on Superset or Tableau but I am familiar with Power BI (it has been rolled out across my org), the type of statistics you can do with it are fairly rudimentary. If you need to do any thing beyond summarizing (counts, averages, min, max etc). It is not particularly easy.
For data analysis I use SAS or R. This software allows you do things like multivariate regression, timeseries forecasting, PCA, Cluster analysis etc. There is also plotting capability.
Both these products are kind of old school, I've been using them since early 2000's, the "new school" seems to be Python. Pretty much all the recent data science people in my organization use Python. Particularly Pandas and libraries like Seaborn (https://seaborn.pydata.org/).
The "power" users of Power BI in my organization tend to be finance/HR people for use cases like drill down into cost figures or Interactively presenting KPI's and other headline figures to management things like that.