HN Explorer

Show HN: Turn your Pandas dataframe into a Tableau-style UI for visual analysis (github.com)

712 points by AwsmDef ↗ HN
Hey, guys. I've just made a plugin which turns your pandas dataframe into a tableau-style component. It allows you to explore the dataframe with easy drag-and-drop UI.

You can use PyGWalker in Jupyter, Google Colab, or even Kaggle Notebook to easily explore your data and generate interactive visualizations.

PyGWalker (pronounced like "Pig Walker", just for fun) is named as an abbreviation of "Python binding of Graphic Walker".

Here are some links to check it out:

The Github Repo: https://github.com/Kanaries/pygwalker

Use PyGWalker in Kaggle: https://www.kaggle.com/asmdef/pygwalker-test

Feedback and suggestions are appreciated! Please feel free to try it out and let me know what you think. Thanks for your support!

62 comments

[ 3.5 ms ] story [ 180 ms ] thread
maneesh
Wow! this is super cool!
faizshah
I love this, it seems like the heavy lifting is done by the web app here: https://github.com/Kanaries/graphic-walker

I’m amazed that this is open source, it’s incredibly useful.

I wish there was a profiler implementation, the best profiler is in GCP’s DataPrep.

anigbrowl
Looks good! was impressed with your web app, having hte same functionality in a Jupyter notebook is a huge plus.
AwsmDef
Thanks a lot for your suggestion. It'll be our next goal.
cjbprime
Nice! Do you think it's useful for working with non-numeric tabular data too? I'm using a dataframe in vscode as a kind of mini SQL database. Vscode visualizations of dataframe are pretty bad, would be great to have another option for rendering.
loa_observer
Yes, it's a general visualization tool. But it depends on what you need for your data. Dimensions in non-numeric tabular data can be used with a 'count' to make charts of the distribution of values in the dimensions.
aarondia
This is very cool. I'm the creator of Mito [1] -- we're also building a data visualization tool in JupyterLab. The Tabluea approach that you took is really interesting! Going to send you a message -- would love to learn more!

[1] https://www.trymito.io

thirdmunky
This is incredible! I've wanted something like this in a notebook for a long time.
chrsig
oh my, thank you! This could definitely fill a very noticeable gap in my tools.

I'm looking forward to playing with it this week

scrollbar
Mega impressed, I can see myself using this regularly. Charting and viz with matplotlib and pandas transforms is great, but a lot of time there’s a benefit of dumping to point-and-click mode with tableau to quickly spin data around without much coding. This will make that workflow much smoother
boredemployee
Yep. And also there are so many alternatives of dataviz that I struggle to decide which one to use. You learn a few syntax, then come a newer and more beautiful visualization lib and you have to learn all basic syntax again.
tyfon
This goes in my toolbox, thanks!
buremba
This is pretty cool! It uses Vega transforms under the hood, right? The default backend uses JS for transformation which is slow for large datasets. Did you consider using Duckdb under the hood via WASM? I run into this project but not sure how active it is: https://github.com/vega/vega-plus
loa_observer
There is a WIP PR for combining DuckDB with Graphic Walker: https://github.com/Kanaries/graphic-walker/pull/18

WASM of Graphic Walker is coming soon！

buremba
Cool! How does it work in practice? Looking at the PR, it seems to pull all the data from Duckdb but is there any plan to apply the transformations (filter, aggregate that the user selected in UI) in Duckdb while rendering the chart?
loa_observer
This PR had just tested running DuckDB-WASM. For more complex computation, I am planning to build a computation engine to generate SQL and push it to DuckDB or other databases. (I built a POC like this in another project, RATH, but the SQL is push to clickhouse instead.)

If you are interested in this, welcome to discuss with us in the PRs/issues on Github or in our Discord: https://discord.gg/Z4ngFWXz2U

short_sells_poo
This looks incredible. Well done! Can we give money to you for this?
tyfon
Yes, I can easily convince my boss this is worth done some money too...
irony123
Yes please send the money to rest in my account and I'll be sure to forward it to the authors
BigElephant
Is it possible to generate the matplotlib (or whatever charting library) is used to make the visuals?
frgtpsswrdlame
Wow this looks absolutely awesome. I'll need to play around with it a bit but this would be huge for me.
TrackerFF
Is there any support or implantation for writeback functions? We use tableau at work for lots of reporting, and it just pissed me off how expensive many functionalities are if you need to purchase from third party vendors.
dgudkov
What do you mean by writeback functions?
lazyasciiart
Edit the content of the viz and have it save back to the datasource.
razor_router
It sounds like you are frustrated with the cost that comes with using third party vendors for additional functionalities in Tableau. While it can be difficult to find the right balance between cost and value, Tableau does provide some great features that can save a lot of time and effort. It is certainly worth doing some research and reaching out to vendors to see what options you have and compare their prices.
[deleted]
(comment deleted)
gota
Thanks, chatGPT, great insight
323
I thing DfWalker would be a better name.
damey
This was quite slow and never worked for 100K rows 20 columns. Any work on improving performance ?
selfawareMammal
Hmm Im trying to open a few csv's but none of them seem to work. The ui opens but I have no filters or ways to display anything, I just can see the name of the columns/fields from my data. Am I doing something wrong?
RandomWorker
This is the best thing since sliced bread! As someone that uses both Jupyter notebooks in lab(and loves to make vis) and uses tableau to prototype this is epic. It works trying it out of the box.

However, the most important feature I love about tableau and why I’m not dropping it is: the data import and sharing dashboard section. I’m sure this is something that could be interesting to investigate.

The data section where you can link data columns (and filter across all datasets after linking) and do pivot work in an intuitive way(see melt in pandas).

The data dashboards are great to share with my clients the output of my analysis work. I’d love it move to a bokeh style, but customizing those dashboards is not for the faint of heart, although you get an opensource very robust product at the end, and you don’t need to pay license fees going forward. Clients that want to keep their analysis for the long term can go for this option.

jredwards
Yeah, I'm going to use this a lot.

Like, a LOT.

Thank you.

stuaxo
I noticed this and the other library it uses assume you know what Tableau is - I don't, but these widgets look useful anyway.

It might be better if they were more up front about what they do, while still acknowledging Tableau.

zyl1n
This is really cool, but I wonder if it will get into legal trouble because it looks very much like Tableau.
fourthark
They have a boatload of patents, too.
dsjoerg
Very cool!

Anyone know how this compares to Apache Superset? https://superset.apache.org/

unixhero
Superset doesn't run in a Jupyter Notebook. So they are used differently. This seems very easily embeddable for instance. Or otherwise really practical.
swyx
any idea why it doesnt? im uninformed of the architectural decisions behind superset but this seems like a huge wasted opportunity
unixhero
No I don't know the technical reasons why it is not feasible to get it running in a Jupyter Notebook.

But my best assumption is that it isn't a Python library. You cannot import Apache Superset from a Python program.

albert_e
Can we "embed" these visualizations into a web application -- say a React app?

Any guidance on how one would go about that would be appreciated. Thanks!

loa_observer
If you want to embed the visualization part only, you can export a vega-lite/vega specification and then use Vega-Embed: https://github.com/vega/vega-embed or React-Vega to embed in your web app.

At the graphic walker toolbar, active debug mode, there will appear a button on the top-right corner of the chart. Click it, and then you can export the chart spec.

Or Embed the entire Graphic Walker as a react component: https://github.com/Kanaries/graphic-walker

__mharrison__
Cool! I just created a course covering visualization with Pandas, Seaborn, Excel, Tableau, and a few more apps. Would be interested to see how easy it would be to recreate some of the visualizations with this.