Oh this is beautiful and I'm so glad it's been reposted because I missed it the first time.
Flamegraphs seem so much more interpretable and informative than the other plots there, at least to me personally. And I never would have thought to use them for this, because usually when I need to clean out disks or take care of storage it's time sensitive and I want to spend the minimum time figuring things out, and poor viz is enough to accomplish the goal.
An ongoing falmegraph of disk usage over time would be super helpful for many systems I'm working with right now.
Flamegraphs are a really lovely tool for visualizing trees. Slightly related anecdote:
A while ago I was experimenting with interactive exploration of (huge) Monte Carlo Tree Search trees. Inspired by file system visualization tools, my first attempts were also tree maps and sunburst graphs, but I ran into the same problems as in the article.
I tried flamegraphs next with the following setup:
- The number of visits in each node maps to the width and order of each bar (i.e., the most visited node was first and was the largest)
- The expected value maps to the color of each bar.
And then it was a perfect fit: it's easy to see what's going on in each branch at the first levels, and the deeper levels can be explored through drilling down.
All of these suck. Use nested bar graphs like TreeSize and it’s instantly obvious what your biggest hitter is for any particular nesting level.
In lieu of that, a flame graph is tolerable. The polar coordinate one is very pretty garbage. EDIT: Use it when you want to mislead people with a flashy graph.
For profiling I like the dual representation of treemap and tree of https://kcachegrind.github.io/html/Home.html a lot. Addresses the criticized points of treemaps of the post (see percentage and estimate areas of sub-trees) better than the examples chosen there.
Treemap is the densest/most accurate information source on a per px basis. Flamegraphs are pretty good but with a fixed Y and variable X your box area is inaccurate, and it wastes a fair amount of plot space with the non-flame area. The sunburst chart is really pretty but bad from an information communication perspective.
The treemap screenshot doesn't look correct. Nearly all charting libs (like Apache Echarts) will group nodes with a heading name, so not sure why they claim it would be hard to notice the "drivers" node. I guess in that screenshot, sure, but that looks like just a bad implementation of a treemap. Maybe this was the case back in 2017?
Flame graphs I have a love/hate relationship with. The hierarchy is very useful, but the name and coloring can be very confusing and misleading. Most people I show them to think red == something bad, but the color is actually just for aesthetics.
Yes, pretty much all treemap disk space tools I've used will perform color gradient grouping on boxes, with directories fitting in larger boxes. The box may not be drawn, but the inner boxes will align, visually making a larger box. Also, mouse hovers go a long way.
Like, one just has to look at the qdirstat screenshot at https://github.com/shundhammer/qdirstat. On the bottom-right corner, there are visually distinct boxes of sub-boxes that guide the eye towards a logical set of files.
Personally, I find treemaps unmatched for disk space analysis. Specifically, I like to use the squarify layout algorithm, to NOT use the "cushion gradient" shading method, to use inset frames to convey depth visually, and to include filenames. This maximizes glanceable information density, for the use case of identifying large objects to delete to recover space.
This is how the old spacemonger app worked, and I liked it so much I had to recreate it for Linux/Mac: https://github.com/alanbernstein/treemonger. My version still needs some work, but it's minimally useable.
Treemaps are indifferent to "unknown" or "unlabeled" nodes. Area is disk space.
Whereas the simple act of labelling a node adds another outer ring arc to the sunburst (thus more coloured area), even though the underlying truth hasn't changed.
The larger problem is that humans are kinda shit at eyeballing the relative size of areas. You can generally tell if one is larger than the other, most people just can't estimate how much larger one area is to another with any sort of accuracy (which is why area comparison graphs are often used when it's desirable to minimize the perceived differences).
Side note: To anyone that reaches out for du and ncdu from time to time. I recommend checking out `dua` (and `dua interactive`). It's way faster on my SSDs
23 comments
[ 3.3 ms ] story [ 48.7 ms ] threadFlamegraphs seem so much more interpretable and informative than the other plots there, at least to me personally. And I never would have thought to use them for this, because usually when I need to clean out disks or take care of storage it's time sensitive and I want to spend the minimum time figuring things out, and poor viz is enough to accomplish the goal.
An ongoing falmegraph of disk usage over time would be super helpful for many systems I'm working with right now.
https://diskanalyzer.com/download
https://www.uderzo.it/main_products/space_sniffer/
A while ago I was experimenting with interactive exploration of (huge) Monte Carlo Tree Search trees. Inspired by file system visualization tools, my first attempts were also tree maps and sunburst graphs, but I ran into the same problems as in the article.
I tried flamegraphs next with the following setup:
- The number of visits in each node maps to the width and order of each bar (i.e., the most visited node was first and was the largest)
- The expected value maps to the color of each bar.
And then it was a perfect fit: it's easy to see what's going on in each branch at the first levels, and the deeper levels can be explored through drilling down.
In lieu of that, a flame graph is tolerable. The polar coordinate one is very pretty garbage. EDIT: Use it when you want to mislead people with a flashy graph.
https://plotly.com/python/icicle-charts/
One could drill down e.g. Groceries > Drinks > Coca-Cola if one is so inclined...
Flame graphs I have a love/hate relationship with. The hierarchy is very useful, but the name and coloring can be very confusing and misleading. Most people I show them to think red == something bad, but the color is actually just for aesthetics.
Like, one just has to look at the qdirstat screenshot at https://github.com/shundhammer/qdirstat. On the bottom-right corner, there are visually distinct boxes of sub-boxes that guide the eye towards a logical set of files.
This is how the old spacemonger app worked, and I liked it so much I had to recreate it for Linux/Mac: https://github.com/alanbernstein/treemonger. My version still needs some work, but it's minimally useable.
E.g.
https://github.com/vasturiano/sunburst-chart
Treemaps are indifferent to "unknown" or "unlabeled" nodes. Area is disk space.
Whereas the simple act of labelling a node adds another outer ring arc to the sunburst (thus more coloured area), even though the underlying truth hasn't changed.