C3 looked like a dead project about a year or so ago, which I thought was a real shame and planned to move away from it. Just checked the repo a few minutes ago and it looks like things are pretty lively these days. Glad to see it's kicking along again.
We've been using this. It's really nice and straight-forward, although it has some edge cases when you are doing small graphs and responsive layouts. Using it with Angular also adds some complexity.
Ehh, that was kind of a thin article that didn’t quite scratch the itch on why D3 is troublesome.
First, I think if you want to use d3, you probably just want a charting library. Convince yourself that HighCharts is not flexible enough and then try d3. So the trouble with d3 is that most people don’t need to use it and are in over their heads.
Second, d3 is full of leaky abstractions. Imagine if cars didn’t have acceleration pedals, and you had to manually open the throttle and engage the fuel pump yourself. You’d need an engineering degree to drive your car! Same thing with d3; it’s ridiculous that you need to know the ins and outs of some of the internet’s most complicated and frustrating technology just to show a couple of data points on a graph. It’d be great if you could work with d3 and pretend like SVG and the DOM didn’t exist.
There's a place for cookie cutter visualizations. However, you're never going to make anything distinctive or memorable that way. All of it is a trade off... Are you trying to fit in or stand out?
I think one of the overarching factors is that those new to data visualization (and web development) see D3's beautiful examples and assume that the D3 library is directly and fundamentally responsible for the visual quality. I've seen this a bit with R's ggplot2 as well among relatively experienced data scientists. To be sure, ggplot2 is a fantastic library and the fact that its defaults look great are a testament to the library's well-thought out design. But such design grace can be achieved in any library (e.g. matplotlib), albeit with less graceful APIs.
You can absolutely get there. But the trouble with most libraries I've used is that they are either limiting or confusing. For example, Seaborn is very easy to learn, but it's mostly just a set of canned visualizations, with limited flexibility. Matplotlib is very flexible, but just. so. hard. to. learn. I had a machine learning class in grad school where most everyone agreed the single most difficult homework was one that asked us to use Matplotlib to generate a few visualizations of what the models were doing.
Comparatively speaking, learning ggplot2 is a sublime experience. After about a week, you feel like you have super powers.
D3 does feel overcomplicated for just building static visualizations, but if you're building custom interactive stuff, learning it can be nearly as joyous an experience.
Regarding ggplot, matplotlib and seaborn I found the opposite. If you want some pre-canned visualisation then ggplot2 is fantastic, but as soon as you stray away from the examples it turns into a mess, you can't just drop down into base graphics but have to mess about with grob and grid.
Whereas seaborn has similar pre-canned plots, but it's all built on matplotlib so you still have that flexibility if you want it.
Yeah. If you come to d3 but you've never made a beautiful image in SVG before you're going to have a bad time. You have to be good at SVG, CSS and javascript already. D3 will not help.
Dunno for me D3 feels like they are trying to do everything at once resulting in an overly complex and hard-to-learn library.
When I look at the examples at their page I see stuff that I could make in 2 ways:
- Just write a small program from scratch that visualizes one in a week or 2. It will be small, I will know the ins and outs, so easy to change and will be a ton of fun to write.
- Use D3.js (I dont know this library, but i've used complex charting libraries in the past). Likely it will take more time, not because I am thinking of the interesting parts, I will spend most of my day cursing on StackOverFlow to figure out how to skin that one tricky part of my app.
So in the end I dont really see a good reason to use d3.js unless my full time job is to develop data visualizations.
I learned (some of) D3 in exactly the fashion the article suggested: find something I want to visualize and make it happen. I thought it was great.
I still wouldn't say I've "learned D3", but I didn't have to. I got my goal accomplished in a couple of days, along with a lot of nice bits I didn't intend along the way as I found good things that D3 could do for me.
For me it was definitely a "design to tools" approach; I looked at the gallery, found something mostly like what I wanted, and then worked to that. That was fine by me; I wasn't looking for ultimate visual control, but just trying to turn a bunch of data into something mortals could comprehend.
What's even more confusing is that v3 -> v4 includes a bunch of breaking changes, and yet they didn't update their typescript definition file. Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib (and versions are never mentioned anywhere).
> Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib
Unfortunately this is widespread across the JS ecosystem. Hell I'm not saying anyone should do it like Linux (break userspace and get shot), but when one does backwards-incompatible changes, is it so hard to stub dead methods or unsupported calling ways with "console.error('Outdated, see https://github.com/x/y/issues/z');"?
Side note (on D3.js) - there is a nice exploration of reusable D3.js charts/components: https://d3-discovery.net/ ("Finding D3 plugins with ease").
It's source (Vue-based, potentially for other showcases/searches): https://github.com/wbkd/d3-discovery (I just asked if it can be open source, and to my awe its author added MIT license :)).
D3 is for high performance custom data visalizations.
For example, I built a in-door navigation app. It would receive its data via MQTT/WebSockets and update markers on custom maps.
D3 was perfect for this, it helped with synchronizing graphics with data, converting mapping cooridnates to DOM/SVG coordinates and easing all the updates so stuff looked smooth.
You just would pump the incoming data into the D3 pipeline and be done with it.
If you just need a few animations, there are a bunch of simple libraries for this. React has react-motion for example, it's not as flexible as D3 but you get most stuff you would expect out of it without much hassle.
If you just need a set of basic charts, use a charting library. Highcharts or Charts.js are both very capable.
My work has been pretty interested in "interactive" charts, being able to modify axis scales on the fly, change filter range, dynamically add and remove series that sort of thing. We use SAS (and a little bit of R internally) and those both have good charting capabilities but trying to do anything interactive in either of those packages is a real pain. In the past we have used a C++ library SL-GMS.
Even when compared with using C++ I have found using JavaScript based libraries difficult D3 in particular I found to be horrendously complex.
I do quite a bit of woodworking and use a lot of handtools, yet a lot of the tools on that wall look completely alien to me. I think this wall might actually belong to an upholsterer or something like that...
Edit: The title of the stock photo is "Timmermansgereedschap" (NL) = Zimmermannsgerätschaften (DE) = carpenter's tools (EN).
The most salient point that the article breaches is this: When people unfamiliar with the DOM, javascript, or indeed in some cases programming itself try to 'pick up' a complex low level data visualization library like D3, it is natural for them to blame the library. But the problem obviously runs much deeper than that.
Running with the analogy in the article -- it is fruitless for someone who's never done any woodworking or furniture design ever to hope to become the next Charles Eames within the span of a few months.
However, d3 seems particularly bizarre. Its the only library i ever remember trying hard to learn, and just giving up. There is too little indication, when you misuse it, of what went wrong.
Ive done a bunch of javascript, DOM and visualisations in OpenGL, so i feel i should have a good chance.
Complexity comes at a price. Making it easy, abstracts certain low level functionality away. Compare Rust to Python. Compare OpenLayers to Leaflet. Compare the dashboard of an Apache to your car's dashboard. Rust, OpenLayers en Apaches aren't bizarre, they have complex functionality which you don't want to abstract away, because you want the ability to use it on that low level.
A lot of D3's oddness comes from the fact that there isn't anything (to my knowledge) that is comparable. It does just one thing - binding data to DOM elements - does it well, and then gets out of the way.
D3 has libraries that do many other things, and TBH I feel like selection is the most "aged" part of D3. IMO modern component libraries such as React, Vue or Mithril do a much better job of maintaining a binding between data and the DOM tree. But many parts of D3 don't particularly care about how you eventually render the result.
Something that saved me was the extensive amount of documentation-via-Stack Overflow, particularly the number of answers provided by D3's author, Mike Bostock--often with links to working code to illustrate his answer! The amount of effort he's put in to helping people understand how to use D3 is almost as impressive as D3 itself.
You absolutely need to be fluent in SVG (or canvas?). SVG is your output, so without that it's like trying to print to the terminal without knowing ASCII. If you don't know SVG you might think d3 knows about graphics. It doesn't.
> The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics.
Visualisation is fundamentally the process of creating a mapping function between the dimensions of your domain data and that of the "screen" (whether it's DOM, SVG, Canvas, etc.). D3 is just a library of higher-order functions that can help you produce that mapping function.
If you can grasp this idea of visualisation as creating a mapping function, then D3 becomes a beautiful toolbox for assisting you. You decide which dimension of your data you want to represent as the horizontal dimension, which as the vertical, as color, as size, etc. You decide how you want to scale them. Then you use D3's high-order functions to create the appropriate individual mapping functions, compose them into your complete visualisation function, and pass it your domain data.
If you think of D3 as a "charting library", or a collection of visualisation classes or objects, then you're going to struggle. But the problem isn't with D3, it's that you haven't really understood the paradigm of the library.
Anecdotal, but my problem with D3 is the need for so many function calls. 'append', 'attr', 'enter', 'datum', ..... Code ends up looking like a centipede ().().().()
I worked on a project that required some pretty standard line charts, but with the addition of an overlaid set of horizontal "benchmark" lines -- and some callouts. After struggling (and failing) to get what we wanted out of two different off-the-shelf graphing libraries (one of which was built on top of D3), we finally bit the bullet and just built the charts we wanted in D3 "from scratch". Once I wrapped my head around the model, it was such a pleasant experience. D3 is soooo nice and well thought out. You mayneed to level up in SVG, but it is so freeing to be able to bind data and display it basically any way you want. In hindsight, I wish we started with D3 from the beginning and I would not hesitate to pick it up for even simple charts again. It really is worth climbing the learning curve, which isn't as big and scary as it appears at first sight.
SVG is tied closely to CSS, so you can go in your CSS and create a style for your D3 graphs and do things like:
.chartline { stroke-width: 3px; color: red; }
D3 has some convenience functions in JS for creating SVG, but you don't have to use them, you could write SVG just like HTML, it will look more like this in the DOM:
Not OP, but a someone who uses D3 every day, it gives you (almost) total freedom. You can decide where to put your x and y axes, legends, labels, etc. You can decide how data determines not just how something appears on the screen (like a bar chart of y height) but how the underlying data determines its classes and styling, etc.
D3 is awe-inspiring in its scope, and learning to use it has a steep learning curve. But for anything beyond trivial use cases the freedom it affords you is worth the effort.
Go to https://beta.observablehq.com and dive in: just start forking other people’s examples and fiddling and looking up stuff in the docs or asking for help whenever you get stuck.
do you have any recommendation on resources on d3 for beginners? Personally I find d3.js very challenging, there is tons of example out there and simple tutorials, but I don't think there are tutorials that bridge those harder examples for beginners.
I must be missing something, this doesn't answer the question at all. Literally everything you said is true for just building the graph in JS/SVG. What does D3 give on top of that?
D3 particularly excels at graph animations and data transforms (as might be helpful to facilitate graphing). Its one of the most comprehensive graphing libraries around. Well written code that uses it looks like pure elegance. Its amazing how much functionality you can get out of so little code. It does take some time and a solid understanding of javascript and scope.
If you do it yourself in straight JS, you'll very quickly start building the common tools of D3: various scaling mechanisms mapping the domain to the pixels, data binding against a set of DOM elements, converting an array of data values to the co-ordinates for the d attribute of an SVG path, etc.
Somewhere else someone described D3 as a hammer, and that's a good analogy, because if you don't start with a hammer, it's the first tool you'll make for yourself when you need to pound nails.
It's an API for building custom charts. It's a huge step up from having to fully 'roll your own'. If you have to do any kind of vis that isn't available in the various off the shelf libraries, you can put it together with D3 easier than you could if you fully DIY a solution. It's well thought out but you have to use it for that to be more obvious. This book (by the D3 author) got me started, it's worth the hassle of learning [0].
Maybe this will help, it creates an SVG graphic by default.
Consider how much JS you'd have to write to even start creating all the frameworks to build in native SVG structures... a lot.
I created html graphs with tables in the late 90s, made Flash graphs in 2000s, upgrades to numerous JS based charting libraries after this, building custom graphs the whole way.
I had looked at D3 so many times over the years when it first showed up online. But it "looked" complicated, no obvious "chart" library, etc...
Finally, when I tried D3, I was like "why the f didn't I start with this?"
Maybe I'm missing something, but it's literally impossible to achieve something with D3 that you can't achieve with JS+SVG, since D3 is a JS library that emits SVG. The question is how much work you can afford to do. You can always roll your own. But you're going to have to a lot of fussing around with things like figuring out where to put tick marks or grid lines, maybe you need to parse some CSV, or figure out how to label dates. There are also things like Voronoi diagrams which are, if not tricky, at least tedious and annoying to implement yourself. And then there's simple statistical operations like making histograms or calculating medians or other quantiles that, well, you can always implement those yourself.
Basically, D3.js strikes a good balance between ease of use and expressive power, that leans pretty far towards expressive power, while still being far, far less work than rolling your own.
So d3 is actually a _set_ of libraries, of various kind, most of which happen to be useful for data visualisation. Through its richness, there's a good chance that at least _some_ of those libraries will fit your use case well enough. There's nothing stopping you from not using it, and in fact my most recent work was with Canvas, and the only thing I used was d3-scale, which aids mapping datasets into number ranges and selecting ticks.
In another case I might not care about it, but use d3-selection, a mechanism for creating a data-driven DOM tree — but it's pretty clear that you can achieve similar mechanics with React.
Importantly, you don't have to pull the whole thing into your dependencies just because you use a bit of it, and it's light on own dependencies. So just using a small subset of functionality is "cheap."
It gives you a well defined set of primitives to work with. It's also a well established library, so there are a number of good resources on it (and it's seems to be mostly bug free).
d3 has a lot of really useful libraries baked in. You can even use them independently and roll your own svg using its components as utilities. I find this super helpful when using d3 as a whole isn't really a good option (as part of a React/React-Native app for example).
That's not really a fair comparison. Just because something is "baked in" doesn't mean you can't get similar functionality from other libraries in a more ala-carte fashion.
D3 gives you a diff algorithm for free. That handles adding new items to your visualization, updating existing ones, and removing them. You implement the enter, exit, and update (update may have been deprecated). That makes it easy to take your data and handle changes. It will handle animations in a very easy way for each of those cases as well.
In data visualization, creating a scale based on the domain of your data (the range of your data) is annoying. This is handled for you by utility libraries in d3. It also can draw the line ticks of the scale without issues.
Also placing labels is not too easy. D3 does that nicely.
The truth is you can do all of that yourself, but if I were to try it, I would go over the d3 utilities first just to see how complicated some of the visual aspects you haven’t thought of but will need are.
The drawing part of d3 (SVG, canvas) are not so hard to do on your own, but the way d3 isolates the context of what you are coding is very helpful. I’ve only run through the examples and read the d3 book as well as several other charting books, but I don’t use d3 on a day to day.
But let’s go over a more realistic example. In one of my books, the author writes about simple line charts and displaying the quartiles (made by standard deviations) of the data. None of my charting libraries handle that. Rolling my own would be very time consuming. Doing this in d3 would be more feasible than the alernatives.
The last thing I did in D3 probably would have taken several times more effort with raw SVG and JS, I expect. I couldn't find a higher level graphing library that would let me do what I really wanted at all (a custom radial graph that had fairly complex user interaction), and D3 was a perfect sweet spot I think.
It gives you a whole lot of helper functions that really speed things up, data binding, animations, transitions, etc.
I had a similar experience, off the shelf didn't do what the customer wanted, so I delved into D3, which to be fair has the strangest API on first glance.
After learning how it works, and more importantly why, it's more obvious why it's useful. It's a chart building toolkit, not a chart builder, and you can dream up all sorts of interesting things with it.
The upside is that the author has a huge library of examples you can build from [0]. It's a great tool!
I had a project with the same requirements and settled on echarts[0] (version 2 at the time, now on 4). Its adoption was, at that time, very diminished due to everything being in Chinese. Its base graphics library, zrender[1], still suffers this issue. Were it to become more introduced to westerners, it could take the place of d3 for many of these use cases since its API is much easier to grok. Granted it's all canvas, but most uses don't care how it's rendered.
Edit: well, browsing zrender's source, I see svg logic too
For the vast majority of data visualizations that are of the standard type, d3 is a not a scalable way to do it. The lack of high level abstractions means developers can't be as productive can be, and the code base implemented for one svg visualization is likely not usable for another, unless you put abstraction wrappers around it, which is basically nvd3, or other d3 wrapper libraries..
Yeah, but then you just end up making another charting library which is less flexible and limited by the constraints forced by your abstractions. There is a fundamental reason why D3 has remained relevant for so long in an ever-changing JS/Web landscape.
If you want reusability, build your abstractions or use a charting library, but recognize that this will in-turn limit the flexibility of your reusable code.
You can't have your cake and eat it too. It's the very reason why D3 is still so relevant today.
you basically re-iterated what I said... the d3 wrapper libraries are likely sufficient for 85% of the use cases out there. For the others, you either do everything in D3, or you extend those wrapper libraries to give you what you need.
Not exactly. "Extend those wrapper libraries" is not at all trivial or easy because when the wrapper library / charting library creates abstractions, it also places some limitations on the overall customizability. So when the wrapper or charting lib isn't cutting it, trying to shoe-horn the feature you need is not always easy and usually involves ugly hacks that reduce the overall simplicity of the code if you had just rolled your own D3 implementation from scratch.
The difference is between tweaking the wrapper library or write everything from the ground up because one can not deal with tweaking the wrapper library.
I could not agree more with you. So many folks here seem to be missing the actual utility of D3 and why it still remains relevant and useful in todays JS/Web landscape.
If you need standard stuff, there are plenty of awesome plotting libraries that utilize D3 on the back-end while abstracting the complexity away from you. Some even try to give you some amount of flexibility and composability to customize them, but they are always going to have limitations compared to the raw power and flexibility you have if you roll your own D3 visualization.
You can't have it both ways, and that's something a lot of people don't seem to understand. If you're absolutely sure that a visualization library can support all your requirements, then it is absolutely the easier way to go about doing things. But you just have to be really sure that your requirements aren't going to change past the limitations of the visualization library of choice.
D3 is hard because it's not really a library. The claimed 'central abstraction', the stage (with data entering and exiting) is a trivial part of almost all D3 code! Almost all of the effort goes into 1) deciding when and how to invoke the D3 statements, 2) preparing the data, 3) event handling. Most of the code that comes with D3 has nothing to do with D3.
If you look at a force-directed graph in D3, for example, you see how little the library gives you. It's up to you to decompose your data into nodes and edges, it's up to you to draw every shape, it's up to you to trigger a redraw using window.setInterval. And the examples are written in a dense functional, mutable style that uses all the advanced features of JS, SVG, and the DOM - and requires at least some understanding of academic visualisation jargon to boot.
I believe that Bostock took the W3C seriously and wrote a toolkit that "goes with the grain of the web", preferring not to reinvent (or abstract) anything that was already widely available. It exposes the simple truth that the surface area of a modern, standards-compliant browser is enormous, and extraordinarily demanding on the mind of the programmer. And also, the dynamic, functional nature of JavaScript points programmers toward a style that is very much like that espoused by D3.
And yet, I admire and use D3. In part, I think it's honest. If you program browsers for a living, you should master your dependencies! D3 really isn't there to hide the web from you, it entices you to learn the web that's already there. SVG is incredibly powerful and still underused by modern applications, IMHO. And no other library has the breadth and depth of layouts that D3 does, to my knowledge, and the layouts are the hard thing to program (especially performant layouts). So basically the price of admission is learning browsers really well (which takes years), and then you get access to all this sweet layout code for free! It's a good deal, if you ask me.
No mention of Vega in that article or this thread? It's a declarative grammar and toolkit built on D3 (and other viz libs). It's very approachable and addresses a lot of the issues raised in the article:
Agree. It's much easier to pick up. Maybe not much easier to debug what's going wrong (but D3 doesn't make it easy either), but at least the ways things can go wrong are very limited.
I picked up D3 and after a few days got to the "all I wanted was some graphs, not a graph framework factory" point. Vega is great for this.
I’m moderately experienced in d3. I remember trying to pick up Semiotic[0], a similar library, but eventually turning to Victory instead. I will probably stick with it in the future if I do more data visualizing in React.
It's fairly simple and in some cases, gives you the ability to overide functionality with your own functions/React components which is fairly sweet. So for instance, I have customised my tooltip.
I hate to see comments like yours go out without a couple concrete points that led you to your decision.
Feedback like yours can be so valuable, especially with the maintainer of Victory here. There's just no way to replace the knowledge/lessons learned from real life developer user experience.
Well D3 is I guess a DSL in disguise. Javascript is just the vessel here, so to speak. Similar to OpenGL with it's state machine or even worse GLSL with it's deceptively familiar syntax.
Here are two tiny examples I've made with D3. I'd probably choose raw svg and a vdom lib next time, since I'm currently working on a project that mixes Mithril.js and SVG very nicely (although D3 or equivalent would be useful for mapping domains and scaling).
This is the first time I've heard anyone complain about GLSL syntax. (I am only a dilettante in gfx.) What is the problem? It seems pretty well suited to the task in my opinion.
D3 is fantastic, but it should be treated essentially the same as OpenGL. Sure you could roll something super simple and quick with just that. But you're insane to do anything large scale without a higher abstraction.
I don't think that's true at all... D3 is already a higher level abstraction to raw SVG+JS. It's fine for large scale things, as long as those things are actually the kind of things that make sense to implement with D3 and you know what you're doing.
I learned D3 in an academic context. In that environment it was great. When learning about the core theory of data visualization d3.js is an excellent abstraction and lets you make very deliberate decisions over core elements of a chart. Decisions that you can have an explanation for and defend.
In a work context, D3.js is overkill most of the time. People expect line charts, maps, bubble charts that are simple and interactive...doing that in d3.js is a lot of copy/paste from examples and spending days to weeks working on simple details that would come OOTB in something like Highcharts.
I would only recommend d3.js if you really want to dive deep into the constructs of the visualization. If you want a vanilla chart, there's plenty of high level open source libraries you can use.
I have a background in both web development and data science, and have used several different visualization libraries in python, R (ggplot), and JS.
It still takes me a crazy amount of time to get a simple chart or graph working in D3 if it's anything more than copy/pasting an example. The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it. It took me 20-30 hours to get it working, and I'm sure I re-invented multiple things that D3 provides because it was easier to just write them from scratch than trying to find where Bostock hid it in the library and then decoding whatever crazy abstractions he came up with for it. I think he's bad at naming stuff and also bad at writing docs.
By the time I get something working in D3, I've come close to the point where I could have just written all the abstractions myself. I can use a lightweight virtual DOM library and just write functions that take data and output DOM objects.
D3 does barely more than that, at the cost of learning some really funky abstractions.
I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
But, I think the problem is just a hard one, because I'm not aware of anything that's much better :(
Longtime D3 (even Protovis) user here. I think you're very correct in that D3 itself is not a good choice for "a simple chart or graph". If that's all I needed, I'd stick with HighCharts, or ggplot, or any of dozens of other libraries that do simple charts well, and often with a lot more batteries included. My unsolicited advice would be to consider staying away from D3 until you need it.
But if you're doing anything remotely bespoke, with any sort of custom interaction or transition, or anything in which you need to multiplex several datasets in a single set of visualizations, this toolkit is hard to beat.
My favorite personal D3 use case so far is modified animated scatter plots/beeswarms using force layouts. Immortalized for me in the "Four Ways to Slice Obama’s 2013 Budget Proposal" ( https://archive.nytimes.com/www.nytimes.com/interactive/2012... ), I've found that set of techniques useful for visualizing business processes in my own work over the years.
I've yet to encounter anything else that comes close to enabling those kinds of data visualizations. It's the "funky abstractions" some folks have trouble with that enable me to swap out X or Y scales with intuitive animations by literally replacing a single object and then re-invoking an idempotent render() function. Working with D3 is the most productive I've ever been in my life from a data visualization standpoint, and I suspect that will continue to be the case for some years.
> It still takes me a crazy amount of time to get a simple chart or graph working in D3
Wrong tool for the job.
D3 fills the void between "just send my data to a bar chart" and "I need some charts, maybe I'll use raw JS to manipulate SVG's". You're probably in the former camp.
Google Charts is an option I use a lot - surprised more people don't know about it. Maybe just because it was developed before the hip new land of React? (A bunch of react wrappers exist for it though)
This. I rewrote pure-JS implementation of some topology visualization in D3 and very much regretted it. The abstractions are simply weird, documentation poor and the resulting code difficult to reason about and maintain. Useful for toy projects, but I wouldn't use it in a serious app anymore.
> I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
Interestingly enough, I find that the combination of React + inline SVG fill this space pretty nicely. The engine is well thought out and documented, as is SVG. And animations are easier to do with CSS anyway.
EDIT: note that React is declarative too, which helps with visualizations a lot.
I've just started learning D3, and my impression is that the biggest problem it has isn't really D3s problem at all, it's everyone else's. That is, every major single page app framework out there needs control over the DOM. I've seen a few articles talking about some pretty weird stuff you have to do to be able to use D3 and ReactJS in the same app. Hopefully it's not like that with all frameworks, but I'd really like to see more info written about this.
I've built a lot of charts/visualization with React + D3.
I normally use D3 only for the math as rendering with React is much faster, easier to reason about and allows for features like server side rendering. Animations/transitions are not a solved problem with React, which is why it's probably far easier to create an animated chart with D3 exclusively, as the render loop of D3 with its enter/update/exit is really powerful.
Nowadays I’m building a lot of visualisations with a Vue.js SVG template and d3 for its helper functions like scale, axis, path, colormaps etc. It’s really powerful and declarative and no need for enter/update/exit as everything is kept in sync by Vue data bindings. However the enter/update/exit pattern stays indeed better for animations.
158 comments
[ 2.2 ms ] story [ 215 ms ] threadFirst, I think if you want to use d3, you probably just want a charting library. Convince yourself that HighCharts is not flexible enough and then try d3. So the trouble with d3 is that most people don’t need to use it and are in over their heads.
Second, d3 is full of leaky abstractions. Imagine if cars didn’t have acceleration pedals, and you had to manually open the throttle and engage the fuel pump yourself. You’d need an engineering degree to drive your car! Same thing with d3; it’s ridiculous that you need to know the ins and outs of some of the internet’s most complicated and frustrating technology just to show a couple of data points on a graph. It’d be great if you could work with d3 and pretend like SVG and the DOM didn’t exist.
Comparatively speaking, learning ggplot2 is a sublime experience. After about a week, you feel like you have super powers.
D3 does feel overcomplicated for just building static visualizations, but if you're building custom interactive stuff, learning it can be nearly as joyous an experience.
Whereas seaborn has similar pre-canned plots, but it's all built on matplotlib so you still have that flexibility if you want it.
When I look at the examples at their page I see stuff that I could make in 2 ways:
- Just write a small program from scratch that visualizes one in a week or 2. It will be small, I will know the ins and outs, so easy to change and will be a ton of fun to write.
- Use D3.js (I dont know this library, but i've used complex charting libraries in the past). Likely it will take more time, not because I am thinking of the interesting parts, I will spend most of my day cursing on StackOverFlow to figure out how to skin that one tricky part of my app.
So in the end I dont really see a good reason to use d3.js unless my full time job is to develop data visualizations.
I still wouldn't say I've "learned D3", but I didn't have to. I got my goal accomplished in a couple of days, along with a lot of nice bits I didn't intend along the way as I found good things that D3 could do for me.
For me it was definitely a "design to tools" approach; I looked at the gallery, found something mostly like what I wanted, and then worked to that. That was fine by me; I wasn't looking for ultimate visual control, but just trying to turn a bunch of data into something mortals could comprehend.
Unfortunately this is widespread across the JS ecosystem. Hell I'm not saying anyone should do it like Linux (break userspace and get shot), but when one does backwards-incompatible changes, is it so hard to stub dead methods or unsupported calling ways with "console.error('Outdated, see https://github.com/x/y/issues/z');"?
It's source (Vue-based, potentially for other showcases/searches): https://github.com/wbkd/d3-discovery (I just asked if it can be open source, and to my awe its author added MIT license :)).
For example, I built a in-door navigation app. It would receive its data via MQTT/WebSockets and update markers on custom maps.
D3 was perfect for this, it helped with synchronizing graphics with data, converting mapping cooridnates to DOM/SVG coordinates and easing all the updates so stuff looked smooth.
You just would pump the incoming data into the D3 pipeline and be done with it.
If you just need a few animations, there are a bunch of simple libraries for this. React has react-motion for example, it's not as flexible as D3 but you get most stuff you would expect out of it without much hassle.
If you just need a set of basic charts, use a charting library. Highcharts or Charts.js are both very capable.
Even when compared with using C++ I have found using JavaScript based libraries difficult D3 in particular I found to be horrendously complex.
Edit: The title of the stock photo is "Timmermansgereedschap" (NL) = Zimmermannsgerätschaften (DE) = carpenter's tools (EN).
Running with the analogy in the article -- it is fruitless for someone who's never done any woodworking or furniture design ever to hope to become the next Charles Eames within the span of a few months.
Ive done a bunch of javascript, DOM and visualisations in OpenGL, so i feel i should have a good chance.
That said, I mostly learned it through the examples rather than a tutorial.
A piece of cake! Here is a quantum game in D3.js: http://quantumgame.io/ :)
(full disclaimer: mine, and I already do too much of its shameless self-promotion)
If you can grasp this idea of visualisation as creating a mapping function, then D3 becomes a beautiful toolbox for assisting you. You decide which dimension of your data you want to represent as the horizontal dimension, which as the vertical, as color, as size, etc. You decide how you want to scale them. Then you use D3's high-order functions to create the appropriate individual mapping functions, compose them into your complete visualisation function, and pass it your domain data.
If you think of D3 as a "charting library", or a collection of visualisation classes or objects, then you're going to struggle. But the problem isn't with D3, it's that you haven't really understood the paradigm of the library.
...
...You might assume that g, fill, or stroke might be d3 terminology but it's really SVG! A quick Google of "svg g" will get you to https://developer.mozilla.org/en-US/docs/Web/SVG/Element/g
It blew my mind open, hope to share the same.
D3 is awe-inspiring in its scope, and learning to use it has a steep learning curve. But for anything beyond trivial use cases the freedom it affords you is worth the effort.
https://github.com/d3/d3/wiki/Tutorials
Somewhere else someone described D3 as a hammer, and that's a good analogy, because if you don't start with a hammer, it's the first tool you'll make for yourself when you need to pound nails.
[0] http://alignedleft.com/work/d3-book
Consider how much JS you'd have to write to even start creating all the frameworks to build in native SVG structures... a lot.
I created html graphs with tables in the late 90s, made Flash graphs in 2000s, upgrades to numerous JS based charting libraries after this, building custom graphs the whole way.
I had looked at D3 so many times over the years when it first showed up online. But it "looked" complicated, no obvious "chart" library, etc...
Finally, when I tried D3, I was like "why the f didn't I start with this?"
Basically, D3.js strikes a good balance between ease of use and expressive power, that leans pretty far towards expressive power, while still being far, far less work than rolling your own.
In another case I might not care about it, but use d3-selection, a mechanism for creating a data-driven DOM tree — but it's pretty clear that you can achieve similar mechanics with React.
Importantly, you don't have to pull the whole thing into your dependencies just because you use a bit of it, and it's light on own dependencies. So just using a small subset of functionality is "cheap."
Such as: https://github.com/d3/d3-shape
https://github.com/d3/d3-scale
Doing it from scratch without the help of utilities like these would be much harder.
A couple examples:
var line = d3.line() .x(function(d) { return x(d.date); }) .y(function(d) { return y(d.value); }) .curve(d3.curveCatmullRom.alpha(0.5));
line(data); // gives you svg path
and
var color = d3.scaleLinear().domain([10, 100]).range(["brown", "steelblue"]);
color(20); // "#9a3439" color(50); // "#7b5167"
All of the functionality in d3 is split up into smaller modules that can be required independently.
In data visualization, creating a scale based on the domain of your data (the range of your data) is annoying. This is handled for you by utility libraries in d3. It also can draw the line ticks of the scale without issues.
Also placing labels is not too easy. D3 does that nicely.
The truth is you can do all of that yourself, but if I were to try it, I would go over the d3 utilities first just to see how complicated some of the visual aspects you haven’t thought of but will need are.
The drawing part of d3 (SVG, canvas) are not so hard to do on your own, but the way d3 isolates the context of what you are coding is very helpful. I’ve only run through the examples and read the d3 book as well as several other charting books, but I don’t use d3 on a day to day.
But let’s go over a more realistic example. In one of my books, the author writes about simple line charts and displaying the quartiles (made by standard deviations) of the data. None of my charting libraries handle that. Rolling my own would be very time consuming. Doing this in d3 would be more feasible than the alernatives.
It gives you a whole lot of helper functions that really speed things up, data binding, animations, transitions, etc.
After learning how it works, and more importantly why, it's more obvious why it's useful. It's a chart building toolkit, not a chart builder, and you can dream up all sorts of interesting things with it.
The upside is that the author has a huge library of examples you can build from [0]. It's a great tool!
[0] https://github.com/d3/d3/wiki/gallery
Edit: well, browsing zrender's source, I see svg logic too
0 - https://github.com/apache/incubator-echarts
1 - https://github.com/ecomfe/zrender
https://developers.google.com/chart/
High quality, very good browser support, good docs, and well-maintained. There are wrappers out there for React/Vue/whatever too
For the vast majority of data visualizations that are of the standard type, d3 is a not a scalable way to do it. The lack of high level abstractions means developers can't be as productive can be, and the code base implemented for one svg visualization is likely not usable for another, unless you put abstraction wrappers around it, which is basically nvd3, or other d3 wrapper libraries..
If you want reusability, build your abstractions or use a charting library, but recognize that this will in-turn limit the flexibility of your reusable code.
You can't have your cake and eat it too. It's the very reason why D3 is still so relevant today.
Pros and Cons with both approaches no doubt.
If you need standard stuff, there are plenty of awesome plotting libraries that utilize D3 on the back-end while abstracting the complexity away from you. Some even try to give you some amount of flexibility and composability to customize them, but they are always going to have limitations compared to the raw power and flexibility you have if you roll your own D3 visualization.
You can't have it both ways, and that's something a lot of people don't seem to understand. If you're absolutely sure that a visualization library can support all your requirements, then it is absolutely the easier way to go about doing things. But you just have to be really sure that your requirements aren't going to change past the limitations of the visualization library of choice.
If you look at a force-directed graph in D3, for example, you see how little the library gives you. It's up to you to decompose your data into nodes and edges, it's up to you to draw every shape, it's up to you to trigger a redraw using window.setInterval. And the examples are written in a dense functional, mutable style that uses all the advanced features of JS, SVG, and the DOM - and requires at least some understanding of academic visualisation jargon to boot.
I believe that Bostock took the W3C seriously and wrote a toolkit that "goes with the grain of the web", preferring not to reinvent (or abstract) anything that was already widely available. It exposes the simple truth that the surface area of a modern, standards-compliant browser is enormous, and extraordinarily demanding on the mind of the programmer. And also, the dynamic, functional nature of JavaScript points programmers toward a style that is very much like that espoused by D3.
And yet, I admire and use D3. In part, I think it's honest. If you program browsers for a living, you should master your dependencies! D3 really isn't there to hide the web from you, it entices you to learn the web that's already there. SVG is incredibly powerful and still underused by modern applications, IMHO. And no other library has the breadth and depth of layouts that D3 does, to my knowledge, and the layouts are the hard thing to program (especially performant layouts). So basically the price of admission is learning browsers really well (which takes years), and then you get access to all this sweet layout code for free! It's a good deal, if you ask me.
https://vega.github.io/vega/about/vega-and-d3/
https://vega.github.io/vega-lite/
I picked up D3 and after a few days got to the "all I wanted was some graphs, not a graph framework factory" point. Vega is great for this.
[0]: https://emeeks.github.io/semiotic/#/
If anyone is looking for a React-based charts library check out Recharts at https://github.com/recharts/recharts.
It's fairly simple and in some cases, gives you the ability to overide functionality with your own functions/React components which is fairly sweet. So for instance, I have customised my tooltip.
Feedback like yours can be so valuable, especially with the maintainer of Victory here. There's just no way to replace the knowledge/lessons learned from real life developer user experience.
Here are two tiny examples I've made with D3. I'd probably choose raw svg and a vdom lib next time, since I'm currently working on a project that mixes Mithril.js and SVG very nicely (although D3 or equivalent would be useful for mapping domains and scaling).
https://htmlpreview.github.io/?https://github.com/dsego/dseg...
https://htmlpreview.github.io/?https://github.com/dsego/dseg...
In a work context, D3.js is overkill most of the time. People expect line charts, maps, bubble charts that are simple and interactive...doing that in d3.js is a lot of copy/paste from examples and spending days to weeks working on simple details that would come OOTB in something like Highcharts.
I would only recommend d3.js if you really want to dive deep into the constructs of the visualization. If you want a vanilla chart, there's plenty of high level open source libraries you can use.
It still takes me a crazy amount of time to get a simple chart or graph working in D3 if it's anything more than copy/pasting an example. The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it. It took me 20-30 hours to get it working, and I'm sure I re-invented multiple things that D3 provides because it was easier to just write them from scratch than trying to find where Bostock hid it in the library and then decoding whatever crazy abstractions he came up with for it. I think he's bad at naming stuff and also bad at writing docs.
By the time I get something working in D3, I've come close to the point where I could have just written all the abstractions myself. I can use a lightweight virtual DOM library and just write functions that take data and output DOM objects.
D3 does barely more than that, at the cost of learning some really funky abstractions.
I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
But, I think the problem is just a hard one, because I'm not aware of anything that's much better :(
But if you're doing anything remotely bespoke, with any sort of custom interaction or transition, or anything in which you need to multiplex several datasets in a single set of visualizations, this toolkit is hard to beat.
My favorite personal D3 use case so far is modified animated scatter plots/beeswarms using force layouts. Immortalized for me in the "Four Ways to Slice Obama’s 2013 Budget Proposal" ( https://archive.nytimes.com/www.nytimes.com/interactive/2012... ), I've found that set of techniques useful for visualizing business processes in my own work over the years.
I've yet to encounter anything else that comes close to enabling those kinds of data visualizations. It's the "funky abstractions" some folks have trouble with that enable me to swap out X or Y scales with intuitive animations by literally replacing a single object and then re-invoking an idempotent render() function. Working with D3 is the most productive I've ever been in my life from a data visualization standpoint, and I suspect that will continue to be the case for some years.
Wrong tool for the job.
D3 fills the void between "just send my data to a bar chart" and "I need some charts, maybe I'll use raw JS to manipulate SVG's". You're probably in the former camp.
http://www.chartjs.org/docs/latest/
The examples do "cubic" instead of just "connecting the dots" but luckily it can be turned off:
http://www.chartjs.org/samples/latest/charts/line/interpolat...
https://developers.google.com/chart/
`The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it.`
That doesn't sound like "just send my data to a bar chart" to me.
> I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction.
Interestingly enough, I find that the combination of React + inline SVG fill this space pretty nicely. The engine is well thought out and documented, as is SVG. And animations are easier to do with CSS anyway.
EDIT: note that React is declarative too, which helps with visualizations a lot.