It's built on top D3 and has the same limits (5k nodes 20k edges should not be a big problem). The "big" here is the ability to combine nodes, for two reasons:
1 Reduce the number of nodes and edges, thus increasing capacity
2 Combine nodes that should be seen together (e.g. alternative spellings and typos), to better deal with the variation-aspect of big data
I see your point, but the goal here was to illustrate the functionality.
At some point you run out of pixels on your screen to visualize it all at once. My suggestion in your case is to not use a graph at the beginning, but rather grouping nodes based on metrics. Like degree (group all contacts of a node that only has a single contact), communities, or value (e.g. IP's in a subnet).
To illustrate the functionality, the point of the original poster is a valid one. The interactions for managing 5k nodes are not necessarily the same as those for managing 10 nodes - not just in terms of performance, but also in how to make sense of all the information shown; so showing a realistic use case is required to assess the possibilities of the tool.
On a side note, the "right-click to select nodes" doesn't work well in Firefox, with its "always show context menu" option. At the "up" event, the context menu shows and the selection is erased.
In general, using right click on web applications is a terrible idea, since the standard function in browsers is to show a context menu for the clicked element. IMHO a lightweight selection mode or quasimode would work better.
It's hard to get a sense of how it'll look with 'big messy data', when it's sample dataset is so tiny, at 14 nodes and 21 edges. Is there a way to see how it'll look with a ton of data?
Yes. Animating node placement has a significant performance penalty, so in larger graphs you would either hide edges or everything during the first couple of seconds.
Are you referring to node placement or selection/merging? Node placement is done with D3 force directed layout, so I don't think I'll be able to do much there. And again, animation makes it more soggy.
Thanks, I had the same question and you answered it nicely — I will stick with Vis.js (http://visjs.org/network_examples.html) for now. It seems to work better for larger graphs and allows me to build a pretty good ui for exploring the graph, too.
For the yesgraph link I had to disable ad-blocking and uBlock Origin and still only got a static image with a Twitter advertisement pasted over the image.
The graphistry link is similarly useless just showing a few sentences on a few pages ending in a "request demo" button.
Hi Ivan, always cool to see what Graphistry users are doing!
Phil, sad to hear you weren't able to see your Twin Graph. Many of us use ad blockers and this is the first report we've gotten like yours, so we'll dig in. Meanwhile, you may be able to try a direct link to my own YesGraph TwinMap: https://labs.graphistry.com/graph/graph.html?dataset=lmeyero... . (Note: best on laptops, and we recently relaunched with Falcor/React, so currently porting all our page load optimizations.)
For more information about graphistry, we have users piloting the three below layers of our stack. Because we can load 10-100X more data at the visual tier than other systems here (so 100K-1M+ things), people have been exploring connections across events/entities for some fascinating reasons:
* Investigation & Response -- Connect to systems like Splunk and get rich, scalable visual graph views and easy workflow automation. Ex: build an investigation template that takes an indicator of compromise and runs queries that connect it to various users, devices, alerts, etc. Or, "here are our ssh trails and anomalies around them."
* Exploration: Data scientists and data analysts will explore connections in their events or samples, e.g., for week-over-week model tuning, security research & forensics, & even now loan analysis. They'll load in a bunch of events or samples where each may have a lot of attributes (IPs, times, amounts, ...), and then they can see correlations. Ex: most false positives are from events with 3 particular combinations of characteristics, or an outage involved 4 distinct phases of behavior and entities.
* Developers: folks building internal apps for scenarios like the above.
For the latter two use cases, a good place to get started is our API: https://github.com/graphistry/pygraphistry . Feel free to contact us at info@ if this may solve a problem for you. (And.. we're hiring! Help us build web-based visual tools with GPUs acceleration to solve real data problems!)
The lasso seems to "give up" if too many nodes are selected - stops updating the selection although it keeps updating the green shaded area. Perhaps it should only give up on the text labels which are probably the expensive part.
32 comments
[ 2.6 ms ] story [ 69.5 ms ] thread1 Reduce the number of nodes and edges, thus increasing capacity
2 Combine nodes that should be seen together (e.g. alternative spellings and typos), to better deal with the variation-aspect of big data
I see your point, but the goal here was to illustrate the functionality.
In general, using right click on web applications is a terrible idea, since the standard function in browsers is to show a context menu for the clicked element. IMHO a lightweight selection mode or quasimode would work better.
https://gransk.com/ggraph-bigger.html
Edit: It's an excerpt of the Enron dataset.
It's getting noticeably laggy for me at 611 nodes, but below you say it shouldn't be a problem to do 5k. Have you actually tested that kind of data?
I was hoping for something that maybe looked like a set of hierarchical clusters. Where I could drill down level by level.
Maybe you can submit it to Neo4j team to be showcased in their "Graph Visualization for Neo4j" section: https://neo4j.com/developer/guide-data-visualization/
The minimal eng required to get good performance was wonderful
The graphistry link is similarly useless just showing a few sentences on a few pages ending in a "request demo" button.
Phil, sad to hear you weren't able to see your Twin Graph. Many of us use ad blockers and this is the first report we've gotten like yours, so we'll dig in. Meanwhile, you may be able to try a direct link to my own YesGraph TwinMap: https://labs.graphistry.com/graph/graph.html?dataset=lmeyero... . (Note: best on laptops, and we recently relaunched with Falcor/React, so currently porting all our page load optimizations.)
For more information about graphistry, we have users piloting the three below layers of our stack. Because we can load 10-100X more data at the visual tier than other systems here (so 100K-1M+ things), people have been exploring connections across events/entities for some fascinating reasons:
* Investigation & Response -- Connect to systems like Splunk and get rich, scalable visual graph views and easy workflow automation. Ex: build an investigation template that takes an indicator of compromise and runs queries that connect it to various users, devices, alerts, etc. Or, "here are our ssh trails and anomalies around them."
* Exploration: Data scientists and data analysts will explore connections in their events or samples, e.g., for week-over-week model tuning, security research & forensics, & even now loan analysis. They'll load in a bunch of events or samples where each may have a lot of attributes (IPs, times, amounts, ...), and then they can see correlations. Ex: most false positives are from events with 3 particular combinations of characteristics, or an outage involved 4 distinct phases of behavior and entities.
* Developers: folks building internal apps for scenarios like the above.
For the latter two use cases, a good place to get started is our API: https://github.com/graphistry/pygraphistry . Feel free to contact us at info@ if this may solve a problem for you. (And.. we're hiring! Help us build web-based visual tools with GPUs acceleration to solve real data problems!)
https://anvaka.github.io/graph-drawing-libraries/#/all