Like, a reasonably complex program is going to have a call graph with tens or even hundreds of thousands of nodes. Graphviz won't save you. Collapsing sccs won't save you. It won't actually be useful to navigate.
Cool tool nonetheless, have built similar things to visualize code architecture and create doc artifacts in CI, usually the biggest challenge is to scope the pieces that will be included and what layout algo to use, maybe a feature to implement?
emacs. LSP (under editor such as emacs[0]) provides the context/way for being able to define a specific language parse specifics about where to add/define emacs hooks/jump points for a given language nodes/files/modules/functions)
I haven't looked very hard, but `pip install codegraph` didn't result in a codegraph command anywhere. (This was macos, using a Python 3.8 venv.)
Digging around a bit found a 'cg' command, which seems to be codegraph (based on the --help output). Didn't show anything interesting for a Django app, though... I'm guessing it couldn't really follow what manage.py was doing?
Anyhow, if 'cg' is the command, might want to update the docs.
README for codequery[0]outlines an overview of tool set to go from source code to visualization. (although, not as a generate live updates per editor changes)
Good concept. visualization is poor. Force Atlas is really not that great for structural analysis, and nodes need to be weighted by degree among other more code-specific factors. Consider bringing the code graph into NetworkX or some similar library and using graphML to leverage existing visualization solutions rather than reinventing the wheel.
I was really surprised to saw a lot of stars on one of my oldest repos )) it was just for my purposes and I didn't continue work on it because thought that it is not needed at all ))
Other posts in this thread are touching on it, but graph layout and layer segmentations are where you could improve.
Screen real-estate is the biggest challenge when visualizing large amounts of code.
Similar to maps show different detail levels based on aspect ratio and zoom, you can decompose your "features" (mapping terminology) based on level, i.e. packages->modules->functions->code
OP, have you ever used Gephi? I used to build static analysis tools for viz of large-scale codebases. Taking a bit of time to export analysis data into a Gephi format (GEXF in my case) really helped.
In Gephi, you can play around with different layout algorithms (peep the plugins). This may help you select a layout algo or combination thereof to help render tons of data in a human-friendly form.
OP here's a couple examples of large scale static analysis diagrams.
[1] is a TypeScript database with ~250K LoC. A fairly advanced product that would benefit from what you're building. The colors represent different architectural layers. Each node is a folder with files in it. Edges represent inbound/outbound dependencies. There's simply no way for it to be labelled effectively.
[2] is the same codebase, but instead of inter-module dependencies, it's showing function/method calls.
One takeaway after working in this space for a while was that visualizing spatial relationships of anything sufficiently complex is extremely hard on a monitor. These representations benefit tremendously from being interactive.
Can anyone recommend something similar for Java / Smali ? Specific use case is reverse engineering Android application with partially obsufcated code.
Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.
Very nice. For those interested, you can get similar information using SourceGraph and LSIF in a standardized, language agnostic form: https://lsif.dev/. It still generally requires build information for each project/language, unfortunately.
I'd be curious if someone loaded these into http://github.com/graphistry/pygraphistry and tried something like overlaying a GNN anomaly detection over the base visuals. Both parts are automated & GPU accelerated, so can probably handle some decently sized projects!
I honestly don't wish to do the original poster down, but has anyone actually found code visualisation things like this (graph or otherwise) useful, as opposed to just pretty?
It seems like you would only really have need for visualisation if your codebase is complex enough. And if it is complex enough, visualising it is generally something of a mess.
What actionable insights can be generated? Has anyone changed anything as a result of such a visualisation?
It always feels to me like you're better off figuring out which modules you have shouldn't depend on certain other modules and finding only those links. Which is often best done with a simple ripgrep, no?
29 comments
[ 2.5 ms ] story [ 70.9 ms ] threadLike, a reasonably complex program is going to have a call graph with tens or even hundreds of thousands of nodes. Graphviz won't save you. Collapsing sccs won't save you. It won't actually be useful to navigate.
This needs more sophistication to be useful is what I'm saying.
Cool tool nonetheless, have built similar things to visualize code architecture and create doc artifacts in CI, usually the biggest challenge is to scope the pieces that will be included and what layout algo to use, maybe a feature to implement?
https://github.com/glato/emerge
The HN title matches the repo's README, which was authored by someone whose English language skills are questionable.
Although in my case it'd ideally be language-agnostic and hook into LSP so that I can click on nodes and jump to various files/modules/functions.
Still a cool project though!
https://www.radare.org/r/
[0] : https://www.masteringemacs.org/article/tree-sitter-complicat...
Digging around a bit found a 'cg' command, which seems to be codegraph (based on the --help output). Didn't show anything interesting for a Django app, though... I'm guessing it couldn't really follow what manage.py was doing?
Anyhow, if 'cg' is the command, might want to update the docs.
[0] : https://github.com/ruben2020/codequery
Other posts in this thread are touching on it, but graph layout and layer segmentations are where you could improve.
Screen real-estate is the biggest challenge when visualizing large amounts of code.
Similar to maps show different detail levels based on aspect ratio and zoom, you can decompose your "features" (mapping terminology) based on level, i.e. packages->modules->functions->code
OP, have you ever used Gephi? I used to build static analysis tools for viz of large-scale codebases. Taking a bit of time to export analysis data into a Gephi format (GEXF in my case) really helped.
In Gephi, you can play around with different layout algorithms (peep the plugins). This may help you select a layout algo or combination thereof to help render tons of data in a human-friendly form.
[1] is a TypeScript database with ~250K LoC. A fairly advanced product that would benefit from what you're building. The colors represent different architectural layers. Each node is a folder with files in it. Edges represent inbound/outbound dependencies. There's simply no way for it to be labelled effectively.
[2] is the same codebase, but instead of inter-module dependencies, it's showing function/method calls.
One takeaway after working in this space for a while was that visualizing spatial relationships of anything sufficiently complex is extremely hard on a monitor. These representations benefit tremendously from being interactive.
[1] https://i.imgur.com/IdY5ghC.jpg [2] https://i.imgur.com/rv4OYun.jpg
Current approach is following “xrefs” in JADX and hooking methods with dynamic instrumentation with Frida (displaying backtraces etc). Got to be a quicker way while chipping away renaming classes and methods and mapping out source/sinks.
I built something similar for working with Java code, but it could be extended to other languages.
https://packagemap.co
I've seen engineers find it hard to work out if their code is organised well together. So I built this tool to show them visually.
It seems like you would only really have need for visualisation if your codebase is complex enough. And if it is complex enough, visualising it is generally something of a mess.
What actionable insights can be generated? Has anyone changed anything as a result of such a visualisation?
It always feels to me like you're better off figuring out which modules you have shouldn't depend on certain other modules and finding only those links. Which is often best done with a simple ripgrep, no?