Ask HN: Visualizing software designs, especially of large systems (if at all)?

This tool is good for making simple UML diagrams and even lets you do it with simplified syntax

https://plantuml.com/

I'd say the big problem in visualizing big systems is that you can't usefully do it in one graph. For instance I worked on a system that had 2000+ database tables if you were going to make a diagram of that which shows everything it is going to take up a long wall. (This can be useful, but it is a big commitment)

A useful tool is going to let you make meaningful diagrams that show the subset of entities that are part of a story. I went to an art show of Mark Lombardi's works

https://en.wikipedia.org/wiki/Mark_Lombardi

who (before he was murdered) drew elaborate diagrams of conspiracies. One thing they showed was drafts that he made in the progress of creating his visualizations and he would sometimes make 40 or more of them. He would start out with a "hairball" that was disorganized and gradually figure out how to lay the diagram out in a way that made the meaning obvious.

Specifically for TypeScript I created a CLI to visualize the call graph

https://github.com/whyboris/TypeScript-Call-Graph

Works for _functions_ not classes. I'm unsure how useful this tool is, but I suspect it might be helpful in some codebases.

I have been working on a natural design system for visually thinking about technology. Maybe it will help you.

Its called MTREES and its a free epub on Apple or Google.

https://mtrees.io

Depends on what you want to understand about the system.

If you're looking to understand it's properties and how it behaves I would look into something more robust like Alloy 6 [0] which has a great visualization system for inspecting models. However if you're looking for a class diagram tool then it's outside your wheelhouse.

There's also something I've played around with a bit but haven't used seriously: Moose [1]. It's basically an IDE for doing analysis of code. The trick is writing good parsers.

[0] https://alloytools.org/alloy6.html

[1] https://moosetechnology.org/

Check out https://c4model.com/

Why are all these comments suggesting new tools instead of answering your question? From what I've seen, high documentation only comes into higher level abstractions such as individual services or architecture design.

In school, I was taught to draw UML for classes within a program. I have never seen that IRL. I think the difference is the time required to comprehend the application.

I gave this a bit of thought earlier this year (https://alexanderell.is/posts/visualizing-code/) and, with the help of HN commenters, collected a small list of ways people are working to help with code visualization. I don't think most of them are production ready (some are just research papers), but you may find them interesting all the same.

SoftVis3D (https://softvis3d.com/): where a "‘code city’ view provides a visualization for the hierarchical structure of the project".

Code Park: A New 3D Code Visualization Tool (2017) (https://arxiv.org/pdf/1708.02174.pdf), a “novel tool for visualizing codebases in a 3D game-like environment” with code represented as “code rooms” with code on the walls.

Code Structure Visualization Using 3D-Flythrough (2016) (https://opus-htw-aalen.bsz-bw.de/frontdoor/deliver/index/doc...), with spatial metaphors and first-person exploration of code.

Primitive (https://primitive.io/), a VR collaboration startup with a Matrix-looking “Immersive Development Environment” with “new tools for visually analyzing software in 3D”.

AppMap (https://appland.com/docs/how-to-use-appmap-diagrams.html), an automated code analysis tool that includes dependency maps and trace views.

plurid (https://github.com/plurid/plurid), a framework for visualizing and debugging code in a 3D explorable structure.

fsn (file manager) (https://en.wikipedia.org/wiki/Fsn_(file_manager)), an experimental application to view a file system in 3D (featured in Jurassic Park).

No automated tool will come close to having a 5 minute conversation with the main designer and having him draw you a diagram freehand on the back of a napkin. This is a social and organizational and communication problem, not technical.

If that can't be done, there are some interesting things you can try. A lot of the suggestions in the thread are "top down" methods; you can get a lot of value out of "bottom up" visualizations too. Things like:

- Histograms of which lines / functions get called the most, or spent the most time in

- Which lines / functions / files get changed the most in the git history

- CPU flamegraphs

- Plain old print-debugging

In over-architected systems it can be difficult to figure out where the real "meat" of the code is, as opposed to the endless layers of configuration and wrappers and interfaces and indirection. UML diagrams may not help, or even be deceiving, but a stack trace never is.

Different types of diagrams highlight different aspects of a system.

If you can, try to put together 2 different diagrams presenting different views. For example, sequence diagrams, model diagrams, etc. each tell you something different. Having a few different perspectives on the same system will give you a richer understanding.

Not specifically related to visualization, but, also, if you spend a lot of time trying to understand a piece of code, once you understand it, document it (e.g. in a function or method comment). Over time, this makes a huge difference and will help you and others out when you revisit the code later.

I'm a big fan of Terrastruct (https://terrastruct.com/). They focus on building great software for visualizing complex software architecture. Their secret sauce is this idea of attention where they allow you to zoom in and out so you can get the 10 foot view or 10,000 foot view, whatever you find most useful.

The challenge is that there are different ways of "mapping" software.

You could map the way programs fit into machines, and the networks between them. This would be the topology.

You can map the way services call upon one another with requests. This is the service graph.

You can map how systems interact over events or shared resources. You could say this is the logical graph.

The problem happens when you try and graph them all at once. It's the same as trying to draw a real map, with all the services, bus routes, railways, shops and administrative regions superimposed on one image. It's very busy.

So I use separate maps.

Tools are another matter. Personally I use Mermaid for graphs. I also have my own tools that create SVG visualisations using DAGre. This can be helpful for interactive visualisations where you can click into different nodes and explore more detail.

My system uses CloudFormation templates and our in house deployment DSLs to figure out the "topology", then let the users see the different superimposed "graphs" as they see fit

I'm a visual learner. I wanted a way to see how documentation of complex systems looked too. I'm working on a https://gainknowhow.com/software-companies.html . It is basically like a mind map for documentation. The high level documentation like core values is on top connected through learning paths to learn the complete context of low-level skills.

We like using PlantUML and it embeds in Asciidoc nicely.

Also the C4 Model seems cool but I don't currently use it

https://c4model.com/

I recently started using https://onemodel.app and it really makes diagramming super fast and easy.

I'm a fan of https://www.ilograph.com/. I've only used it for a few small things, but the author has good samples, including a diagram of ilograph itself - https://app.ilograph.com/demo.ilograph.Ilograph/Request.

UML is not a documentation tool, it´s a communication tool. It is a standardized way to communicate. It removes ambiguity. UML is the best way to manage, communicate and handle a large amounts of complexity. Most of the free UML tools are very basic and does not have traceability features (we can see many in the comments), and this is really a no-go if you want to understand large systems.

The most important rule is that you can’t include everything in a single doc as it quickly gets too big and complicated.

So you need to break it down into multiple documents that cover different use cases and include only the components for a small set of use cases.

Then use the C4 model to break up the docs.

I like using https://c4model.com/ - the Level 2 diagram is particularly useful.

I use https://mermaid-js.github.io/mermaid/#/ for the diagram itself because Github natively supports it in markdown files, so you can revision control the diagram. I managed to get reasonably close to the C4 diagrams minus a few features that mermaid does not support.

A nice talk at kubecon last year spoke about visualizing protocols and more complex multi-party exchanges.

The suggestion it made was to look across module boundaries or microservice boundaries, while making a particular request. So I go to update a Project with a new Pipeline, and I find out that the project service talks to the pipeline service which then has to create resources for which it talks to various resource services, all of that.

Since there aren't great digital tools for doing this, for visualizing these things, it was suggested in that talk that one of the more outside the box things you could do is to just take cardboard and modeling clay and strings and cut out some shapes to represent the different services, let each string be an RPC, and the actual diagram you would imagine being alive in time, like little beads flying across these strings, to indicate requests going out and then responses coming back... But the key was less to witness the time but more to get a timeless sense of connection, “oh, it turns out ResourceService is very hairy in these diagrams, it is kind of the central hub for this microservice cluster.” and for that it was helpful that the visualization had a certain physicality to it, it had weight and structure and engaged more senses than just the visual...

I've actually thought about just digitizing these ideas, even though it misses half the point LOL. I think that could be some wonderful documentation and diagrams in our onboarding for new folks.

I want to make a computer runtime which doesn't execute the program but instead allows for a programmer to understand what's going on.

a metacomputer of sorts; maybe epicomputer would be a better name?

For a small but complicated project I got thrown into a while ago, the only way for me to understand it was to print out all the source directly, vertically tape together the pages for a single file, and then lay them all out on a huge table. Then I took multicolored markers and started physically drawing out the call chains. I then I sers-toi the system, and also found an enraging bug: the system widely used the variables "blah_name" and "blah_id", including in many functions' parameters. Except, in one case, blah_id was passed in as blah_name and thenceforth became known as blah_name.

I don't know if an automated visualization system is possible, but you'll have to understand the whole thing before doing so. Pen and paper was the most expedient solution for me at the time.

I use what I call "UML Lite" (UML, without all the frou-frou). It's handy for illustrating points[0].

What I generally do, is start with what I call a "napkin sketch" (which can be UML-ish)[1], and try to avoid writing down too much stuff, in order to reduce "concrete galoshes"[2].

I try to use tools like Doxygen and Jazzy, to document the code, in an inline fashion[3]. Doxygen will generate a UML "Lite" diagram[4].

[0] https://littlegreenviper.com/miscellany/swiftwater/the-curio...

[1] https://littlegreenviper.com/miscellany/forensic-design-docu...

[2] https://littlegreenviper.com/miscellany/concrete-galoshes/

[3] https://littlegreenviper.com/miscellany/leaving-a-legacy/

[4] https://doxygen.nl/manual/diagrams.html

Some thoughts and attempt to visualize Go code in a meaningful way: https://divan.dev/posts/visual_programming_go/

And, tangently related, visualizing concurrency: https://divan.dev/posts/go_concurrency_visualize/

In my experience it tends not to be done because there's an inevitable drift between the system as it actually operates and the relevant visualizations and diagrams. I have seen them used as a starting off point at times, although that's also been infrequent.

There is however a growing movement of being able to visualize how a system is functioning at various levels. XState/Statecharts are a good example (https://xstate.js.org/viz/). Another example in the Ops space would be https://github.com/spekt8/spekt8 for K8S. I work at Grafana and we're more or less trying to expose these things in ways that make sense. Our bread and butter is timeseries data but we're adding more in that regard (it's possible to build node graphs from running systems).