Ask HN: Would you use an architecture diagram tool to save infra costs?
Here’s what the tool will offer:
1. Tag-Based Diagram Creation: Users can create architecture diagrams using the tags assigned to their infrastructure. This allows for easy visualization and organization.
2. Resource Tracking: The tool can track resource wastage and identify machines that are running but have no usage, helping you optimize your infrastructure. Check cost per infra stack.
3. High-Level Overview: Create a comprehensive mesh of diagrams that provide a 10,000-foot overview of all services and their interactions.
4. Infrastructure Management: Ability to delete unused infrastructure directly from the tool, check usage metrics, and upscale/downscale resources as needed.
I’m keen to know:
- Would this tool be useful in your tech operations? - Are there any specific features or improvements you would like to see added to this tool? - Any thoughts on potential challenges or limitations?
30 comments
[ 2.7 ms ] story [ 65.8 ms ] threadThis tool will aim to bridge that gap by providing clear, visual representations of your deployed infrastructure, which can make the complexities of architecture more transparent and easier to understand. By centralizing this information, you can reduce the reliance on a few key individuals and make it easier for everyone to contribute and stay informed. This way, the plan becomes more than just a set of tasks—it becomes a shared understanding.
I cannot read sheet music. I lack such experience. That experience is earned from practice. Likewise architecture is a skill earned through practice. My experience has taught me:
1. It cannot be communicated to the inexperienced developer because they do not understand it. Sometimes they will pretend to understand it and just superimpose a hasty, wrong, interpretation on top of it that just reinforces some singular aspect of comfort.
2. When architecture is defined in terms of goals it is better understood by business owners and project managers than inexperienced developers. They may not understand the code and conventions but they can frame the second and third order consequences in terms of measures, risks, and effort.
3. You can generally identify those capable of architectural discussion by where they invest their interests. When the conversation never rises above syntax and vanity, as in how to write code, the higher level considerations of organization and structure become lost and lofty.
Do you think its possible to simpify architectures for most if not all involved? If so, how?
Simple is not easy. In the context of logic simple just means fewer, or numerically less than something else. Please see this most excellent video by Rich Hickey, the creator of Clojure language: https://www.youtube.com/watch?v=SxdOUGdseq4
The primary problem with simplifying architecture is that simplification segregates a population. For people who are capable of understanding and communicating concepts of architecture simplification dramatically increases expressiveness and reduces labor. For people who have not earned comfort with architecture simplification imposes barriers to understanding. It's the difference between adapting an abstract subject into an essay versus needing to be told what the research material is as well as what words and thoughts are appropriate.
Consider subjects like lexical scope, functions as first class citizens, definitions via interfaces, tree models and taxonomies, and so forth. None of those subjects on their own are challenging, at least for most of us. Most people use these concepts outside of programming everyday without really thinking about it. The challenge then is to get people to think about these concepts intentionally. In most cases people not already comfortable expressing logic in these concepts will only do so if alternatives are removed. This is more challenging than it sounds because it requires separating people from competing institutions they are reliant upon.
In my case, just putting my own bias out there, I am self taught. I learned to program while traveling across Afghanistan, though I had already taught myself XML Schema the year prior. There were many times I did not have internet access so I could not simply access unlimited references or rely upon unlimited dependencies. I just had to figure it out logically and through trial and error. I also did not have prior teaching in the subject so I was a clean slate. The result was to do that which required less memorization and less formal education instead favoring that which was more visual and immediately expressive. Perhaps this is why I find DOM traversal amazing while most people are absolutely horrified by it.
Coming back, simplifying architecture will never be easy. And if that is the case, how can communication of architectures be made simpler? Is it possible to build abstractions and represent them as such to make it simple for most to understand? Are there any tools have come closest to solving this? In a world full of complex micro services.. someone has to come and solve for this problem.
The prior year I was involuntarily reassigned from a web design position to a front-end developer position at Travelocity. That year at Travelocity I was taking my first baby steps into programming, as required to retain employment, but did not really understand what I was doing. I learned use of basic functions and syntax. I learned much more trying to write original programs on my own in my down time in Afghanistan.
I suppose an excellent comparison to software architecture is public speaking or writing for publication. You know talent when you see it and it looks so incredibly simple when performed. The real challenge is attempting to communicate that simplicity in a meaningful way because not everybody is willing to consume the challenging material that allows for such talent. I don't claim to be talented, but I know what I have performed relative to other people. There are some people I can explain this to with immediate consumption. There are some people who are capable of listening to it, but find the effort required untenable. Then there are people who are not capable of consuming the material in any form.
But it'll be very hard to get it past our architect. Maybe if it's open source so we can test and prove the value before using a 3rd party.
You get the idea of what I am trying to build, if you were to do this for your org, how would you solve the problem? What will you do?
We are in the process of developing a freeware Excel plugin for the PlantUML component. This plugin will enable users to generate diagrams directly from spreadsheet templates. If you’re interested in trying out the beta version of this plugin, please send me an email and I’ll be happy to share it with you.
A couple of questions:
- Tag-based diagram creation: what information would you put into a tag and how are tags managed (manually? automatically assigned in some way?)
- Resource tracking: what is a resource? are they machines / nodes? or do you plan to track database/cache/queues/etc? How do you define waste? We used the simple definition that a machine with no tag, though that is relatively simple and useful at the beginning when you are mapping out the infra you do not know, but might not be great on an ongoing basis.
- High-level overview: how do you know if serviceA is interacting with serviceB? Many cases it would also be great to know if services are interacting with 3rd parties and that is also a big part of the infrastructure. Knowing that serviceA and serviceB is interacting is only the first step, though knowing why they are interacting (is it a critical part of serviceA? what is the business value of that connection? how is the customer affected if it gets broken?) and why was it implemented in a given way? (sync/async, retries and timeouts, what matters to the customer). Communicating this type of context of the architecture seems hard and auto-generated diagrams usually fail to do.
- Infrastructure management: I am not that averse about giving some delete rights for the infrastructure, maybe you can get partial rights, though I assume a lot of companies are already using something for upscale/downscale and might have strict processes for deleting things. What if a delete needs an approval or code review like step?
I’m targeting startups/corporates that are over 5 years old but haven’t become large corporates. Employee count from 50-500. At this stage, companies often lose track of their deployed infrastructure, leading to significant costs. Typically, around 10% of the infrastructure is underutilized or unused. And beyond this stage, things are either sorted or way too messy. Additionally, documentation struggles are common, with only a few individuals aware of the full architecture. Democratizing this knowledge can be incredibly beneficial.
To your questions:
1 - The assummption is you already have some tags for your infra. Which can include can include service names, environments, resource types, ownership. And you start from there.
2- Resource could be machines, databases, load balancers, even object storage. If you run unmanaged datastores, you'd have an option to identify that infra. You are right, to be useful on an ongoing basis, we'd regularly have to churn up suggestions based on underutilization.
3- I am not sure this can even be done automatically, but plan is to provide tools to easily document all intereractions and business logic. And depend on service owners to create this mesh.
4 - This has been a constant feedback. Yes something like an approval workflow should do, or may be skip this altogether.