Ask HN: How do you keep track of the components you develop and run?
When the number of these components grows rapidly, it can become increasingly annoying having to search for these components, navigate between their source code, documentation, CI setup and history, recent log entries, deployment information (number of instances, endpoints), metrics, monitoring dashboards, etc.
I know Spotify tries to approach this with an in-house tool called System-Z (see https://dzone.com/articles/modeling-microservices-at-spotify-with-petter-mari). Have you seen any other interesting tools in this space?
Do you think it is beneficial to have tools that make this easier and show all the necessary information in one place (via UI or an API), possibly along with visualizing application dependencies? Or do you believe this is basically a non-issue, given how lots of organizations are moving into the cloud, where vendors can provide a lot of these tools for you?
5 comments
[ 3.6 ms ] story [ 26.8 ms ] threadHaving a tool that visualises all dependencies in one place is nice to have but it shouldn't be essential to run a system.
If you're running a larger number of services service discovery and properly naming services is vital rather than being able to visualise them.
I named my common library "based":
- Uses reStructuredText + docutils to display content in a consistent fashion (https://devel.tech/features/django-vs-flask/, https://www.hskflashcards.com/downloads)
- Common fabric source code for deploying Django websites, updating the sites
- Customized crispy-form templates and SCSS files
- Using Yarn "Workspaces" to load the common JS/SAAS code (https://yarnpkg.com/lang/en/docs/workspaces/)
- In Webpack, I share common webpack code via wrapping the configs in a function and passing in dependencies.
- Common Python code is available to be accessed for reusable Django applications, but also plain-old python
- And use of "import_string" in django settings to load additional plugin configurations [1]
[1] https://docs.djangoproject.com/en/1.11/ref/utils/#django.uti...
P.S. I am writing an article (in draft atm) about import_strings in Django at https://devel.tech/tips/n/djms3tTe/demystifying-djangos-impo..., intended for people new to Python and Django. You can also use import_string in your own Django code to 'opt-in' to any sort of feature you keep in a common codebase. In my circumstance, I use it as a way to register docutils roles and directives via django settings. This pattern also holds true without Django, too (https://docs.python.org/3/library/importlib.html).
P.P.S. I highly recommend Yarn Workspaces. It's fast, and really makes it a lot easier to share front-end code.
We use consul UI to show all the services that we have, where they're running, number of instances and so on.
We use ELK for service specific metrics / centralized logs.
We use Confluence WIKI for user / dev docs. Plus the README on the project git.
Also, we have a Kibana dashboard which does have information from different sources and it easily tell us if everything is good or not.