I think that this is more than a bit of a misunderstanding and a generalization. Most of the Apache projects are designed to fix large-scale problems in a scalable way. For instance, you would have no good use for Mesos for a personal pet project. At the scale that many of them are designed for, configuration is the norm and the Apache projects save hassle compared to writing a custom solution or trying to hack other tools together.
With the Apache projects that I've worked with, there also seems to be an underlying concept of modular components and generalized interfaces, so that they can be used in as many cases as possible. For some people, Mesos (with Marathon) competes with Kubernetes, but there's more to Mesos than just container orchestration.
If you're looking for low-configuration, low-maintenance solutions to common problems, Apache projects probably aren't what you're looking for, in general. On the other hand, if you have big problems to solve, they might be just what you need.
I cannot stand Apache, not the foundation, the web server. We are stuck with it at work, and I groan when I have to venture into an httpd.conf. XML is so clunky and verbose, I have no idea why it caught on as the defacto config language for so many “enterprise” projects other than there weren’t any better choices at the time. Which, that’s fine, XML sucks, but so does YAML (though not nearly as much). Aside from that, which for me at least if I see a project that uses XML I immediately start looking for alternatives, in my experience Apache’s documentation sucks. It feels archaic to get through, not a lot of useful examples, just mostly things like “here how to pass in the default values”. The default Apache logs suck. Reverse Proxying requires way more XML config to setup than it should. I could go on and on about my disdain for Apache/httpd.
When we have options like Nginx, Caddy, or even Traefik, which mostly work out of the box, things like reverse proxying take a line or two of config, and they have good to really good docs, I have no idea why anyone would pick Apache over any of those unless they are working in an environment that is soo entrenched in enterprise software, like Apache, switching isn’t an option.
Because of httpd, I’ve mostly avoided Apache projects. So, I don’t have too much hands on experience for much else to compare, only a lot of (I’m sure biased/frustrated) reading of others experiences.
However, I did decide to give OpenWhisk recently. If I hadn’t already known, I would’nt have believed you if you told me it was an Apache project. I haven’t yet had to look at any XML, minimal time spent on config, it supports modern platforms/techniques out of the box, and it has a CLI that doesn’t suck. Pretty much the opposite of my experiences with httpd. I assume all of this is due to it being a new project without 20+ years of built up enterprise cruft.
So, at least for me, I’ll be avoiding any older Apache projects if I’m able to find a comparable modern one. But, just because there old enterprise stuff sucks to deal with for the average not-enterprise person, that doesn’t mean there new stuff will too, and I’ll be totally willing to give it a shot. One thing Apache has proven is even though the user experience may kinda suck, the underlying tech is usually pretty fucking solid, open source, and you’ll likely not have to worry about breaking changes every month (or decade).
Many Apache projects are written in Java. Java seems to have a culture of overengineering.
Many Apache projects have originated in corporate environments.
I have a theory of how the incentive structure in such an environment is pushing towards overengineering in many parts of a software projects, as opposed to an open-source project which is usually much more cohesive as a result of the stewardship of a single person. When a larger team is in charge of developing a project, there's an incentive on each member of the team to signal ability, competence, and their contribution. As such they will try to make use of their piece of the project/their responsibilities as a means of presenting their knowledge, by sprinkling/making use of various techniques which in most cases are not needed for the project at hand => overengineering. To read more about this effect in other fields as well, there's a recent book by Robin Hanson, you might want to check that out.
Also, keep in mind, that even in a corporate environment, it is not uncommon to have a project started by a single person, which usually leads to a more coherent and less over-engineered design.
Also, beware that the above is a mere anecdote, armchair philosophising of mind, which has not been tested at all. I still expect it to be roughly true though.
So that's why indeed some of Apache's projects seem overly complex, IMVHO.
5 comments
[ 2.8 ms ] story [ 22.5 ms ] threadWith the Apache projects that I've worked with, there also seems to be an underlying concept of modular components and generalized interfaces, so that they can be used in as many cases as possible. For some people, Mesos (with Marathon) competes with Kubernetes, but there's more to Mesos than just container orchestration.
If you're looking for low-configuration, low-maintenance solutions to common problems, Apache projects probably aren't what you're looking for, in general. On the other hand, if you have big problems to solve, they might be just what you need.
When we have options like Nginx, Caddy, or even Traefik, which mostly work out of the box, things like reverse proxying take a line or two of config, and they have good to really good docs, I have no idea why anyone would pick Apache over any of those unless they are working in an environment that is soo entrenched in enterprise software, like Apache, switching isn’t an option.
Because of httpd, I’ve mostly avoided Apache projects. So, I don’t have too much hands on experience for much else to compare, only a lot of (I’m sure biased/frustrated) reading of others experiences.
However, I did decide to give OpenWhisk recently. If I hadn’t already known, I would’nt have believed you if you told me it was an Apache project. I haven’t yet had to look at any XML, minimal time spent on config, it supports modern platforms/techniques out of the box, and it has a CLI that doesn’t suck. Pretty much the opposite of my experiences with httpd. I assume all of this is due to it being a new project without 20+ years of built up enterprise cruft.
So, at least for me, I’ll be avoiding any older Apache projects if I’m able to find a comparable modern one. But, just because there old enterprise stuff sucks to deal with for the average not-enterprise person, that doesn’t mean there new stuff will too, and I’ll be totally willing to give it a shot. One thing Apache has proven is even though the user experience may kinda suck, the underlying tech is usually pretty fucking solid, open source, and you’ll likely not have to worry about breaking changes every month (or decade).
Many Apache projects have originated in corporate environments.
I have a theory of how the incentive structure in such an environment is pushing towards overengineering in many parts of a software projects, as opposed to an open-source project which is usually much more cohesive as a result of the stewardship of a single person. When a larger team is in charge of developing a project, there's an incentive on each member of the team to signal ability, competence, and their contribution. As such they will try to make use of their piece of the project/their responsibilities as a means of presenting their knowledge, by sprinkling/making use of various techniques which in most cases are not needed for the project at hand => overengineering. To read more about this effect in other fields as well, there's a recent book by Robin Hanson, you might want to check that out.
Also, keep in mind, that even in a corporate environment, it is not uncommon to have a project started by a single person, which usually leads to a more coherent and less over-engineered design.
Also, beware that the above is a mere anecdote, armchair philosophising of mind, which has not been tested at all. I still expect it to be roughly true though.
So that's why indeed some of Apache's projects seem overly complex, IMVHO.