"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."
I'd say that organizations are not just constrained to do that -- they should. When you design systems and infrastructure, keep in mind that systems and solutions are under evolutionary pressure, and engineers have to change it all the time. When optimizing for change, you need to optimize for teams and interactions.
If the whole team is sitting in the same room, then there's nothing wrong with a monolithic app and shouting to others that 'hey, does anyone have a problem if I release the latest master branch?'; or it's also fine to have a QA server that the team is using. At a remote company, it's much more important to be able to test subsystems in isolation or even without internet access, automated integration tests are more useful and important.
Probably you're just oversimplifying for brevity or something. But I've worked at plenty of projects where the team was sitting together, the product was a monolith, and it was still a problem. Coordination becomes easier, sitting together, but the need to coordinate, or the level with which people start "stepping on each other's toes" (code-wise) isn't reduced by it. I'm not saying all monoliths are bad, or that you would never want one, but when it becomes a problem, there's little consolation in sitting together.
Of course I'm not trying to provide the ultimate recipe to big IR problems on HN in 10 lines of text written on my phone :)
Just wanted to say that communication patterns and specifically communication overhead can and should be reflected in modularization of software. If two people can talk to each other very easily (because they're sitting near each other) then maybe (!) separating their code through e.g. APIs doesn't need to be that strict compared to teams who never talk to each other in person. These are not hard rules, but good guidelines.
(Modularization isn't for free, so there's always a balance.)
Conway’s law doesn’t directly claim anything about the quality or effectiveness of the resulting system, but I suspect a corollary would also apply: that the effectiveness of a system is limited by the effectiveness of the communication structure of the organization that develops it.
Conway's law doesn't make those claims, but this has been tested by researchers and industry giants (a Microsoft team published a paper on this a part of a Vista retrospective) and those papers do make such claims.
What if the communication structures are poorly designed and you are building a system on top of an organisation that is struggling to communicate effectively?
The project leadership might hope that building a new system will be able change the communication problem. This suggests that it will only compound the problem.
If the company knows that communication structures are poorly designed, the first priority should be to fix those. Fixing those first will probably have a higher impact on productivity.
I think the flaw here is that what works today may not work tomorrow and in many situations, it is easy to see where those breaking points will arise.
Take your team sitting in one room building a monolith. Unless the software is intended to always be smaller and targeted, or the software fails in the market, that team will eventually scale and no longer fit in this room, and now you've got a monolithic application that can't be effectively maintained and enhanced by the organization.
There is value in designing your system for the organization that will own it, but there is also a value in designing your organization for the system you wish to develop.
Experience and published research have shown time and time again that whey you try to fight Conway's law, you will lose, so one or the other will have to give. For long-lived applications, you'll find more success if you can alter the organizational structure to fit the system.
Conway's Law validates a decision to move to a service-oriented and/or microservice architecture. Each service ~= a business unit or department.
Even better is that such an architecture requires (or at least greatly benefits from) specifying precisely the communication between the interacting parts in the form of interfaces.
IIRC there is a study that Steve McConnell mentions in one of his books (sorry, I can't find the link) that shows that the number of bugs introduced into system is highly correlated to the number of lines of communication between the people or teams that write it. Add another team and that number potentially rises exponentially. Conway's Law is the scaffold on which all these understandings hang.
> Conway's Law validates a decision to move to a service-oriented and/or microservice architecture. Each service ~= a business unit or department.
I wouldn't say Conway's Law validates a decision to move to an SOA, so much as further enshrines it in code. There is something to be said for forcing different business units or departments to work together.
Mongols figured out around year 1200 that in a proper military organization, you have one boss and maximum of about 10 subordinates and 10 peers. Though it was already written about by Sun Tzu way earlier. Chain of command ought to be clear from private to general. Romans had earlier found out that a group of about 100 men is good department size for day to day operations. (Roughly the Dunbar number of the least capable members of the group.)
Then commercial companies noticed that they don't have to abide to military hierarchy. But they didn't notice the underlying biological tendencies. As a result every second industry fad is reinventing the same social structure over and over again.
Factory with departments -> open plan office -> teams -> matrix organization -> sprints -> (I lost track here) -> microservice?
Every company has problems. If the biggest problem is poor flow of information between departments/teams, then that company has competitive advantage to the guys who have some serious issues.
Not necessarily. It depends how you define "service". There are two interpretations of Conway's Law: 1) orgs "just" end up making software partitioned like the org structure due to human nature, or 2) orgs should partition software like the org structure.
My experience heavily leans to #2. If your software doesn't "fit" the org, friction happens. For example, if you try to share a common service but there is nobody with sufficient power and staff to manage and coordinate changes in that service, then it's not clear who to blame when bleep happens.
A technician can make a shared service, but then the technician is left holding the responsibility bag when there are conflicts or complaints among the shared service users. Managers don't want to let "lowly" techies control such decisions. Almost nobody is used to that.
Either create a formal org sub-structure with a sufficiently ranked manager(s) to coordinate the shared service, or split it and live with redundancy.
I am aware of this theoretical law but never really considered it. I suppose this could explain why I encountered such push back against a distributed SOA architecture on commodity hardware at a large organization that was built around centralized systems using mainframes.
Ultimately after three years of justifying and re-justifying the decisions I made for which they explicitly hired me for I moved on. It was abundantly clear that though they brought me in to implement SOA and lower physical costs, the culture just did not support that decision. What struck me as odd was that fact that those who hired me to do it did not seem to support it either.
I should note that Conway's law seems like a reasonable assumption as to why due to the monolithic and centralized nature of communication there. It was considered "taboo" to walk in to an executive managers office to speak to them. The expectation was to follow this ridiculously verbose chain of command whereby response to even a simple question could take days or even weeks.
I can't get a clear meaning of "SOA". Data and processes have been shared for many decades. Stored procedures are a common example of such before the Web era. Shoving stuff into JSON doesn't necessarily make it better, and often creates more maintenance work. Do things for a clear reason, not because it's in style.
depends on how they talk on slack and who talks to who.
The company I'm working with at the moment has separate slack per team with each time being ~5 persons and then some people from other team on each slack who pass message.... the communication is not great and the collaboration between services isn't either
The medium likely doesn't matter nearly as much as all the irrational human tribal boundaries. Trying to fix those with technological solutions is kind of the fundamental error.
I feel old when these kind of Wikipedia pages make it to the front page and I was assuming that it was known and obvious for everyone. </grand parent mode off>
Yes. When speaking and writing, I constantly struggle with the fact that a lot of things that are eye-rollingly last year's or even last decade's news for some are eye-opening new concepts for others. It's easy to assume that what's almost cliche in your specific circle is as well known everywhere.
I’ve found that going into the speaking or writing engagement, that assuming at least one person will encounter the info for the first time alleviates any worry I have about others finding it banal.
When I was younger I thought of the course of knowledge as steadily advancing. Because that was my experience as a beginner. Now I think of generations of knowledge, each a circle in a venn diagram, messily overlapping one another. Not an astonishing insight, but it helps me prepare for mentoring some of my younger colleagues.
It's maybe interesting to consider Conway's Law in respect of the systems designed by IETF Working Groups (as distinct from the wider scope of systems that have an RFC but were designed wholly outside the IETF or brought to the IETF largely finished for just a bit of spit and polish)
Whether the IETF is an organisation per se is a fun question in itself of course since it has no membership. It is clearly an activity, but so is break-dancing or reading poetry and we do not suppose that people who read poetry are an "organisation". But if it /is/ an organization, then is that organisation reflected in the things it designs?
I think this may be one of the reasons for IETF's success. Compare with, for example, 3GPP standards. The fact that each WG does a single task and is disabanded early means, according to Conway's law, that individual RFCs will be relatively independent.
This also applies to performance. Nobody cares that the system is nightmarishly slow in bureaucratic corporations where even the simplest software projects take 6 months minimum.
If it is one thing that I have learned at a large software company it is the fact that Conway's Law is fundamentally true. Any software system is a mirror of the team setup and the interactions between the involved people. And many bugs are found in the interfaces/interactions between components of isolated teams.
However, this also means that as a software developer you are unable, or at least very constrained, in the ability to improve or refactor any complex legacy system. You are basically fighting uphill battles in such situations, because you are not in the authority to change/improve the organizational structure.
This also means that as programmers we may need to take more time and care to invest in human relationships and in our communication skills, as this could have an significant effect on code and product quality.
I often see this in the company webpages. Does a college organize for new students? By division? Department? You can tell a lot about the college culture based on how the website is organized!
Have anyone deep dived into this ? I'm interested in learning more. What is the most common communication structures and what designs does those correspond to ?
I always suspected this could be applied to civilization and the human mind as well - that the structure of society has some similarities to the structure of our brains.
52 comments
[ 2.8 ms ] story [ 126 ms ] threadI'd say that organizations are not just constrained to do that -- they should. When you design systems and infrastructure, keep in mind that systems and solutions are under evolutionary pressure, and engineers have to change it all the time. When optimizing for change, you need to optimize for teams and interactions.
If the whole team is sitting in the same room, then there's nothing wrong with a monolithic app and shouting to others that 'hey, does anyone have a problem if I release the latest master branch?'; or it's also fine to have a QA server that the team is using. At a remote company, it's much more important to be able to test subsystems in isolation or even without internet access, automated integration tests are more useful and important.
And so on.
Just wanted to say that communication patterns and specifically communication overhead can and should be reflected in modularization of software. If two people can talk to each other very easily (because they're sitting near each other) then maybe (!) separating their code through e.g. APIs doesn't need to be that strict compared to teams who never talk to each other in person. These are not hard rules, but good guidelines. (Modularization isn't for free, so there's always a balance.)
The project leadership might hope that building a new system will be able change the communication problem. This suggests that it will only compound the problem.
Take your team sitting in one room building a monolith. Unless the software is intended to always be smaller and targeted, or the software fails in the market, that team will eventually scale and no longer fit in this room, and now you've got a monolithic application that can't be effectively maintained and enhanced by the organization.
There is value in designing your system for the organization that will own it, but there is also a value in designing your organization for the system you wish to develop.
Experience and published research have shown time and time again that whey you try to fight Conway's law, you will lose, so one or the other will have to give. For long-lived applications, you'll find more success if you can alter the organizational structure to fit the system.
Even better is that such an architecture requires (or at least greatly benefits from) specifying precisely the communication between the interacting parts in the form of interfaces.
IIRC there is a study that Steve McConnell mentions in one of his books (sorry, I can't find the link) that shows that the number of bugs introduced into system is highly correlated to the number of lines of communication between the people or teams that write it. Add another team and that number potentially rises exponentially. Conway's Law is the scaffold on which all these understandings hang.
I wouldn't say Conway's Law validates a decision to move to an SOA, so much as further enshrines it in code. There is something to be said for forcing different business units or departments to work together.
Organising the business units around the desired deployables is also know as "the Inverse Conway Maneuver"
https://www.thoughtworks.com/radar/techniques/inverse-conway...
http://betica.com/blog/2016/06/17/transform-your-organizatio...
Mongols figured out around year 1200 that in a proper military organization, you have one boss and maximum of about 10 subordinates and 10 peers. Though it was already written about by Sun Tzu way earlier. Chain of command ought to be clear from private to general. Romans had earlier found out that a group of about 100 men is good department size for day to day operations. (Roughly the Dunbar number of the least capable members of the group.)
Then commercial companies noticed that they don't have to abide to military hierarchy. But they didn't notice the underlying biological tendencies. As a result every second industry fad is reinventing the same social structure over and over again.
Factory with departments -> open plan office -> teams -> matrix organization -> sprints -> (I lost track here) -> microservice?
Every company has problems. If the biggest problem is poor flow of information between departments/teams, then that company has competitive advantage to the guys who have some serious issues.
My experience heavily leans to #2. If your software doesn't "fit" the org, friction happens. For example, if you try to share a common service but there is nobody with sufficient power and staff to manage and coordinate changes in that service, then it's not clear who to blame when bleep happens.
A technician can make a shared service, but then the technician is left holding the responsibility bag when there are conflicts or complaints among the shared service users. Managers don't want to let "lowly" techies control such decisions. Almost nobody is used to that.
Either create a formal org sub-structure with a sufficiently ranked manager(s) to coordinate the shared service, or split it and live with redundancy.
Ultimately after three years of justifying and re-justifying the decisions I made for which they explicitly hired me for I moved on. It was abundantly clear that though they brought me in to implement SOA and lower physical costs, the culture just did not support that decision. What struck me as odd was that fact that those who hired me to do it did not seem to support it either.
I should note that Conway's law seems like a reasonable assumption as to why due to the monolithic and centralized nature of communication there. It was considered "taboo" to walk in to an executive managers office to speak to them. The expectation was to follow this ridiculously verbose chain of command whereby response to even a simple question could take days or even weeks.
The company I'm working with at the moment has separate slack per team with each time being ~5 persons and then some people from other team on each slack who pass message.... the communication is not great and the collaboration between services isn't either
They also have mails, Skype, Yammer, Ms Team, and a few different conferencing system on top of that.
it's a mess. For example nearly each team have their own Bitbuck, gitlab instance, github org , gitlab org or whatever VSTS use
This should be pretty obvious. You build all systems around a single messagebroker and each service/unit uses a chrome browser to connect to it.
[1]https://xkcd.com/1053/
New people are born every day!
Whether the IETF is an organisation per se is a fun question in itself of course since it has no membership. It is clearly an activity, but so is break-dancing or reading poetry and we do not suppose that people who read poetry are an "organisation". But if it /is/ an organization, then is that organisation reflected in the things it designs?
However, this also means that as a software developer you are unable, or at least very constrained, in the ability to improve or refactor any complex legacy system. You are basically fighting uphill battles in such situations, because you are not in the authority to change/improve the organizational structure.
This also means that as programmers we may need to take more time and care to invest in human relationships and in our communication skills, as this could have an significant effect on code and product quality.