The article mentions a Xamarin application -- that sounds interesting. I'd like to hear more about how far this approach can be taken in a single-process application. Are they using an existing message bus, or did they build one?
I'm more familiar with the traditional (SmallTalk-style) MVC setup where the data model entities implement the observer pattern. In that case, observers subscribe to change events by attaching themselves to model values (event sources). I guess MOP differs from this by replacing centralised model value objects with some kind of identifiers used to key message bus subscriptions (?).
Xamarin.Forms comes with a basic message passing system built in, which is what I assume the author is using. I've used it myself for a client mobile app project. It makes more sense with an MVVM setup like what Xamarin.Forms and WPF expect for user interfaces, since viewmodels can listen for model state changes that need to be reflected in the UI without requiring loads of explicitly coded delegates/events.
I agree with the author roughly about the benefits and disadvantages. And would probably add to the disadvantages that synchronous sequences are harder to model, since with pure asynchronous messaging all components will get state machines.
Regarding the advantages: Interfaces and dependency injection (with or without frameworks) also solve the first two bullet points also rather nicely. And they keep the synchronous nature and strictly typed interfaces.
The third point can be achieved with data binding or interfaces which allow for subscriptions (e.g. via classical "Event"s or Observables).
In your first statement did you intend to mean that asynchronous was an advantage or disadvantage? I’d take objects being modeled as independent state machines as a significant benefit.
It doesn’t seem that dependency injection really provides the benefits of the second advantage as the author describes it. Specifically he mentions that the recipient can decide wether and/or how to respond to a message. DI doesn’t provide that. The point isn’t _just_ that you can dynamically swap objects, which seems precarious in a lot of ways as values/results in referenced objects could change under your feet which without pervasive usage of observers means lots of easy to make mistakes for devs.
That means for a DI system to really be dynamic in the same sense that you need an event system / observer pattern in addition to DI to provide similar benefits while being more complicated due to now three layers of abstraction (at least) required. Whereas an “MOP" system encourages devs to think about outside changes from the start and makes it the path of least resistance while reusing the same abstraction to enable dynamic target and 1-to-many fanning. The usage of DI systems results in much more ceremony and generally results in overly complicated system architecture as compared to a good messaging system as the author highlights. Now a really complex message topology can become really confusing which results in similar architectural issues as an DI / event based system if not checked.
> MOP is a flavour of object-oriented programming (OOP) with the core idea that your objects shouldn’t directly call each other, but communicate by passing messages via a message bus.
Sounds pretty much like a textbook definition of event-drive programming given by someone oblivious to event driven programming.
The main difference is lack of shared state. Events are not copied necessarily nor immutable while messages are. Holding to that model also makes it easier to interpose a mediator or a buffer. Or serialize them to disk or database.
The benefit is easier multithreading, drawback is memory use (incl. GC churn and bandwidth) and/or complexities of copy on write in multithreaded environment.
The style is also not amenable to passing big amounts of data around directly.
(That is often worked around via explicit memory sharing or a database - and passing handles instead of data.)
Another name for a different flavour of this is reactive programming.
I wonder if its possible/useful to construct a programming paradigm based on the architecture of the internet and TCP packets. Instead of having just one queue with messages, one could have multiple queues , each with its own address (analogous to a url?)
So, there might be a message queue for each object - This seems to be the way that ROS (the robot operating system) is architected when you have nodes that process messages. Still it becomes difficult to disentagle a datum that needs to be accessed and modified by different queues - and avoiding race conditions, etc. But the solution could use message passing itself. If a queue modified a shared datum, it could message all the other queues using that datum and they could then update themselves. Conceivably, each queue could have its own core and so making parallel processing effortless. You would never need to worry about messing up shared data since the data would look after itself. But the 'problem' is that you have traded time and complexity for storage space - each datum now has a 'train' of addresses it has to visit and it has to carry those addresses with itself.
It sounds to me more like the Smalltalk version of OOP in which Objects send messages to each other - and Smalltalk is where the term Object-Oriented Programming originates.
Event-driven programming has Event handlers (not objects) that pick up specific events. Events aren't messages to a specific Object.
I think event driven programming does not require a queue.
What the article states is that multiple sections of the program create messages on a queue, all parts look at the queue and may do something if the messages are addressed to them.
In event-driven programming it is enough to create an event, an event handler, and then pass the event to the handler. Hence, you can define everyhing related to the handling of the events in a single object (event handler).
Although you may also achieve this via implementing a message queue, and multiple event handlers built-in to multiple sections of your code, it is not necessary.
What is the recommended message passing library for mobie {iOS, Android}? In particular, if one is mixing javascript within webView with native code, message passing between the native and js components can be valuable.
I've used EventBus in the past on Android. It always felt a bit hacky and like "cheating", and navigating flows through the app could be tricky, but it kinda worked?
RxJava is the new hotness at the moment though, which kind of replaces EventBus, so I don't think you could convince a team to develop an app using entirely messages. Still, I'd like to try it for a personal project.
RxJava works but debugging is a pain in that system. Unlike an explicit message bugs it is hard to get a real overview in Rx system as the buses are hidden, multiple and wrapped. You get to add an interposer manually too every place of use to fix this... quickly gets unwieldy but can convert your application to MVI model by force.
weird to assume a bus, since normal message-passing in something like Smalltalk doesn't, and neither does MPI (whose programs are obviously more prevalent, at least in terms of cycles consumed...) the bus-iness does make it look more like event programming.
It is not a complete match. A quality of the actor concurrency model is unreliable message delivery. The system described in the post could be done with reliable delivery expectations.
Kind of. The proposal in the link is more pub/sub than Erlang offers by default (you can do this in Erlang, but it's not the typical way). So with the article's presentation they're using a message bus that multiple subscribers can listen to. So you send a message X and subscribers A, B, C each receive it and do the right thing.
A may be a logger, just track that an X was sent and when. Useful for metric gathering and tracking, but can be disabled (it's unneeded from a functional perspective).
B may contain some actual business logic, take X and transform it into a Y and broadcasting this new message. If we need to change how Y is constructed we can come directly here.
C may use X to update values in a database. Isolating it here allows you to easily replace the DB backend. Create a D process which handles the new database, evaluate that it's working correctly (by comparing to the database C is using) and then disable C later.
These 3 processes are all listening to the same message broadcast. in Erlang we'd need one process which received X and rebroadcasts to an arbitrary number of subscribers (again, easily doable, but not the default behavior people go for). In Go you have channels you can pass around, but as I understand once read the data is removed so only the first subscriber gets it. You'd similarly need some "bus" intermediary that took the input from one channel and rebroadcasts on subscriber channels.
Really, they're describing CORBA and similar systems.
My Lua system uses MOP too, though I never heard that term before. And structured pipes. I like it and wrote it for a dataflow bsased event driven system.
The benefits listed in the article can also be realised with simple java style interfaces and dependency injection. That approach also solves all three disadvantages listed.
They're two slightly different tools. In MOP you use a mediator to facilitate message parsing. Anyone can subscribe to, and send messages, usually dynamically. That makes highly complex systems more approachable and distributable. It comes at an indirection cost.
Interfaces and DI are strongly controlled at the composistion root, but it's not easier for highly complex and distributed systems. However the amount of indirection is simpler to reason about and it makes sense for single process systems.
Process A should not care how Process B implements anything, only how it responds to messages in a platform agnostic manner. There is zero guarantee that the stack keeps looking the same, hence brittle, and since you need to know something about how it's implemented to use the stack, it's also leaky. As soon as an exception happens, it should be logged. Not shipped off to a different process. From that log you know something about the state and the messages leading to the exception, and you can debug directly.
If A send "Process this data X" to B, and B fails with an OutOfMemoryException, it should respond "Unable to process X", or perhaps "Insufficient ressources to process X".
OK thanks, makes sense in genuinely decoupled microservice architecture (with messages crossing language and environment boundaries). Monolithic systems (distributed or not) seem not to have the same problems though, in which case it would still seems like a good idea to me, for debugging purposes.
I'm not convinced. If you're trying to debug a process, you should only operate on the messages and state. The messages you can log, and the state should ideally come only from messages, so you truly only need the messages.
One of the strong points about this way of implementing interprocess communication, is that it allows you to have a new and old version of the same process working concurrently, with the new only running and comparing it's output with the old one. Once you're convinced they work the same, you can retire the old process. To fully leverage this, every message should be platform and implementation agnostic. Even if it's a monolithic system on a single server.
Well, all I can say is that I personally often find the full stack interesting when debugging exceptions, and might still try and implement passing stack info in my monolithic system (in spite of this discussion which might have been above my pay-grade honestly). In case it breaches some important CS principle that I don't fully grasp yet then it will surely come back and bite me - which has happened before ... :) . Anyway, thanks for your time.
You're welcome. I juggle those concepts and requirements daily, and I value explaining it to the developers. The "why are we doing it this way" is really important for their job satisfaction, and it happens that they have an aspect I didn't consider, or a better solution.
Your intuition is right. As a programmer your primary job is in most cases not mental masturbation and/or adhering to CS principles but shipping working software.
We are all bound by the 80/20 rule and our job is do decide when to adhere to the principles and when to cut corners.
In practice no one will care that you implemented a 100% pure microservice if you miss an important deadline, your company loses a $300M deal and your team ends up laid off.
I'm not a business person, but a programmer. I also like writing beautiful code but we have to know the tradeoffs. If the tradeoff of having a beautiful service vs a nice service is 3 weeks of additional keyboard bashing it is not worth it.
I once had a colleague who insisted on implementing all REST endpoints between microservices using HATEOAS/HAL because it will enable the consumption of such endpoints by programs and generated code. He spent a month refactoring our services and we ended up never using that feature, but modifying endpoints still carry that overhead he added.
This only makes sense if you work with different languages and/or a plugin architecture. I know that what you say is written in the books but in practice in 95% of the cases you will never write your new ser ice in language X and you will never switch from Hibernate to EclipseLink, etc.
I'm not saying that abstractions are not useful. What I'm saying that over-architecting something on the possibly false premise that "at some time we might need to support X so let's do it by creating an additional abstraction" will waste time, money and will just end up as code clutter which you have to maintain.
This is a typical over-architecting issue you describe.
- All the programs I work on are running on the JVM so I know how the trace looks like and how it is implemented so it is not leaky nor brittle
-Tracing back what caused what is a pain if you don't see the full trace.
- The indirection caused by an eventbus message passing a microservice boundary makes it practically impossible to debug
- passing on the trace will let you know __immediately__ where the problem is
I'm not a purist, I'm a pragmatist. I don't care if a solution is not pure if it works, spares my time, and most importantly: does not come with a burden. This tracing can be put behind a feature toggle, or an aspect/decorator/whatever.
You might call this a hack, but it lets me ship, maintain and support software while others still stand at the whiteboard planning.
The article is written in the context of a Xamarin mobile app, not overly complex or distributed at all. So I'm not sure if the added indirection cost is worth it.
I totally agree that message/event passing is better suited for a complex distributed system
Classic programming generally doesn't impose restrictions about which direction the data flow is.
By default, methods take value from their callers as input, and return values to their callers as output.
An "notification"/"fire-and-forget" interface allowing no data to go back to the caller (no return value, no "output parameter", no visible change to the program state, from the caller's POV) has very interesting characteristics (Like being trivially mockable).
I believe this is what Command Query Responsibility Segregation (CQRS) is all about. By having one object handling writes, and another for reads, things are easier to reason about among other benefits. An event bus is kind of required in this kind of architecture if the view that's reading wants to stay updated, as it may no longer access a return value.
If you think it’s necessary you can still explicitly list receivers on the message handler instead of using a list of objects that implement a certain interface. It makes code a bit more kludgey though.
For me the interesting thing is that the very lack of coupling seems like a problem to me. If A depends on B, I'd actually like B to be coupled to A. Back in the old days when I believed "generic components" were a cool idea, I'd always run into that problem. It's too general and so it has not structure. And then it is actually implicitly coupled (even if it has a network system and a communications protocol in the middle) -- it's just that you've abstracted away the interfaces with a bunch of adaptors. It's like having a plug on your electronic device that can fit any socket, but only one of the many sockets actually works.
I imagine the author has techniques for dealing with that kind of problem, but has unfortunately not shared them. Maybe another time.
Alan Kay, one of the fathers of Object Oriented programming mentioned this :
> I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging"
What he actually talks about is Erlang style programming. Erlang is the most OO language out there according to his definition. Which is funny because Erlang is also a functional programming language.
I don't think this article is very good. The main achievement of message passing is to decouple call stacks, and the reason why you do that is that you get more control about which threads a function is called from. You turn to message passing because you want to get away from callback spaghetti. The article doesn't even mention concurrency.
I like the general idea and have felt the same way for a while. For the benefits they list a message bus isn’t necessary. Even dispatching to multiple subscribers can be straight-forward with a proxy object. Instead you can design your OOP system to behave as though it’s passing messages and get a majority of the benefit along with a straight-forward path to refactor towards a message bus if/when you’re ready.
Honestly, I'm surprised by how these very opinionated and seemingly evidence-lacking posts that propose "new cool solutions" are able to get to the front-page of HN. They spice me with some information "look it'll be easy to test" and immediately slap me in the face without a shred of evidence and without even referencing similar or prior research - at least a comparison to event sourcing seems necessary.
Not the author's fault - he probably just felt the need to share his thoughts on this cool new idea he had.
I just wish HN users would take better care of the implications of clicking the +1 button. You're wanting others to read compelling information you've found.
On the other hand, I don't mind reading a short article on a different way of approaching programming. I agree that I would prefer something a lot more in depth, but if it eventually comes along I won't be surprised about it now.
For me the key here is that it's short, easy to understand, describes an interesting idea (even if I'm sceptical), and avoids trashing other ideas.
I certainly prefer this kind of article to the majority of stories unrelated to tech or business that are very popular on HN.
I appreciate reading it first, then coming here, reading a valid criticism, and then a rebuttal. It's an incredibly fast way to help me separate useful information. :)
I'm not too worried about reading bad programming advice, because i'm confident i will ignore it (this is not arrogance - i ignore all programming advice, good or bad). But there are a lot of impressionable minds who take HN quite seriously, and bad advice risks escaping through them into the real world.
Or you could just use Erlang/Elixir and be done with it. Presented approach to message bus has its own share of problems, like mentioned in the article message-type explosion.
Generally in an Actor/Agent based systems messages are still passed to specific 'entities' rather than using a Pub/Sub bus where 'entities' are only coupled through the messages they send and those they subscribe for. I think the article describes the latter.
Although I've written buses in the past that let the user address messages to different 'entities' and groups, even ad hoc groups defined by features like spatial locality.
I have developed a somewhat similar system for Python / C / C++.
I declare my data structures in YAML. From this I generate code for memory allocation; data validation; serialization and de-serialisation routines.
I also declare the topology of the data flow graph in a separate YAML file, decorating each edge with the data type.
Finally, I have a separate YAML file that specifies which computers/processes each node will run on.
Nodes communicate by passing messages. For nodes in the same process, this is simply shared memory (zero overhead). For nodes in different processes that share a memory bus, this is a shared memory queue. For nodes on different machines, the data goes over the network.
Nodes can be python, C or C++.
Very nice. Very declarative. The bare minimum of duplication and boilerplate. Changing from MIL to SIL to HIL is (mostly) just a configuration change. Logging, measurement and replay is built into the framework, as is support for parameter tuning.
Within a single process, the model of computation is synchronous data flow. Across processes, it is asynchronous data flow, but because we have queues with blocking reads and nonblocking writes, we still keep the ability to replay data through the system in a deterministic manner. (Modulo nodes exceeding their time budget).
It is mostly written in Python: Most of the work is done at 'compile-time' -- reading configuration files and generating code for the runtime. It would not be too difficult to port the runtime, I suspect.
Problem domain: Embedded machine vision / Embedded sensor data processing / Autonomous systems.
Process orchestration: Currently very simple. (Python multiprocessing). May change to something more complex in future.
Message versioning: It depends. My configuration management / deployment model means that most of the time backwards compatibility is not required. (The entire system is deployed as a single unit). When messages are persisted, the data recording system files the messages in a configuration management system that knows the system configuration that generated those messages -- so that when they are replayed back -- e.g. to test a new configuration -- the appropriate reading/de-serialization logic can be selected.
For Python, say I'm building a Docker container in Jenkins for Python app that includes your code. Does the YAML get evaluated when pushing the Jenkins "build now" or when executing "docker run <args>"?
The YAML is evaluated at 'compile time'. I.e. the equivalent of the 'build now' step.
It is worth noting: this is part of a custom code-generation/build/test/parameter-tuning system that I developed for embedded machine-vision/sensor-data-processing systems.
It is not intended to be used by third-parties; or distributed as a library -- it is very much integrated into my scripted workflow.
One important advantage not mentioned is the sending side. You can respond to websocket messages, push notifications or HTTP callbacks using the same message structure so every new source of messages can be added with only one line of code and the receivers can be unaware of a new source.
Win32 is much closer to object orientation with duck typing. You normally need a handle to the window you want to talk to, which is explicitly the kind of coupling that MOP rejects, inverting the dependency.
Further, with Win32, you can choose between an async or synchronous message send, depending on whether you are interested in side effects or not.
Highly dynamic behavior like this is great in very clear scenarios but think carefully before using it for the majority of your program.
Dynamic behaviors are gained at the loss of type safety (function calls are type checked, messages are usually not) and system coherence (just because you can add and remove components on the fly, doesn't mean that it's safe to do so). To compensate for these losses, you need to increase documentation and increase testing. Most people don't and the result is significantly less reliable code.
> Coupling between callers and responders is removed.
But the number of dependencies is the same (reference versus message type). And now it's unregulated dependency without any compiler-level contract between the two. Are the parameters still what you expected? Are the parameters in the correct format? The message type will continue to match, even if these details change.
> Messages can be dispatched to multiple subscribers
Which is great for communication that is intended to be observable but if you're using public message passing for everything, it's going to be tempting to snoop on the message bus and pull data intended for other recipients instead of passing it cleanly. Now your program is a fragile mess because everything is dependent on precise implementation details of unrelated parts of the program.
> Dependencies can be changed on the fly.
Which is another way of saying that you need to handle dependencies changing without notice. What happens if you're expecting a response but the generator of that response is removed between request and sending response? What if you get a timeout? What if you get a partial response?
For the first problem, union types are the clear answer (abstract marker classes can approximate it somewhat).
The compiler guarantees that all sent messages conform to one of a list of possible valid message types. If the subscribers are static, the compiler can also guarantee that every valid message type has at least one handler; if the subscribers are dynamic, you're treading much more dangerous waters but at least you have a comprehensive list of possible messages to verify at runtime that each one has at least one handler.
For the second concern, I agree but I would invert it: if a publisher writes messages intended for a particular recipient, that's a misuse of a message bus. You'd be better off making a regular method call in that case (nobody says the bus should solve every one of your problems). Ditto for the third - if you absolutely need a specific response to your message, don't use a message bus to send it. The challenge of message-bus-based architecture at that point becomes minimizing the number of publishers which need to make direct calls.
Yeah I don't know why the author doesn't call this out as a con. Error cases are much harder to reason about, what if another object blows up halfway through processing a message? How do retries work? Transactions all become a consensus problem or a two phase commit.
95 comments
[ 3.5 ms ] story [ 170 ms ] threadI'm more familiar with the traditional (SmallTalk-style) MVC setup where the data model entities implement the observer pattern. In that case, observers subscribe to change events by attaching themselves to model values (event sources). I guess MOP differs from this by replacing centralised model value objects with some kind of identifiers used to key message bus subscriptions (?).
Related ideas for client/server are:
Event Sourcing https://martinfowler.com/eaaDev/EventSourcing.html
and
CQRS https://martinfowler.com/bliki/CQRS.html
Regarding the advantages: Interfaces and dependency injection (with or without frameworks) also solve the first two bullet points also rather nicely. And they keep the synchronous nature and strictly typed interfaces. The third point can be achieved with data binding or interfaces which allow for subscriptions (e.g. via classical "Event"s or Observables).
It doesn’t seem that dependency injection really provides the benefits of the second advantage as the author describes it. Specifically he mentions that the recipient can decide wether and/or how to respond to a message. DI doesn’t provide that. The point isn’t _just_ that you can dynamically swap objects, which seems precarious in a lot of ways as values/results in referenced objects could change under your feet which without pervasive usage of observers means lots of easy to make mistakes for devs.
That means for a DI system to really be dynamic in the same sense that you need an event system / observer pattern in addition to DI to provide similar benefits while being more complicated due to now three layers of abstraction (at least) required. Whereas an “MOP" system encourages devs to think about outside changes from the start and makes it the path of least resistance while reusing the same abstraction to enable dynamic target and 1-to-many fanning. The usage of DI systems results in much more ceremony and generally results in overly complicated system architecture as compared to a good messaging system as the author highlights. Now a really complex message topology can become really confusing which results in similar architectural issues as an DI / event based system if not checked.
https://en.wikipedia.org/wiki/Event-driven_architecture
> MOP is a flavour of object-oriented programming (OOP) with the core idea that your objects shouldn’t directly call each other, but communicate by passing messages via a message bus.
Sounds pretty much like a textbook definition of event-drive programming given by someone oblivious to event driven programming.
The benefit is easier multithreading, drawback is memory use (incl. GC churn and bandwidth) and/or complexities of copy on write in multithreaded environment. The style is also not amenable to passing big amounts of data around directly. (That is often worked around via explicit memory sharing or a database - and passing handles instead of data.)
Another name for a different flavour of this is reactive programming.
So, there might be a message queue for each object - This seems to be the way that ROS (the robot operating system) is architected when you have nodes that process messages. Still it becomes difficult to disentagle a datum that needs to be accessed and modified by different queues - and avoiding race conditions, etc. But the solution could use message passing itself. If a queue modified a shared datum, it could message all the other queues using that datum and they could then update themselves. Conceivably, each queue could have its own core and so making parallel processing effortless. You would never need to worry about messing up shared data since the data would look after itself. But the 'problem' is that you have traded time and complexity for storage space - each datum now has a 'train' of addresses it has to visit and it has to carry those addresses with itself.
Event-driven programming has Event handlers (not objects) that pick up specific events. Events aren't messages to a specific Object.
The article seems to be describing a particular style of event-driven architecture, used to organize code within a single process.
What the article states is that multiple sections of the program create messages on a queue, all parts look at the queue and may do something if the messages are addressed to them.
In event-driven programming it is enough to create an event, an event handler, and then pass the event to the handler. Hence, you can define everyhing related to the handling of the events in a single object (event handler).
Although you may also achieve this via implementing a message queue, and multiple event handlers built-in to multiple sections of your code, it is not necessary.
https://github.com/greenrobot/EventBus
RxJava is the new hotness at the moment though, which kind of replaces EventBus, so I don't think you could convince a team to develop an app using entirely messages. Still, I'd like to try it for a personal project.
A may be a logger, just track that an X was sent and when. Useful for metric gathering and tracking, but can be disabled (it's unneeded from a functional perspective).
B may contain some actual business logic, take X and transform it into a Y and broadcasting this new message. If we need to change how Y is constructed we can come directly here.
C may use X to update values in a database. Isolating it here allows you to easily replace the DB backend. Create a D process which handles the new database, evaluate that it's working correctly (by comparing to the database C is using) and then disable C later.
These 3 processes are all listening to the same message broadcast. in Erlang we'd need one process which received X and rebroadcasts to an arbitrary number of subscribers (again, easily doable, but not the default behavior people go for). In Go you have channels you can pass around, but as I understand once read the data is removed so only the first subscriber gets it. You'd similarly need some "bus" intermediary that took the input from one channel and rebroadcasts on subscriber channels.
Really, they're describing CORBA and similar systems.
Interfaces and DI are strongly controlled at the composistion root, but it's not easier for highly complex and distributed systems. However the amount of indirection is simpler to reason about and it makes sense for single process systems.
If A send "Process this data X" to B, and B fails with an OutOfMemoryException, it should respond "Unable to process X", or perhaps "Insufficient ressources to process X".
One of the strong points about this way of implementing interprocess communication, is that it allows you to have a new and old version of the same process working concurrently, with the new only running and comparing it's output with the old one. Once you're convinced they work the same, you can retire the old process. To fully leverage this, every message should be platform and implementation agnostic. Even if it's a monolithic system on a single server.
We are all bound by the 80/20 rule and our job is do decide when to adhere to the principles and when to cut corners.
In practice no one will care that you implemented a 100% pure microservice if you miss an important deadline, your company loses a $300M deal and your team ends up laid off.
I'm not a business person, but a programmer. I also like writing beautiful code but we have to know the tradeoffs. If the tradeoff of having a beautiful service vs a nice service is 3 weeks of additional keyboard bashing it is not worth it.
I once had a colleague who insisted on implementing all REST endpoints between microservices using HATEOAS/HAL because it will enable the consumption of such endpoints by programs and generated code. He spent a month refactoring our services and we ended up never using that feature, but modifying endpoints still carry that overhead he added.
I'm not saying that abstractions are not useful. What I'm saying that over-architecting something on the possibly false premise that "at some time we might need to support X so let's do it by creating an additional abstraction" will waste time, money and will just end up as code clutter which you have to maintain.
- All the programs I work on are running on the JVM so I know how the trace looks like and how it is implemented so it is not leaky nor brittle
-Tracing back what caused what is a pain if you don't see the full trace.
- The indirection caused by an eventbus message passing a microservice boundary makes it practically impossible to debug
- passing on the trace will let you know __immediately__ where the problem is
I'm not a purist, I'm a pragmatist. I don't care if a solution is not pure if it works, spares my time, and most importantly: does not come with a burden. This tracing can be put behind a feature toggle, or an aspect/decorator/whatever.
You might call this a hack, but it lets me ship, maintain and support software while others still stand at the whiteboard planning.
I totally agree that message/event passing is better suited for a complex distributed system
Classic programming generally doesn't impose restrictions about which direction the data flow is. By default, methods take value from their callers as input, and return values to their callers as output.
An "notification"/"fire-and-forget" interface allowing no data to go back to the caller (no return value, no "output parameter", no visible change to the program state, from the caller's POV) has very interesting characteristics (Like being trivially mockable).
Source: https://martinfowler.com/bliki/CQRS.html
-- I think this is the key cons which might lead us into nightmare.
I imagine the author has techniques for dealing with that kind of problem, but has unfortunately not shared them. Maybe another time.
> I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging"
http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-...
Not the author's fault - he probably just felt the need to share his thoughts on this cool new idea he had.
I just wish HN users would take better care of the implications of clicking the +1 button. You're wanting others to read compelling information you've found.
For me the key here is that it's short, easy to understand, describes an interesting idea (even if I'm sceptical), and avoids trashing other ideas.
I certainly prefer this kind of article to the majority of stories unrelated to tech or business that are very popular on HN.
I definitely liked the idea of tuple spaces but have never seen it implemented in a production system.
Although I've written buses in the past that let the user address messages to different 'entities' and groups, even ad hoc groups defined by features like spatial locality.
I declare my data structures in YAML. From this I generate code for memory allocation; data validation; serialization and de-serialisation routines.
I also declare the topology of the data flow graph in a separate YAML file, decorating each edge with the data type.
Finally, I have a separate YAML file that specifies which computers/processes each node will run on.
Nodes communicate by passing messages. For nodes in the same process, this is simply shared memory (zero overhead). For nodes in different processes that share a memory bus, this is a shared memory queue. For nodes on different machines, the data goes over the network.
Nodes can be python, C or C++.
Very nice. Very declarative. The bare minimum of duplication and boilerplate. Changing from MIL to SIL to HIL is (mostly) just a configuration change. Logging, measurement and replay is built into the framework, as is support for parameter tuning.
Within a single process, the model of computation is synchronous data flow. Across processes, it is asynchronous data flow, but because we have queues with blocking reads and nonblocking writes, we still keep the ability to replay data through the system in a deterministic manner. (Modulo nodes exceeding their time budget).
I'm building something on similar lines with Redis and Protobuf, but I'm in over my head a little bit
- How do you version your messages?
- What problem domain are you working on?
- What kind of process orchestration system do you use?
Process orchestration: Currently very simple. (Python multiprocessing). May change to something more complex in future.
Message versioning: It depends. My configuration management / deployment model means that most of the time backwards compatibility is not required. (The entire system is deployed as a single unit). When messages are persisted, the data recording system files the messages in a configuration management system that knows the system configuration that generated those messages -- so that when they are replayed back -- e.g. to test a new configuration -- the appropriate reading/de-serialization logic can be selected.
It is worth noting: this is part of a custom code-generation/build/test/parameter-tuning system that I developed for embedded machine-vision/sensor-data-processing systems.
It is not intended to be used by third-parties; or distributed as a library -- it is very much integrated into my scripted workflow.
Further, with Win32, you can choose between an async or synchronous message send, depending on whether you are interested in side effects or not.
Dynamic behaviors are gained at the loss of type safety (function calls are type checked, messages are usually not) and system coherence (just because you can add and remove components on the fly, doesn't mean that it's safe to do so). To compensate for these losses, you need to increase documentation and increase testing. Most people don't and the result is significantly less reliable code.
> Coupling between callers and responders is removed.
But the number of dependencies is the same (reference versus message type). And now it's unregulated dependency without any compiler-level contract between the two. Are the parameters still what you expected? Are the parameters in the correct format? The message type will continue to match, even if these details change.
> Messages can be dispatched to multiple subscribers
Which is great for communication that is intended to be observable but if you're using public message passing for everything, it's going to be tempting to snoop on the message bus and pull data intended for other recipients instead of passing it cleanly. Now your program is a fragile mess because everything is dependent on precise implementation details of unrelated parts of the program.
> Dependencies can be changed on the fly.
Which is another way of saying that you need to handle dependencies changing without notice. What happens if you're expecting a response but the generator of that response is removed between request and sending response? What if you get a timeout? What if you get a partial response?
The compiler guarantees that all sent messages conform to one of a list of possible valid message types. If the subscribers are static, the compiler can also guarantee that every valid message type has at least one handler; if the subscribers are dynamic, you're treading much more dangerous waters but at least you have a comprehensive list of possible messages to verify at runtime that each one has at least one handler.
For the second concern, I agree but I would invert it: if a publisher writes messages intended for a particular recipient, that's a misuse of a message bus. You'd be better off making a regular method call in that case (nobody says the bus should solve every one of your problems). Ditto for the third - if you absolutely need a specific response to your message, don't use a message bus to send it. The challenge of message-bus-based architecture at that point becomes minimizing the number of publishers which need to make direct calls.
Seems you'll need a consensus layer and the ability to rollback if you don't want to end up in a terrible mess later right?