This article resonated with me. We are going through a large platform re-launch, and using Kafka and events as a big part of the architecture. After much discussion, we decided against storing events, eventual consistency, or CQRS and so on. Instead, we use events to simply de-couple services. We have strong domain APIs and workers that respond to creates/updates/deletes. We use Kafka to partition things where it makes sense. Just stop there, and there's a ton of benefit. Things get simpler to reason about, easier to test, and result in less dependencies between teams.
We are going through a similar exercise but we avoided the approach of event driven CRUD and stuck to capturing customer's intent and an ability to read its side-effects as our de-facto API.
Consider for instance an API to create and account. Now let's say we want ability to update address. One approach would be an API that changes the address by updating the account but the customer never asked to update the account. The customer wants to change the address. How about we give them the API with change of address request/semantics? This way, no matter what happens, the API and the events remain in sync and you don't expose internal data storage semantics to customers that have no interest in them.
The claims here are factually accurate and helpful for people new to using event sourcing as a pattern. The pattern does lend itself very neatly to command query responsibility segregation with eventually consistent reads (except when reading just the last or all events). In addition, while the pattern itself wouldn't qualify as a systems or platform architecture, if the API for the system is designed to capture customer intent on the write side and read its side-effects on the read side, you can see how it all comes together in a neat architecture that not only allows the ability to reason about the system, it also mirrors and meets the needs of its callers/customers automatically because the API is simply a capture of customer's intent. And because the customer's intent (commands) and its outcomes (events) are immutable, you also get automatic change management even if the code continues to evolve and new features added.
The core idea of event sourcing is that whenever we make a change to the state of a system, we record that state change as an event, and we can confidently rebuild the system state by reprocessing the events at any time in the future. The event store becomes the principal source of truth, and the system state is purely derived from it.
I think this can be easily misunderstood to describe an unnecessarily strict view of how event sourcing can work. To use the most common example, a bidding system, someone might see the state as the record of accepted offers, bids, and sales, and the events as recording changes to that state. That would be one way to design the system. However, that means events can only be issued after state changes are decided. That means after synchronization and business logic have been applied. Suppose a user submits a bid that is rejected by the system because it does not meet the rules, comes after bidding is closed, or because the system rejecting the bid does not recognize the item id. That bid does not get recorded as an event and cannot be replayed, so if it was rejected in error, the erroneous action is not recorded and can never be reconstructed. This kind of event-based system amounts to little more than capturing a database commit log. Actions taken by users and peripheral parts of the system are obscured, leaving a very subjective account of what happened.
If you enlarge the concept of state to include more of the distributed state of the application, then this description can yield a larger and potentially more useful set of events. The user submitted a bid — that's an event. The system rejected the bid — that's an event. The system closed bidding — that's an event. Then you can replay bugs. Was the bid incorrectly rejected as invalid? Did the system close bidding prematurely? If I fix a bug in a system and replay events to it, does it do the same thing or something different? You can also discover performance problems. If a bid event is relayed through the system before it is processed, you might see that the difference in timestamps between the bid being placed and the bid being rejected is higher than expected.
A simple example of this trap is when an event is used as a passive-aggressive command. This happens when the source system expects the recipient to carry out an action, and ought to use a command message to show that intention, but styles the message as an event instead.
I think this is another part that can be misunderstood, or perhaps I disagree with it altogether.
Commands can be significant events that are helpful for event sourcing. Ask yourself, if you exclude them from the event log, are you still able to reconstruct what happened to the degree necessary for debugging? Capturing commands as events might allow replay with only a subset of the system running, which can be helpful. Also, computers are more precise than humans. If I tell my girlfriend, "There is someone at the front door," there may or may not be an implied request. There is no reason to design computer systems with that kind of ambiguity, so there is no need to create a separate category of "commands." "System A has scheduled task 23 to execute next on executor Z" is an event that we can program executor Z to respond to in a certain way. The behavior can be specified and tested.
Commands have their own problems. I have seen too many systems that report spurious errors because a programmer did not want to report "success" unless a "command" was carried out. "Command" evokes a human relationship that applies to a small subset of interactions between computer systems. When you change the shipping address on an Amazon order, your browser does not say, "Change the shipping address on the damned order!" It says, "The buyer requested to change the shipping address on the order." Now, that kind of message is sometimes modeled as a "co...
9 comments
[ 3.2 ms ] story [ 25.4 ms ] threadConsider for instance an API to create and account. Now let's say we want ability to update address. One approach would be an API that changes the address by updating the account but the customer never asked to update the account. The customer wants to change the address. How about we give them the API with change of address request/semantics? This way, no matter what happens, the API and the events remain in sync and you don't expose internal data storage semantics to customers that have no interest in them.
I think this can be easily misunderstood to describe an unnecessarily strict view of how event sourcing can work. To use the most common example, a bidding system, someone might see the state as the record of accepted offers, bids, and sales, and the events as recording changes to that state. That would be one way to design the system. However, that means events can only be issued after state changes are decided. That means after synchronization and business logic have been applied. Suppose a user submits a bid that is rejected by the system because it does not meet the rules, comes after bidding is closed, or because the system rejecting the bid does not recognize the item id. That bid does not get recorded as an event and cannot be replayed, so if it was rejected in error, the erroneous action is not recorded and can never be reconstructed. This kind of event-based system amounts to little more than capturing a database commit log. Actions taken by users and peripheral parts of the system are obscured, leaving a very subjective account of what happened.
If you enlarge the concept of state to include more of the distributed state of the application, then this description can yield a larger and potentially more useful set of events. The user submitted a bid — that's an event. The system rejected the bid — that's an event. The system closed bidding — that's an event. Then you can replay bugs. Was the bid incorrectly rejected as invalid? Did the system close bidding prematurely? If I fix a bug in a system and replay events to it, does it do the same thing or something different? You can also discover performance problems. If a bid event is relayed through the system before it is processed, you might see that the difference in timestamps between the bid being placed and the bid being rejected is higher than expected.
A simple example of this trap is when an event is used as a passive-aggressive command. This happens when the source system expects the recipient to carry out an action, and ought to use a command message to show that intention, but styles the message as an event instead.
I think this is another part that can be misunderstood, or perhaps I disagree with it altogether. Commands can be significant events that are helpful for event sourcing. Ask yourself, if you exclude them from the event log, are you still able to reconstruct what happened to the degree necessary for debugging? Capturing commands as events might allow replay with only a subset of the system running, which can be helpful. Also, computers are more precise than humans. If I tell my girlfriend, "There is someone at the front door," there may or may not be an implied request. There is no reason to design computer systems with that kind of ambiguity, so there is no need to create a separate category of "commands." "System A has scheduled task 23 to execute next on executor Z" is an event that we can program executor Z to respond to in a certain way. The behavior can be specified and tested.
Commands have their own problems. I have seen too many systems that report spurious errors because a programmer did not want to report "success" unless a "command" was carried out. "Command" evokes a human relationship that applies to a small subset of interactions between computer systems. When you change the shipping address on an Amazon order, your browser does not say, "Change the shipping address on the damned order!" It says, "The buyer requested to change the shipping address on the order." Now, that kind of message is sometimes modeled as a "co...