GP asked how this system deals with Kafka's at least once semantics. Pointing out that Kafka has at least once semantics is redundant.
What GP was asking: what happens if Kafka delivers the "item added" event twice, but the "item deleted" event only once? Do you end up with an invalid/unrecoverable state?
In this particular example, both events (item_reported and item_deleted) end up in destructive actions identified by unique id. That idempotency makes safe working with Kafka's at-least-once semantics. You can see that it's safe to execute several times this line of code:
val newItemsToBeDeleted = items.filterNot(_ == item)
Awesome, I'll definitely have a play with the :ssh module then, even if only from the Elixir side of the fence.
Just been to ElixirConf here in Europe and was super impressed with Erlang Solutions guys and the general quality of the Erlang and Elixir people. Lots of interesting chats!
8 comments
[ 3.3 ms ] story [ 34.4 ms ] threadhttps://kafka.apache.org/08/design.html
"So effectively Kafka guarantees at-least-once delivery by default ...."
Edit: Downvote if you want. I was trying to help you.
What GP was asking: what happens if Kafka delivers the "item added" event twice, but the "item deleted" event only once? Do you end up with an invalid/unrecoverable state?
In this particular example, both events (item_reported and item_deleted) end up in destructive actions identified by unique id. That idempotency makes safe working with Kafka's at-least-once semantics. You can see that it's safe to execute several times this line of code:
I also liked that Ericsson engaged with the community and considered input and updated code accordingly as they worked on it.
You can see the whole process and discussion in:
https://github.com/erlang/otp/pull/960
Just been to ElixirConf here in Europe and was super impressed with Erlang Solutions guys and the general quality of the Erlang and Elixir people. Lots of interesting chats!
:-)