HN Explorer

Ask HN: Is There a “Smart” Message Queue?

3 points by bgdam ↗ HN
I'm currently using RabbitMQ for a variety of message queuing needs, and I'm running into a specific issue repeatedly.

If I push 10 messages to a queue, and say 5 of them have the same 'key' parameter, then I want to automatically discard the previous messages still in the queue with the same key.

Is there any message queue which already has this baked into it out of the box?

1 comment

[ 3.8 ms ] story [ 11.7 ms ] thread
brudgers
Architecturally, the idea of discarding duplicate messages from a queue is:

  queue -> parser -> queue
Discarding duplicate messages is a piece of business logic. The parser could be as simple as a hashmap/dictionary or bloom filter that emits or does not emit the message based on the selected criteria.

This is a case where I like the idea of Unix pipes and where I might think in terms of log data structures rather than queues. Your mileage may vary.

Good luck.