This might be useful to some if you need a very light pub/sub inside one process.
I was building a small multiplayer game in Go. Started with a channel fan-out but (for no particular reason) wanted to see if we can do better. Put together this tiny event bus to test, and on my i7-13700K it delivers events in 10-40ns, roughly 4-10x faster than the plain channel loop, depending on the configuration.
I didn't look, but I don't think of channels as a pub/sub mechanism. You can have a producer close() a channel to notify consumers of a value available somewhere else, or you can loop through a bunch of buffered channels and do nonblocking sends.
A different design, without channels, could improve on those.
> High Performance: Processes millions of events per second, about 4x to 10x faster than channels.
Wow - that’s a pretty impressive accomplishment. I’ve been meaning to move some workers I have to a pub/sub on https://www.typequicker.com.
I might try using this in prod. I don’t really need the insane performance benefits as I don’t have my traffic lol - but I always like experimenting with new open source libraries - especially while the site isn’t very large yet
Really cool site.
One small issue that I found is that you cannot change the keyboard layout. Not sure if it should somehow automatically pick it up through some browser feature. But it would be nice to change it manually :)
It’s always worth discussing what features were thrown out to get the performance boost, whether it’s fair for those features to impose a tax on all users who don’t or rarely use those features, and whether there’s a way to rearrange the code so that the lesser used features are a low cost abstraction, one that you mostly only pay if you use those features and are cheap if not free if you don’t.
There’s a lot of spinoff libraries out there that have provoked a reaction from the core team that cuts down cost of their implementation by 25, 50%. And that’s a rising tide that lifts all boats.
Interesting, I need to dig into the guts of this because this seems cool.
I'm a bit out of practice with Go but I never thought that the channels were "slow", so getting 4-10x the speed is pretty impressive. I wonder if it shares any design with LMAX Disruptor...
> I wonder if it shares any design with LMAX Disruptor...
I've recently switched from using Disruptor.NET to Channel<T> in many of my .NET implementations that require inter-thread sync primitives. Disruptor can be faster, but I really like the semantics of the built-in types.
OP: the readme could really benefit from a section describing the underlying methodology, and comparing it to other approaches (Go channels, LMAX, etc...)
After a brief skim, it looks like this implementation is highly optimized for throughput and broadcasts whereas a channel has many other usecases.
Consumers subscribing to the same event type are placed in a group. There is a single lock for the whole group. When publishing, the lock is taken once and the event is replicated to each consumer's queue. Consumers take the lock and swap their entire queue buffer, which lets them consume up to 128 events per lock/unlock.
Since channels each have a lock and only take 1 element at a time, they would require a lot more locking and unlocking.
There is also some frequent polling to maintain group metadata, so this could be less ideal in low volume workloads where you want CPU to go to 0%.
Seems that the trick would be detecting if there is a queue building up and dispatching multiple events per lock if so. Double buffering is a common enough solution here. The reader gets one buffer to write to and the writer gets another, and when the read buffer is drained the write buffer is swapped.
For low traffic messages you need only send one message at a time, but if the receiver slows down the sender can avoid resorting to back pressure until the buffer is more than half full.
I wonder if a broadcast could be implemented as a specialization of a ring buffer:
Use 1 ring buffer, and N read pointers. The writer determines how far it can write by taking the minimum of the read pointers. ... I think that's all you'd need? The writer will block if a reader is too slow, but any broadcast that preserves messages and has bounded memory will have that problem. Maybe the difficulties will be in adding and removing readers.
This is pretty neat, code looks minimal as well. At pico.sh we wrote our own pubsub impl in Go that leveraged channels. We primarily built it to use with https://pipe.pico.sh
I see what you’re getting at, but if you add a database call the I/O blocking time will completely eclipse CPU time. It would not be a useful comparison, similar to if you added a time.Sleep to each event handler.
The reviews by some other people here lead me to believe that it works fairly well both when there is something to process and when it's just chattiness.
If you mean Amdahl's Law (and maybe Little), messaging is generally considered part of the unavoidable. However TCP and this library both seem to be aware that while you cannot eliminate messaging overhead, you can amortize it over dozens and dozens of messages at a time and that reduces that bit of serialized cost by more than an order of magnitude.
How else do you compare “web frameworks” except foe comparing their overhead?
No everyone wants to write a database application. There are absolutely other types of applications in the world. Applications can be CPU and/or memory bound.
24 comments
[ 3.2 ms ] story [ 46.8 ms ] threadI was building a small multiplayer game in Go. Started with a channel fan-out but (for no particular reason) wanted to see if we can do better. Put together this tiny event bus to test, and on my i7-13700K it delivers events in 10-40ns, roughly 4-10x faster than the plain channel loop, depending on the configuration.
I'd be interested to learn why/how and what the underlying structural differences are that make this possible.
A different design, without channels, could improve on those.
Wow - that’s a pretty impressive accomplishment. I’ve been meaning to move some workers I have to a pub/sub on https://www.typequicker.com.
I might try using this in prod. I don’t really need the insane performance benefits as I don’t have my traffic lol - but I always like experimenting with new open source libraries - especially while the site isn’t very large yet
There’s a lot of spinoff libraries out there that have provoked a reaction from the core team that cuts down cost of their implementation by 25, 50%. And that’s a rising tide that lifts all boats.
I'm a bit out of practice with Go but I never thought that the channels were "slow", so getting 4-10x the speed is pretty impressive. I wonder if it shares any design with LMAX Disruptor...
I've recently switched from using Disruptor.NET to Channel<T> in many of my .NET implementations that require inter-thread sync primitives. Disruptor can be faster, but I really like the semantics of the built-in types.
https://learn.microsoft.com/en-us/dotnet/core/extensions/cha...
https://learn.microsoft.com/en-us/dotnet/api/system.threadin...
Consumers subscribing to the same event type are placed in a group. There is a single lock for the whole group. When publishing, the lock is taken once and the event is replicated to each consumer's queue. Consumers take the lock and swap their entire queue buffer, which lets them consume up to 128 events per lock/unlock.
Since channels each have a lock and only take 1 element at a time, they would require a lot more locking and unlocking.
There is also some frequent polling to maintain group metadata, so this could be less ideal in low volume workloads where you want CPU to go to 0%.
For low traffic messages you need only send one message at a time, but if the receiver slows down the sender can avoid resorting to back pressure until the buffer is more than half full.
Use 1 ring buffer, and N read pointers. The writer determines how far it can write by taking the minimum of the read pointers. ... I think that's all you'd need? The writer will block if a reader is too slow, but any broadcast that preserves messages and has bounded memory will have that problem. Maybe the difficulties will be in adding and removing readers.
https://github.com/picosh/pubsub
With this impl can you stream data or is it just for individual events?
Add a database call or some simple data processing and then show some numbers comparing between channels or throughput.
I hate these kind of claims. Similar with web frameworks that shows reqs/s for an empty method.
If you mean Amdahl's Law (and maybe Little), messaging is generally considered part of the unavoidable. However TCP and this library both seem to be aware that while you cannot eliminate messaging overhead, you can amortize it over dozens and dozens of messages at a time and that reduces that bit of serialized cost by more than an order of magnitude.
No everyone wants to write a database application. There are absolutely other types of applications in the world. Applications can be CPU and/or memory bound.
Many, many real apps process millions of events per second. You may not have worked on any of them but that does not mean they don't exist.
Isn’t this the opposite? Generic is usually implying any type would do.