Unfortunately Go's lack of generics hurts in letting me use these helpers in a way that guarantees the data types going in/out of the generic channels.
Right. Everything gets converted to Go's "any" type, "Interface{}". At the output end you have to use a type assertion to get it back into the actual type. Note that an "Interface{}" is a pointer; you're not queuing the data, you're queuing a pointer to the data. So you're sharing data between goroutines. The sender must allocate a new variable for each send.
(This is my biggest complaint about Go. It's too easy to accidentally share data between goroutines, because so many things are implicitly references to mutable objects. Slices, for example, are references. Erlang and Rust, which also use queues heavily, avoid doing this.)
I think you misunderstood the blog post and go channels. Shuffle is absolutely not want you want to use (in the README example). Sending to a channel blocks until the message is accepted by the receiver (unless you're using buffered channel).
What you want to do instead, create 1 channel, and make multiple goroutines (Sinks) read from it, then whenever goroutine is finished with the task it would take a new message from that channel.
By picking a channel to dispatch yourself you don't take into account busy-ness of it, so you might wait on a channel even though there are others that are idling waiting for messages to be accepted.
Hi @epimenov thank you for the feedback. The shuffle method starts a goroutine for each inbound channel, so one busy outbound channel will not block the whole system. That being said I probably should add two more features, one of which is to add a channel size rather than always using a 0 size channel, and the other is to change it to select the first non-blocking outbound channel. I don't consider the blocking part always negative, because handling back-pressure is need at one point or another, but you are right, the method should not introduce starvation.
You probably know this already, but "topo" means mouse in Italian.
Looks like an interesting project. It'd be interesting to see someone who knows both Erlang and Go well explain this in comparative terms. I know Erlang reasonably well, but not Go.
7 comments
[ 2.6 ms ] story [ 28.1 ms ] threadhttps://github.com/eapache/channels
Unfortunately Go's lack of generics hurts in letting me use these helpers in a way that guarantees the data types going in/out of the generic channels.
(This is my biggest complaint about Go. It's too easy to accidentally share data between goroutines, because so many things are implicitly references to mutable objects. Slices, for example, are references. Erlang and Rust, which also use queues heavily, avoid doing this.)
What you want to do instead, create 1 channel, and make multiple goroutines (Sinks) read from it, then whenever goroutine is finished with the task it would take a new message from that channel.
By picking a channel to dispatch yourself you don't take into account busy-ness of it, so you might wait on a channel even though there are others that are idling waiting for messages to be accepted.
Looks like an interesting project. It'd be interesting to see someone who knows both Erlang and Go well explain this in comparative terms. I know Erlang reasonably well, but not Go.