HN Explorer

Ask HN: How to become better at writing concurrent code?

13 points by yurish ↗ HN
I am mostly backend programmer. From time to time I need to solve various concurrensy problems and this may take me a lot of time. I may spend a week trying to solve some problem that looks pretty simple. Note, I know basic stuff like mutexes and semaphors work etc. My problem is that combining these primitives to invent solution that is deadlock and race conditions free is very time consuming and exostive. Do you have any advice on how to become better at this? Any approaches, ideas, notations, formal methods, books anything? Thanks in advance.

9 comments

[ 3.3 ms ] story [ 27.5 ms ] thread
PaulHoule
Mostly KISS (Keep It Simple Stupid), try not to be creative, try to reuse simple patterns as much as you can.

For instance, in the Java ecosystem there are the various Executors; Executors aren't good at everything, but they are good at many cases that turn up, and I've frequently spent a few days trying to understand what was going on with "sophisticated" solutions in Scala and/or Clojure and then spent 20 minutes coding it up with an Executor and at that point had something that worked correctly all the time and used all the CPU cores to boot.

seanwilson
> Mostly KISS (Keep It Simple Stupid), try not to be creative, try to reuse simple patterns as much as you can.

I'd agree with this! I'd go further and say in most languages avoid concurrency entirely if you can if it's not going to lead to measurable improvements. Concurrent code can be a minefield of problems and creates difficult to debug code so go with non-concurrent if it's all the same.

bsvalley
collaborate with other experienced dev folks. You learn a lot from reading people's code in a professional environment. Or, use high level libraries. If you still manipulate semaphores in 2017, you are wasting a lot of time since there are a couple of high level libraries out there that do the job for you. They're maintained by the open source community, etc.

Part of being a good programmer involves trusting other developers. Sharing code, reusing code, etc. That's how our community works and trying to use basic principles in your code isn't always the way to go. To manage concurrent code today it's better to use a Queue for instance. Setup your queue and just push task into it. Synchronously or asynchronously, you get to pick what ever you want. It usually comes with a bunch of other features, like maintaining a certain order when asynchronous tasks get executed. In most of the modern languages, they offer thread safe and none thread safe data structures. Pick the one you need. And that's just one example.

yurish
I have no problem with reusing existing libraries, queues whatever. But sometimes you have to work with lower level stuff. Even in 2017. So the question is less about general recommendations and more about specific methods that can help in designing and analysing concurrent code. Sorry if I did not make it clear.
brudgers
Tools choice may be an alternative. For example, Erlang addresses many concurrency issues by design and the required code may be easier to develop and the finished code may be easier to understand.

Of course the price of using a tool like Erlang [1] is that all that knowledge of mutexes and semaphores goes to waste and different concepts like immutability and let-it-crash must be understood if not mastered.

[1]: or perhaps Clojure or Go.

lovelearning
Two techniques that have helped me:

1. Diagrams that help visualize what's happening. I prefer a hybrid of sequence, timeline and activity diagrams. I don't have a ready example at hand, but [1] comes close to what I use. YMMV though - I know some team members in my past jobs preferred state transitions to these, because they found them clearer to think through.

2. A technique we used during team reviews was borrowed from the book "Debugging Applications" by John Robbins [2]. Although the book was about debugging Windows applications, it had good tips on general software engineering and debugging, and I still recommend it now. One technique in it was the multithread simulation review - an excerpt:

The trick is to assign one person to each thread in your code and one person to each synchronization object. When you review the code, pretend that each thread is running at realtime priority on its own dedicated CPU. Each "thread person" walks through the code paying attention only to the particular code that his thread is supposed to be executing. When the "thread person" is ready to acquire a synchronization object, the "object person" literally moves behind the "thread person." When the "thread person" releases a synchronization object, the "object person" goes to a neutral corner of the room. In addition to the thread and object representatives, you should have some developers who are monitoring the overall thread activity so that they can assess the program's flow and help determine the points at which different threads deadlock.

Obviously, you need a number of like-minded and enthusiastic team members to carry it off.

[1]: http://community.qnx.com/sf/wiki/do/viewAttachment/projects....

[2]: https://www.amazon.com/Debugging-Applications-DV-MPS-Program...

mbrock
You could try learning a model checker for concurrent algorithms, like TLA+.
yurish
Do you have experience with it? How hard is it to use TLA+ to solve every-day problems? Are there any other model checkers that worth looking at? Or TLA+ is the most sutable for concurrency problems?