An Afternoon of Code Golf in Lua to Achieve 4x Performance in Redis (amplitude.engineering)

What is Code Golf?

Hi, that's a very interesting post, thanks a lot ! It's cool to see that sometimes a small lua script can do exactly what you need, instead of adding a new Redis command to the already very long list.

I have a question regarding "deployment" of Lua scripts (I've never used Redis for more than toys, so I may be wrong on something). I would assume a caller would do something like this just after the Redis cluster is up:

- calculate script's SHA

- EVALSHA a script

- script is not there

- EVAL script

And then later in the lifetime of the caller:

- calculate script's SHA

- EVALSHA script

Which means you wouldn't need to care about the script size, nor would you need to fiddle with your deployment pipeline.

Am I missing something ?

What did you you use to make the flowcharts?

It's a commonly used primitive and I'm tempted to do it sometimes, but the problem is that there is no clear semantics. For instance in the blog post, apparently, the author is interested in incrementing multiple fields of the same key, while the proposal on the Github issue, wanted to increment different keys. Unfortunately to extend HINCRBY to make it variadic is hard because the return value would change... At the same time some users may absolutely not want to have all the items returned after the increment, since if you increment a batch of 1000 items and are not interested in the result, to get a 1000 items reply is not great.

Given all this problems, and in order to avoid introducing new commands, probably what could be done is to introduce options into HICNRBY, so that it can model both the variadic thing (only switching to a different reply when some option is given) and also to have specific options in order to switch on/off the reply of the incremented values and so forth.

Commands designed recently, after the old errors, have such capabilities. For instance BITFIELD is pretty powerful, and is a command doing increments: https://redis.io/commands/bitfield

Hi, I'm the author of the post. I'll have to look into getting the metrics from Twemproxy in the future. I typically only run MONITOR for a few seconds 'cause I'm usually just looking for high volume commands & keys.

OP may want to try "INFO commandstats" to get the same metrics in a much cheaper way. Example output:

    # Commandstats
    cmdstat_info:calls=1,usec=57,usec_per_call=57.00
    cmdstat_cluster:calls=8,usec=211,usec_per_call=26.38
    cmdstat_zrange:calls=4,usec=1990,usec_per_call=497.50