I wonder if it would make sense to add a similar kind of support to redis. Maybe if a sorted set's read/write ration goes above some fixed threshold then it could substitute an implementation based on a sorted array instead of a balanced tree.
I guess the other problem is probably due to however redis was storing the actual data under the covers (stringified ints? is binary an option). This probably is one of the major contributors to the massive memory usage.
Still for something this specific (and intensive) a custom application makes a lot of sense. I wonder what exactly their IPC mechanism was, they said that they had the downloading process 'signal' the intersecting service when new data was available. i wonder if they just sent a SIGUSR?
1 comment
[ 3.1 ms ] story [ 10.4 ms ] threadI wonder if it would make sense to add a similar kind of support to redis. Maybe if a sorted set's read/write ration goes above some fixed threshold then it could substitute an implementation based on a sorted array instead of a balanced tree.
I guess the other problem is probably due to however redis was storing the actual data under the covers (stringified ints? is binary an option). This probably is one of the major contributors to the massive memory usage.
Still for something this specific (and intensive) a custom application makes a lot of sense. I wonder what exactly their IPC mechanism was, they said that they had the downloading process 'signal' the intersecting service when new data was available. i wonder if they just sent a SIGUSR?