HN Explorer

Ask HN: What's everyone using for background job processing?

3 points by blooberr ↗ HN
It's been awhile since I've had to revisit this topic and I found a great site: http://queues.io/

Any suggestions or recommendations? I want to be able to issue jobs, and track if they're completed. On a failure, I'd resend it to another worker.

Sounds simple enough, but it's hard to pick out the right one from that long list.

6 comments

[ 33.1 ms ] story [ 173 ms ] thread
natch
Haven't used it but doesn't AWS have something for this? Curious why that doesn't fit the bill if so.
blooberr
Can't use AWS unfortunately - this is not within my control.
SEJeff
python-rq[1], celery[2], beanstalkd[3], or mod_gearmand[4].

For several of my production projects, I like mixing celery with redis. They are a great fit.

[1] http://python-rq.org [2] http://www.celeryproject.org [3] http://kr.github.io/beanstalkd/ [4] http://gearman.org

blooberr
Thanks I'll check that out.
SEJeff
Care to up vote? :P
stevekemp
I have enjoyed using beanstalkd on several projects. It is simple, blazingly fast, and very very reliable.