HN Explorer

Ask HN: Would a server written in a native language still need job schedulers?

3 points by hasenj ↗ HN
I have a suspicion that things like Celery or RabbitMQ or whatever else people are using with Ruby/Python based servers are only needed because these languages are slow and don't really support threads. Even redis would probably be not needed.

Am I missing something important?

7 comments

[ 2.5 ms ] story [ 25.9 ms ] thread
mneil
Distribution. Redis, rabbitmq, and celary are all used to share state or messages across many servers. Unless you can run the entirety of an app like facebook on a single machine then you will, at some point, have to run several instances of the same application.

Micro services also utilize these databases and frameworks.

hasenj
But most people are not Facebook (and will never be).

If you have to have several hardware servers, I don't see why the application server itself cannot contain basic boilerplate code to coordinate communication between the different instances of itself that are running on different machines.

UK-AL
Because you want your web servers to handle requests fast, and serve as many requests as possible.

You want offload your non-time dependent requests to other machines to handle without slowing down your request processing on the web servers.

Also most queues provide a mechanism distributing load and autoscaling.

jetti
RabbitMQ is written in Erlang which definitely supports threads (called processes) and are actually built into the BEAM. I think it is a strong candidate for what Erlang brings to the table and I think while you could do it in a native language for sure, it would take more time and most likely be more error prone (unless there was immutability in the language as well).
le-mark
> Am I missing something important?

Asynchonous tasks like sending emails and processing credit card payments. Example, user signups, and your flow is to send a welcome email. You don't want the signup response to wait on sending the email for example. So you drop it on a queue to be processed "later".

hasenj
In python, yes. But if I was in Go, I'd just fire (and forget) a goroutine to do that.
le-mark
That's a good counter example, but doesn't address the other advantages of third party solutions like message durability, atomicity, and guaranteed completion. If the email only ever exists in the memory of the executing process, it can be easily lost during server restart for example.