I had no idea what this is, and I guess I'm not alone, so:
"Celery is a task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
"The execution units, called tasks, are executed concurrently on one or more worker servers. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
"Celery is already used in production to process millions of tasks a day.
"Celery was originally created for use with Django, but is now usable from any Python project. It can also operate with other languages via webhooks."
I'll second that.
It is good to know that Celery is being used in production... but give us a story. What were you doing? What was the problem? How did creating and deploying Celery help you solve it? Perhaps an example or two of other potential uses would be cool too.
Processing the file might take seconds or more. So instead of processing the file in the view, you schedule the processing job and return html output to the user "we are processing your request".
Eventually, the scheduled job will complete, and the result will be available to the user.
Celery seems to try to distinguish itself from other job queues by supporting multiple worker servers with a single queue, and concentrating on real-time processing.
I've used it on a few apps for when we have to send out emails. Maybe I'm doing it wrong, but any page that sent out an email always lagged a bit, this cleared out that delay for the user on their end and processed the email in the queue. Works great
9 comments
[ 3.5 ms ] story [ 34.7 ms ] threadEDIT: I appologize, it's the box shadow around the content container.
"Celery is a task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
"The execution units, called tasks, are executed concurrently on one or more worker servers. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
"Celery is already used in production to process millions of tasks a day.
"Celery was originally created for use with Django, but is now usable from any Python project. It can also operate with other languages via webhooks."
Thanks =)
Processing the file might take seconds or more. So instead of processing the file in the view, you schedule the processing job and return html output to the user "we are processing your request".
Eventually, the scheduled job will complete, and the result will be available to the user.