Yet another and easier way to daemonize Celery

Posted on Mon 21 November 2016 in Django • Tagged with django, python, celeryLeave a comment

I wrote a post about how to run celery as daemon and it has many steps to do, many files to copy to your server, configure permissions, and configure upstart, but after sometime I found another and easier way to daemonize Celery.

In this method we will use supervisor to daemonize Celery, and this can be achieved by 3 simple steps, so let's do it.

Step 1

Install Supervisord sudo apt-get install supervisor on Ubuntu server. For other OSs please visit the official documentation

Step 2

This is the most important step, and in this step we will create supervisor configuration file to daemonize Celery.

Supervisor Configuration File

Celery project provide the basic configuration file in their Github repository.

Supervisor configuration files live in \etc\supervisor\conf\

Celery official configuration file

; ==================================
;  celery worker supervisor example
; ==================================

[program:celery]
; Set full path to celery program if using virtualenv
command=celery worker -A proj --loglevel=INFO

directory=/path/to/project
user=nobody
numprocs=1
stdout_logfile=/var/log/celery/worker.log
stderr_logfile=/var/log/celery/worker.log
autostart=true
autorestart=true
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs …
Continue reading

How to run Celery as daemon?

Posted on Tue 20 September 2016 in Django • Tagged with django, python, celeryLeave a comment

celery smoothy

Install upstart script

Permissions

We need to make the user which will be used to run celery an owner of log and run directories.

sudo chown -R root:root /var/log/celery/
sudo chown -R root:root /var/run/celery/

Celery upstart

1- Download celeryd file from Celery official github repository.

2- Copy celeryd to /etc/init.d/celeryd

sudo cp celeryd /etc/init.d/

3- Make it executable

sudo chmod +x celeryd

4- Make it run on startup

sudo update-rc.d celeryd defaults
sudo update-rc.d celeryd enable

5- make it owned by root

sudo chown root:root celeryd

6- Done, now you can start/stop/restart celeryd as service

sudo service celeryd start
sudo service celeryd stop
sudo service celeryd restart

Celerybeat upstart

1- Download celerybeat file from Celery official github repository.

2- Copy celerybeat to /etc/init.d/celerybeat

sudo cp celerybeat /etc/init.d/

3- Make it executable sudo chmod +x celerybeat

4- Make it run on startup

sudo update-rc.d celerybeat defaults
sudo update-rc.d celerybeat enable

5- make it owned by root

sudo chown root:root celerybeat

6- Done, now you can start/stop/restart celerd as service

sudo service celerybeat start
sudo service …
Continue reading

Send Emails Asynchronously from Django

Posted on Sun 24 January 2016 in Django • Tagged with django, python, celery, emailsLeave a comment

In this article you will learn:

  1. How to send emails from Django.
  2. How to use Celery and Django-Celery-Email to send email asynchronously.

Last week I had the chance to get my hand dirty with Celery, I used it to send emails from Django project asynchronously using Django app call djnago-celery-email. What am I mean by sending emails asynchronously? I mean that user doesn’t wait for SMTP/mail server to send email message and give feedback to the web server, instead Django will call the send email procedure, pass it to Celery to do it away from Django application, so that user won’t wait, for example you want to send email to customer once he save customer information, you don’t want to keep user wait for application to save the customer data and send email, by sending email asynchronously user will only wait for saving confirmation and sending email celery will take care of it in the background.

Alt Text

Now let’s see how we can do that.

Step 1: Install and Setup Celery

  • In terminal, active your virtualenv and type $ pip install django-celery
  • Now celery and its Django app djcelery are installed into your virtualenv.
  • Next step is …
Continue reading