How can to start tasks with celery-beat?

Question

Why i can't run a periodic tasks?

proj/settings.py

REDIS_HOST = 'localhost'
REDIS_PORT = '6379'
CELERY_BROKER_URL = 'redis://localhost:6379'
BROKER_URL = 'redis://' + REDIS_HOST + ':' + REDIS_PORT

CELERY_BEAT_SCHEDULE = {
    'task-first': {
        'task': 'app.tasks.one',
        'schedule': timedelta(seconds=1)
    },
    'task-second': {
        'task': 'app.tasks.two',
        'schedule': crontab(minute=0, hour='*/3,10-19')
    }
}

proj/celery.py

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')
app.config_from_object('django.conf:settings')

app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

proj/__init.py__

from .celery import app as celery_app

__all__ = ['celery_app']

celery -A proj worker -l info

[2019-10-31 16:57:57,906: INFO/MainProcess] Connected to redis://localhost:6379//
[2019-10-31 16:57:57,912: INFO/MainProcess] mingle: searching for neighbors
[2019-10-31 16:57:58,927: INFO/MainProcess] mingle: all alone
[2019-10-31 16:57:58,951: INFO/MainProcess] celery@lexvel-MS-7A72 ready.

Tasks are found

celery -A proj beat -l info

Configuration ->
    . broker -> redis://localhost:6379//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%INFO
    . maxinterval -> 5.00 minutes (300s)
[2019-10-31 16:58:02,851: INFO/MainProcess] beat: Starting...

The celerybeat-shedule file is created. But besides these lines nothing more is displayed.

tasks

@task()
def one():
    print('start 1', datetime.now())
    driver = init_driver()
    parse(driver)
    driver.close()
    driver.quit()
​
​
@task()
def two():
    print('start 2', datetime.now())
    driver = init_driver()
    parse2(driver)
    driver.close()
    driver.quit()
    print('end 2', datetime.now())

Answer 1

---

Celery beat command

celery -A proj worker -l info -B --scheduler django_celery_beat.schedulers:DatabaseScheduler

This command has used for start the celery beat.

Firstly add the django_celery_beat module in installed apps in settings file.

And then apply the django migrate command, this will create the tables in admin pannel.

After completing all the process like in celery file and create task in tasks.py.

you will apply the beat command as mentions in above.

---

proj/settings.py

INSTALLED_APPS = [

    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    'django_celery_beat',
]

REDIS_URL = "redis://localhost:6379/1"

CELERY_BROKER_URL=REDIS_URL

CELERY_RESULT_BACKEND=REDIS_URL

CELERY_ACCEPT_CONTENT = ['application/json']

CELERY_RESULT_SERIALIZER = 'json'

CELERY_TASK_SERIALIZER = 'json'

CELERY_BEAT_SCHEDULE = {

        'task-first': {
        'task': 'app.tasks.one',
        'schedule': timedelta(seconds=1)
       },
      'task-second': {
        'task': 'app.tasks.two',
        'schedule': crontab(minute=0, hour='*/3,10-19')
      }
}

proj/celery.py

from celery import Celery

from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')

app.config_from_object('django.conf:settings')

app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

proj/__init.py__

from .celery import app as celery_app

**__all__** = ['celery_app']

Answer 2

When you ran your worker, did it say that app.tasks.one and app.tasks.two are registered tasks? If they do not appear there as registered tasks then your beat is scheduling tasks that can't be executed. - They will just wait in the queue, and eventually expire. Another way to check whether you have them registered is via celery -A proj.celeryapp inspect registered (change proj.celeryapp to whatever is the location of your Celery application).