72
votes

I have set up a cronjob for root user in ubuntu environment as follows by typing crontab -e

  34 11 * * * sh /srv/www/live/CronJobs/daily.sh
  0 08 * * 2 sh /srv/www/live/CronJobs/weekly.sh
  0 08 1 * * sh /srv/www/live/CronJobs/monthly.sh

But the cronjob does not run. I have tried checking if the cronjob is running using pgrep cron and that gives process id 3033. The shell script calls a python file and is used to send an email. Running the python file is ok. There's no error in it but the cron doesn't run. The daily.sh file has the following code in it.

python /srv/www/live/CronJobs/daily.py
python /srv/www/live/CronJobs/notification_email.py
python /srv/www/live/CronJobs/log_kpi.py
13
Bear in mind that cron jobs do not run with all the environment that an interactive command would - for instance, when you call "python", that's being looked up to a full file path based on the environment variable $PATH. You should specify the path in full to avoid that dependency. - IMSoP
Another class of crontab problem not covered here are PATH/environment related problems, which are covered in Running a bash script from a cronjob fails with “No such file or directory”. - Martin Prikryl

13 Answers

235
votes

WTF?! My cronjob doesn't run?!

Here's a checklist guide to debug not running cronjobs:

  1. Is the Cron daemon running?
  • Run ps ax | grep cron and look for cron.
  • Debian: service cron start or service cron restart
  1. Is cron working?
  • * * * * * /bin/echo "cron works" >> /tmp/file
  • Syntax correct? See below.
  • You obviously need to have write access to the file you are redirecting the output to. A unique file name in /tmp which does not currently exist should always be writable.
  • Probably also add 2>&1 to include standard error as well as standard output, or separately output standard error to another file with 2>>/tmp/errors
  1. Is the command working standalone?
  • Check if the script has an error, by doing a dry run on the CLI
  • When testing your command, test as the user whose crontab you are editing, which might not be your login or root
  1. Can cron run your job?
  • Check /var/log/cron.log or /var/log/messages for errors.
  • Ubuntu: grep CRON /var/log/syslog
  • Redhat: /var/log/cron
  1. Check permissions
  • Set executable flag on the command: chmod +x /var/www/app/cron/do-stuff.php
  • If you redirect the output of your command to a file, verify you have permission to write to that file/directory
  1. Check paths
  • check she-bangs / hashbangs line
  • do not rely on environment variables like PATH, as their value will likely not be the same under cron as under an interactive session
  1. Don't suppress output while debugging
  • Commonly used is this suppression: 30 1 * * * command > /dev/null 2>&1
  • Re-enable the standard output or standard error message output by removing >/dev/null 2>&1 altogether; or perhaps redirect to a file in a location where you have write access: >>cron.out 2>&1 will append standard output and standard error to cron.out in the invoking user's home directory.
  • If you are trying to figure out why something failed, the error messages will be visible in this file. Read it and understand it.

Still not working? Yikes!

  1. Raise the cron debug level
  • Debian
    • in /etc/default/cron
    • set EXTRA_OPTS="-L 2"
    • service cron restart
    • tail -f /var/log/syslog to see the scripts executed
  • Ubuntu
    • in /etc/rsyslog.d/50-default.conf
    • add or comment out line cron.* /var/log/cron.log
    • reload logger sudo /etc/init.d/rsyslog restart
    • re-run cron
    • open /var/log/cron.log and look for detailed error output
  • Reminder: deactivate log level, when you are done with debugging
  1. Run cron and check log files again

Cronjob Syntax

# Minute  Hour  Day of Month      Month         Day of Week    User Command    
# (0-59) (0-23)   (1-31)    (1-12 or Jan-Dec) (0-6 or Sun-Sat)  
         
    0       2       *             *                *          root /usr/bin/find

This syntax is only correct for the root user. Regular user crontab syntax doesn't have the User field (regular users aren't allowed to run code as any other user);

# Minute  Hour  Day of Month      Month         Day of Week    Command    
# (0-59) (0-23)   (1-31)    (1-12 or Jan-Dec) (0-6 or Sun-Sat)  
         
    0       2       *             *                *          /usr/bin/find

Crontab Commands

  1. crontab -l
    • Lists all the user's cron tasks.
  2. crontab -e, for a specific user: crontab -e -u agentsmith
    • Starts edit session of your crontab file.
    • When you exit the editor, the modified crontab is installed automatically.
  3. crontab -r
    • Removes your crontab entry from the cron spooler, but not from crontab file.
21
votes

Another reason crontab will fail: Special handling of the % character.

From the man file:

The entire command portion of the line, up to a newline or a
"%" character, will be executed by /bin/sh or by the shell specified
in the SHELL variable of the cronfile.  A "%" character in the
command, unless escaped with a backslash (\), will be changed into
newline characters, and all data after the first % will be sent to
the command as standard input.

In my particular case, I was using date --date="7 days ago" "+%Y-%m-%d" to produce parameters to my script, and it was failing silently. I finally found out what was going on when I checked syslog and saw my command was truncated at the % symbol. You need to escape it like this:

date --date="7 days ago" "+\%Y-\%m-\%d"

See here for more details:

http://www.ducea.com/2008/11/12/using-the-character-in-crontab-entries/

14
votes

Finally I found the solution. Following is the solution:-

  1. Never use relative path in python scripts to be executed via crontab. I did something like this instead:-

    import os
    import sys
    import time, datetime
    
    CLASS_PATH = '/srv/www/live/mainapp/classes'
    SETTINGS_PATH = '/srv/www/live/foodtrade'
    sys.path.insert(0, CLASS_PATH)
    sys.path.insert(1,SETTINGS_PATH)
    
    import other_py_files
    
  2. Never supress the crontab code instead use mailserver and check the mail for the user. That gives clearer insights of what is going.

5
votes

I want to add 2 points that I learned:

  1. Cron config files put in /etc/cron.d/ should not contain a dot (.). Otherwise, it won't be read by cron.
  2. If the user running your command is not in /etc/shadow. It won't be allowed to schedule cron.

Refs:

  1. http://manpages.ubuntu.com/manpages/xenial/en/man8/cron.8.html
  2. https://help.ubuntu.com/community/CronHowto
2
votes

To add another point, a file in /etc/cron.d must contain an empty new line at the end. This is likely related to the response by Luciano which specifies that:

The entire command portion of the line, up to a newline or a "%"
character, will be executed
1
votes

I've found another reason for user's crontab not running: the hostname is not present on the hosts file:

user@ubuntu:~$ cat /etc/hostname
ubuntu

Now the hosts file:

user@ubuntu:~$ cat /etc/hosts
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

This is on a Ubuntu 14.04.3 LTS, the way to fix it is adding the hostname to the hosts file so it resembles something like this:

user@ubuntu:~$ cat /etc/hosts
127.0.0.1 ubuntu localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
1
votes

For me, the solution was that the file cron was trying to run was in an encrypted directory, more specifcically a user diretory on /home/. Although the crontab was configured as root, because the script being run exisited in an encrypted user directory in /home/ cron could only read this directory when the user was actually logged in. To see if the directory is encrypted check if this directory exists:

/home/.ecryptfs/<yourusername>

if so then you have an encrypted home directory.

The fix for me was to move the script in to a non=encrypted directory and everythig worked fine.

1
votes

I found useful debugging information on an Ubuntu 16.04 server by running:

systemctl status cron.service

In my case I was kindly informed I had left a comment '#' off of a remark line:

Aug 18 19:12:01 is-feb19 cron[14307]: Error: bad minute; while reading /etc/crontab
Aug 18 19:12:01 is-feb19 cron[14307]: (*system*) ERROR (Syntax error, this crontab file will be ignored)
1
votes

It might also be a timezone problem.

Cron uses the local time.

Run the command timedatectl to see the machine time and make sure that your crontab is in this same timezone.

1
votes

There is already a lot of answers, but none of them helped me so I'll add mine here in case it's useful for somebody else.

In my situation, my cronjobs were working find until there was a power shortage that cut the power to my Raspberry Pi. Cron got corrupted. I think it was running a long python script exactly when the shortage happened. Nothing in the main answer above worked for me. The solution was however quite simple. I just had to force reinstallation of cron with:

sudo apt-get --reinstall install cron 

It work right away after this.

0
votes

I experienced same problem where crons are not running. We fixed by changing permissions and owner by Crons made root owner as we had mentioned in crontab AND Cronjobs 644 permission given

0
votes

I had a similar problem to the link below.

similar to my problem
my original post

My Issue

My issue was that cron / crontab wouldn't execute my bash script. that bash script executed a python script.

original bash file

#!/bin/bash

python /home/frosty/code/test_scripts/test.py

python file (test.py)

from datetime import datetime

def main():
    dt_now = datetime.now()
    string_now = dt_now.strftime('%Y-%m-%d %H:%M:%S.%f')
    with open('./text_file.txt', 'a') as f:
        f.write(f'wrote at {string_now}\n')

    return None

if __name__ == '__main__':
    main()

the error I was getting

  File "/home/frosty/code/test_scripts/test.py", line 7
    string_to_write = f'wrote at {string_now}\n'
                                               ^
SyntaxError: invalid syntax

this error didn't make sense because the code executed without error from the bash file and the python file.

** Note -> ensure in the crontab -e file you don't suppress the output. I sent the output to a file by adding >>/path/to/cron/output/file.log 2>&1 after the command. below is my crontab -e entry

*/5 * * * * /home/frosty/code/test_scripts/echo_message_sh >>/home/frosty/code/test_scripts/cron_out.log 2>&1

the issue

cron was using the wrong python interpreter, probably python 2 from the syntax error.

how I solved the problem

I changed my bash file to the following

#!/bin/bash

conda_shell=/home/frosty/anaconda3/etc/profile.d/conda.sh
conda_env=base

source ${conda_shell}
conda activate ${conda_env}

python /home/frosty/code/test_scripts/test.py

And I changed my python file to the following

from datetime import datetime

def main():
    dt_now = datetime.now()
    string_now = dt_now.strftime('%Y-%m-%d %H:%M:%S.%f')
    string_file = '/home/frosty/code/test_scripts/text_file.txt'
    string_to_write = 'wrote at {}\n'.format(string_now)
    with open(string_file, 'a') as f:
        f.write(string_to_write)

    return None

if __name__ == '__main__':
    main()
-1
votes

Sometimes the command that cron needs to run is in a directory where cron has no access, typically on systems where users' home directories' permissions are 700 and the command is in that directory.