Python module not found when launched in cron

I am running a little ubuntu server (version 18.04)

Before upgrading to 18.04 the cron was executing just fine.

This is my crontab (sudo crontab -e) :

*/5 * * * * PYTHONPATH=/usr/bin/python3 /usr/bin/python3 /home/louis/backup-server/main.py >> /var/log/MYbackup.log 2>&1

But when the cron start the following is logged :

Traceback (most recent call last): File "/home/louis/backup-server/main.py", line 7, in <module> import config.config as config File "/home/louis/python_helpers/config/config.py", line 2, in <module> from dotenv import load_dotenv
ModuleNotFoundError: No module named 'dotenv'

however if I run sudo python3 /home/louis/backup-server/main.py the script is correctly executed

I did run sudo pip install python-dotenv

Content of scripts :

# /home/louis/backup-server/main.py
# permissions : -rwxr-xr-x
import os
import requests
import json
import datetime
import sys
sys.path.insert(0, '/home/louis/python_helpers')
import config.config as config
import mail
import utils
def main(): # some code
if __name__ == "__main__": main()

--

# /home/louis/python_helpers/config/config.py
# permissions : -rwxr-xr-x
import os
from dotenv import load_dotenv
dotenv_path = os.path.join(os.path.dirname(__file__), '.env')
load_dotenv(dotenv_path)
# some config var

I do have an __init__.py file in /home/louis/python_helpers/config, let me know if for any other information I should add.

What bug me the most is that the scripts run just fine if started manually but not as cron.

Edit : I was unsure whether this question belongs here or on stackoverflow, so please tell me if I should move it (even though I don't know how beside creating a new question)

4 Answers

Well the issue was pretty dumb, I did run pip install python-dotenv and sudo pip install python-dotenv (the script need root permission to access some folder) but I didn't do sudo -H pip install python-dotenv.

After running this command, the cron execute just fine.

I had similar issues today, and here is how I solved it! Hope it'll help you.

Do you have two different python? Try to run:

where python3

If you have both /usr/local/bin/python3 and /usr/bin/python3 there, and your package is installed in /usr/local/lib but you're specifying to use /usr/bin/python3 in PYTHONPATH, it'll throw ModuleNotFoundError.

You can either:

1. specify /usr/local/bin/python3 in your PYTHONPATH
2. or run /usr/local/bin/python3 FILE in your cronjob.
3. or put #!/usr/local/bin/python3 in the first line of your file

The issue is probably this part:

PYTHONPATH=/usr/bin/python3

because PYTHONPATH should point to where your libraries are. What happens if you remove that part of the crontab?

for every user which you use for cron jobs, you should do pip install your-library. Because cron uses the environment whose user run the job.