There are five ways to use scheduled jobs in Python

Most applications built today require some sort of scheduling mechanism. Polling API s or databases, constantly checking system health, archiving logs, etc. are common examples. Kubernetes and Apache Mesos And other software using auto scaling technology need to check the status of deployed applications. For this purpose, they use the survival probe running regularly. Scheduling tasks need to be decoupled from business logic, so we need to use decoupled execution queues, such as Redis Queue.

Python has several ways to schedule a task regularly, which is what we will learn in this article. I will discuss scheduling tasks in the following ways:

1. Simple Loops
2. Simple Loops but Threaded
3. Schedule Library
4. Python Crontab
5. RQ Scheduler as decoupled queues

Simple loops

Scheduling tasks using simple loops is effortless. Using an infinitely running while loop to periodically call functions can be used to schedule jobs, but this is not the best way, but it is very effective. You can use slleep() of the built-in time module to delay execution. However, this is not the scheduling method of most jobs, because it looks ugly and its readability is poor compared with other methods.

import time

def task():
    print("Job Completed!")

 while 1:
    task()
    time.sleep(10)

When it comes to schedules such as 9:00 a.m. every morning or 7:45 p.m. every Wednesday, things become more difficult.

import datetime

def task():
    print("Job Completed!")

 while 1:
    now = datetime.datetime.now()
    # schedule at every wednesday,7:45 pm
    if now.weekday == 3 and now.strftime("%H:%m") == "19:45":
        task()
    # sleep for 6 days
    time.sleep(6 * 24 * 60 * 60)

This is the first solution I came up with. You're welcome! One problem with this method is that the logic here is blocked, that is, once this code is found in the python project, it will get stuck in the while 1 loop, blocking the execution of other code.

Simple loops but threaded

thread Is a concept in computer science. Small programs with their own instructions are executed by the process and managed independently, which can solve the blocking of our first method. Let's see how it works.

import time
import threading

def task():
    print("Job Completed!")

def schedule():
    while 1:
        task()
        time.sleep(10)

# makes our logic non blocking
thread = threading.Thread(target=schedule)
thread.start()

After the thread is started, its underlying logic cannot be modified by the main thread, so we may need to add resources through which the program can check specific scenarios and execute logic according to them.

Schedule Library

Earlier, I said that scheduling with a while loop looked ugly, dispatch Library can solve this problem.

import schedule
import time

def task():
    print("Job Executing!")

# for every n minutes
schedule.every(10).minutes.do(task)

# every hour
schedule.every().hour.do(task)

# every daya at specific time
schedule.every().day.at("10:30").do(task)

# schedule by name of day
schedule.every().monday.do(task)

# name of day with time
schedule.every().wednesday.at("13:15").do(task)

while True:
    schedule.run_pending()
    time.sleep(1)

As you can see, in this way, we can easily create multiple scheduling plans. I especially like the method of creating jobs and Method Chaining. On the other hand, this fragment has a while loop, which means that the code is blocked, but I believe you already know what can help us solve this problem.

[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (IMG dnycdmcf-1633830381091)( https://unsplash.com/ @redviking509?utm_source=medium&utm_medium=referral)] on Unsplash](https://cdn-images-1.medium.com/max/8634/0*fT92Yr0JununUEEl)

Python Crontab

The crontab utility in Linux is an easy-to-use and widely accepted scheduling solution. Python Library python-crontab Provides an API to use CLI tools in Python. In crontab, a timing schedule is described in UNIX cron string format (* * *). It is a line with a set of five values, which indicates that when the job should be executed, Python crontab converts the plan to write crontab in the file into a write programming method.

[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-cqkztfwc-1633830381094)( https://linuxhint.com/add-crontab-freebsd/ )]]( https://cdn-images-1.medium.com/max/2824/1 *x-Uv0ZZvTMDXTqqvpdEJQQ.png)

from crontab import CronTab

cron = CronTab(user='root')

job = cron.new(command='my_script.sh')

job.hour.every(1)
cron.write()

Python crontab will not automatically save the plan. You need to execute the write() method to save the plan. There are more features, and I strongly recommend that you view their documentation.

RQ Scheduler

Some tasks cannot be executed immediately, so we need to create task queues and pop up tasks according to LIFO or FIFO queue systems. python-rq Allow us to do this by using Redis as a proxy to queue jobs. The entry for the new job is stored as a hash map with information, such as created_at, enqueued_at, origin, data, description.

The queued task is executed by a program named worker. workers also has an entry in Redis cache, which is responsible for listing tasks and updating task status in Redis. Tasks can be queued when needed, but to arrange them, we need to rq-scheduler.

from rq_scheduler import Scheduler

queue = Queue('circle', connection=Redis())
scheduler = Scheduler(queue=queue)

scheduler.schedule(
    scheduled_time=datetime.utcnow(), # Time for first execution, in UTC timezone
    func=func,                     # Function to be queued
    args=[arg1, arg2],             # Arguments passed into function when executed
    kwargs={'foo': 'bar'},         # Keyword arguments passed into function when executed
    interval=60,                   # Time before the function is called again, in seconds
    repeat=None,                     # Repeat this number of times (None means repeat forever)
    meta={'foo': 'bar'}            # Arbitrary pickleable data on the job itself
)

RQ worker must be started separately in the terminal or through Python RQ worker. Once the task is triggered, it can be seen in the work terminal. Separate function callbacks can be used in both success and failure scenarios.

Conclusion

There are also some libraries for scheduling, but here I've discussed the most common libraries. It is worth mentioning that Celery , another advantage of celery is that users can choose between multiple agents. I appreciate you reading to the end. You can also read my other articles. Cheers!

Translation source: https://python.plainenglish.io/5-ways-to-schedule-jobs-in-python-99de8a80f28e