Python program running at regular time interval tutorial using time and datetime module

This tutorial will teach you how to set up a Python program to run at regular time intervals using the time and datetime module. This is useful if you want to check something constantly, so you have a python program running in the background doing the checking for you at regular intervals.

Solution

I will put the solution on top. If you are interested in how I found it, you can read about it below.

Simple solution for if what you do is lightweight

Lightweight means that it doesn't use much CPU power and time, like a simple print function.

Assuming you want to run the code every 10 seconds.

import time
from datetime import datetime

def do_things():
    print(datetime.now())

while True:
    do_things()
    time.sleep(10)

you should get something like this

2022-08-24 10:59:35.247516
2022-08-24 10:59:45.259224
2022-08-24 10:59:55.272715
2022-08-24 11:00:05.284097
2022-08-24 11:00:15.290823

There you have it, you can replace the sleep time if you want a different interval, and you can replace the content of the do_things() function with what you want the program to do.

Problem with the simple solution

But if you are like me, what you wish to do every interval takes time, whether it is waiting for I/O from another program, or it is doing something very CPU intensive. Then if we still do what we did, we will get inaccuracies in wait time like the below.

import time
from datetime import datetime

def do_things():
    number = 50_000_000
    sum(i*i for i in range(number))
    print(datetime.now())

while True:
    do_things()
    time.sleep(10)

Here I used sum(i*i for i in range(number)) on a huge number to force the CPU to do a lot of work, don't mind the underscore in 50_000_000, it is the same as 50000000. The result of this program is:

2022-08-24 11:07:54.377208
2022-08-24 11:08:07.599538
2022-08-24 11:08:20.838108
2022-08-24 11:08:34.043164
2022-08-24 11:08:47.255688

As you can see, we want the program to execute every 10 seconds, but it took the program about 14 seconds each loop, this is because some extra time is spent computing. This is why you need the advanced solution.

Advanced solution

import time
from datetime import datetime, timedelta

def do_things():
    number = 50_000_000
    sum(i*i for i in range(number))
    print(datetime.now())

while True:
    previous_datetime = datetime.now()
    do_things()
    time_took_running_do_things = datetime.now() - previous_datetime
    remaining_time_in_secs = (timedelta(seconds=10) - time_took_running_do_things).total_seconds()
    time.sleep(remaining_time_in_secs)

and the result is:

2022-08-24 11:14:40.821099
2022-08-24 11:14:50.797054
2022-08-24 11:15:00.806950
2022-08-24 11:15:10.834795
2022-08-24 11:15:20.845102

Explanation

Here you can see that the codes run every 10 seconds even though we are doing the heavy lifting. This is because we compensated for the time taken to run do_things() by waiting for less afterwards. If it took the program 3 seconds to run do_things(), we only wait 7 seconds afterwards, so the program still do things every 10 seconds.

Limitation

This solution is limited by, the action you want to do must to take longer than the interval you want to loop. If you want to do an operation that takes 10 seconds every 1 second, this solution cannot help you. You might want to look into Concurrency from this RealPython article.

How I got there

Now I will talk about how I figured this out.

I am working ona program that sees if I'm playing computer games at regular intervals to remind myself how long I'm playing, but the problem is that it is very inaccurate. I would play a game for 45 minutes, then the program will tell me that I only played for 30 minutes.

I first thought this is a problem with time.sleep(), maybe because the CPU is doing a lot of work while gaming, the wait is wrong.

But this is proven wrong by when I'm not playing games, the program still runs at incorrect intervals.

Then I commented out the things the program does within each interval, only leaving the print time statement, and then the problem was fixed. It turns out the problem is what I do inside the interval is costing above 2 seconds!

I then immediately went on to try to over-engineer the problem. I thought to myself, maybe I want the loop and the do_things() function in separate threads via threading or asyncio, or maybe even separate CPU using multiprocessing. I watched the entire tutorial on Speed Up Your Python Program With Concurrency by Jim Anderson on RealPython until I drew this graph.

I then realize, what if I just do them all linearly, then compute how much time I need to wait? It is then that it dawned on me, that the problem was never this complicated. So I fixed the problem and decided to write this article to help others like me. So there you have it.