Cup of Code

Posted on Dec 31, 2023 • Originally published at cupofcode.blog

Weekend Coding: Daily Email Sending Through Code

#programming #beginners #tutorial #productivity

A fun end-to-end project, from requirements to the final product

A few weeks ago I went to visit my parents and we sat down for dinner. While we were catching up, my dad mentioned that every evening he goes on his computer, and sends my mom an email with several Excel files she has been updating that day, for backup. I almost choked on my food! Every evening? Manual backup? In 2023?!

My first optimization idea was to swap Excel for Google Drive’s Sheet. This idea was declined because Excel’s user experience is so much better, that it is non-negotiable. Then my next suggestion was to set up a Google Drive syncing directory on the desktop. This idea was rejected as well because apparently, my dad had a resiliency threshold so high, that he did not want to put all his eggs in one basket: He wanted a multi-company backup, and he is achieving it by sending the files to two different email addresses, hosted by two different email providers.

Now, let me be clear: I know that Google Drive is good enough for my mother’s precious spreadsheet files. This double-company backup approach is cracking a nut with a sledgehammer. With that said, this is what the customer wants, and I was excited to discover how to automate email sending, so we started laying down the requirements!

(Initial) Requirements

I was naive to think that we had everything covered at our first discussion. Later on, we realized that every little adjustment we decided to make along the way, resulted in additional, more specific, requirements, which I will cover later.

Now, we know the final goal is to have a daily email sent from my father’s email address to my mother’s email address but to be exact, we want:

Daily automatic email from X@hotmail.com to Y@gmail.com.
Easy manual send upon request as well.
Attaching multiple Excel files, while the names of files may vary.

Therefore, I had a few things to figure out, with which I could create the whole program:

Send an email through code.
Attach files to that email.
Extract the code to .exe file.
Automate a daily send.

You might wonder why step 3 is necessary. There are two reasons for that: First, I want the program to be user-friendly for non-developers (like my parents). Second, I want to hide the code because, as you will see soon, it contains a Gmail app password. I am assuming .exe files can be disassembled back to code, but at least this is an extra layer of protection.

Luckily, I wasn’t the first one on the internet trying to code email sending, even with files attached! Nor I was the first to extract a .exe file from Python, and I was definitely not the first to want to automate an action on Windows. I will link useful videos at the end of the blog post. Now, let’s get started!

1. Send an Email through Code

I will admit that when thinking of automation and scripts, the first thing that came to mind was the shell — using the command line. But a quick YouTube search showed Python is a more friendly way, and that is what I went with.

Whatever coding language you are choosing, there is one thing you’ll need to use in order to send out emails: SMTP, or Simple Mail Transfer Protocol. This is how you send email messages between servers. If you will be using a Hotmail email address, you will need to authenticate through the Hotmail SMTP server.

This is the first point where our initial plan shifted. Setting up an SMTP connection to Gmail, which is what I used, requires you to set up a 2-step verification in your account. A 2-step verification is verifying who you are with more than just your password (usually sending code through email/SMS). Higher security for your account is never a bad thing, but the fact that this is a requirement shows that connecting to an SMTP server using your email address credentials is compromising your account.

I want an email sent, and I don’t mind from where (well, this will come back to bite us, stay tuned ;) ) — so I can create a new account, one I don’t mind compromising its security, which only purpose will be my new cool software!

The way you set up your SMTP is by using app passwords. An app password is a 16-digit passcode that gives a less secure app or device permission to access your Account. Here are instructions for setting up an app password on Google.

Let’s start coding

If I’m being honest, the first thing I did was refresh my memory about working with environment variables file. Those are variables whose value is set outside of the program. This is a critical step because the customers will use their own email recipients and their own file paths, and they shouldn’t need to open the code in order to set those.

ENV_VARS.env:

shoe-maker.py:

So what do we see in the screenshots above? First of all, we see that I called my program shoe-maker.py, and you might wonder why. I will tell you at the end of this blog post.

lines 6–7 are for directing the program to the location of the .env file. By the way, you can call it whatever you want, but ENV_VARS is the standard.
lines 9–12 are saving the values we extracted from the file into local variables. Grabbing the values externally every time we want to use them is expensive.
Lastly, lines 1–4 are all the imports the IDE complained you are missing. ;)

Wait a minute — PASSWORD in the env vars?

Let’s reflect on that screenshot for a minute. Despite being worried about compromising the email account, the app password is out and about in the env vars file. How come?

This is because I was mixing up who my audience is. I knew I was going to write this blog post, and I wanted to give you (people from the software field) the software with which you can set up your own email sender (and app password).

The problem is, that this is a different goal than providing a final product aimed at “real” users. Real users shouldn’t be exposed to any password, and it’s better to have as few env vars as possible so that the user will have fewer opportunities to make oopsies in the configuration.

So, from this point forward, I am setting the email address and password within the code.

Straightaway, after figuring out the env vars, I stopped and moved to step 3. I did it because I wanted to ensure I could successfully export the .py file into a .exe file before I invested any additional time writing in Python. As I mentioned before, .exe file was an essential goal of this project, and I was willing to switch to another programming language if I couldn’t achieve it there.

3. Extract The Code to .exe File

Thanks to YouTube, I found this tutorial. Long story short, you need to install PyInstaller. Pretty easy to do, I’ll just mention that for me, when I used the regular pip install pyinstaller I got an error (CommandNotFoundException) and was able to bypass it with py -m pip install pyinstaller.

Could I’ve gone to figure out how to make pip the command work directly without the py -m? Yes. Was I going to do it while I was in the fun flow of my email-sending program? Nope. :D

Speaking of errors, when I was trying to verify the installation went smoothly with the -v flag, I got an “I don’t know what are you talking about” error, and it took me too long to realize it was because it requires a camel case in the word PyInstaller:

Once you have it installed, you can create the .exe file by running py -m PyInstaller --onefile --icon=image.ico my_file.py. This way you create a my_file.exe file with the icon of your choice, using the code from my_file.py. Note that it has to be a .ico file. I used this website to convert my image from jpg. After a few seconds, you’ll see a new dist folder in your directory with your new .exe file.

Now that step 3 is behind us — we can continue with our code!

Back to Sending an Email Through Code

Sending a simple email

To focus on email sending, I temporarily set the variables in the program, instead of moving between files. You can see that in line 2 in the screenshot below. Actually, at this stage, I didn’t even include files — I just wanted to see that I could send the most simple email.

Lines 4,17 are for catching exceptions, which I will touch on later.

I used the email-sending code from this video. You might noticed I renamed my sender email from EMAIL to GMAIL_ADDR. This is a good reminder that the code is set up to use the Gmail server (line 5). If my program was meant for people from the software world to use, setting up their own server (that might not be Gmail), I wouldn’t have hard-coded it. But for my use case, even if I have more users within my family, Gmail is good enough.

So, I started the connection (lines 6,7), logged in with the credentials (line 8), wrote a simple email (lines 10–13), and sent it (line 14). Lastly, on line 16, I notified the software user that the email was sent successfully (if we were on the happy path).

This is what it looks like on the recipient’s side:

Sending email to multiple recipients

This is nothing complex, I am just showing the order in which I progressed. Sending the email to multiple recipients is easily done by a simple for loop, over the ‘recipients’ variable, which is an array of email addresses (strings).

Testing usage of env vars file

I know the pieces work well separately, but it doesn’t guarantee they will work well together too. So, at this stage, the program runs smoothly when I set the recipients in the env vars file, but later on, the env vars are going to cause one of my biggest bugs. Any guess at which point?

Error handling

I know that adding error handling is a bit excessive, considering I only plan to have between 1–2 customers (who are obligated to love me unconditionally). What can I say? I have a standard for coding, and I can’t half-bake it. Now that I think about it, graceful failure is important in this case because the user might update the env vars file incorrectly and will need some guidance in order to fix it.

So, what do we have here?

In lines 3,5 I added colors because I wanted to differentiate between the error message and the instructions for the user.
In the instructions (lines 6–8), I informed the user that this process failed and that they would need to send an email themselves. Also, I invited them to contact support, and lastly — to close the program.
I split the instructions into 3 lines because it’s easier to read for the developer (rather than to scroll right).

The reason I added the input() in the end is so that the user will have as much time as they need to read the messages.

I’ve added more error handling later on, but let’s continue the story in chronological order.

Refactoring

This refactoring is coming from the tutorial I linked earlier. We have two changes to make:

First, If you swap line 10:
with smtplib.SMTP(‘smtp.gmail.com’, 587) as smtp: with
with smtplib.SMTP_SSL(‘smtp.gmail.com’, 465) as smtp:
then you can remove lines 11,12 because they now happen behind the scenes.

What exactly is happening behind the scenes? SMTP_SSL connects to the server using a secure encrypted SSL protocol (SSL stands for Secure Sockets Layer,), while SMTP doesn’t, which is why we need to add the starttls() function right after connecting. ehlo() function is for the server to say hello and identify :)

Note that the different connections use different ports (587 vs. 465).

The second refactoring is happening in the message content: Swapping
msg=f’Subject: {subject}\n\n{body}’
in line 18 with a more comfortable structure. This is done by using EmailMessage import.

Using the new import, we set the message object like this:
msg = EmailMessage()
msg[‘Subject’]=subject
msg.set_content(body)

The most important thing in refactoring is checking we didn’t botch anything that used to work fine :D

Still working! Now we can progress to step 2.

2. Attaching Files to The Email

Let me remind you, that the whole purpose of this project was to send Excel files. Adding an attachment to the email is pretty straightforward: Given the file path, you gather the variables needed for the msg.add_attachment(…) function:

And for multiple files — just wrap this code in a loop.

That’s cool if we have the paths in the correct form. Unfortunately, it turns out that gathering the paths from the env vars file wasn’t as easy as I thought it would be:

Encountering My First Major Bug

What is this error saying? The error is rising from the beautiful ast.literal_eval(..) function. This function parses strings representing types, into the actual type (here we use it for arrays). In this case, the function can’t parse my file paths string from the env vars file because of Unicode escape errors: My file path location (C:\Users\…) triggers the function to convert the string to the format of a 4-digit hexadecimal code point (for example, “ABC” becomes “\u0041\u0042\u0043”), which makes the path absurd.

One solution for this problem is adding the r prefix to all the paths (r"myfriend@gmail.com"). This r stands for “raw”, and with it, the string will be read exactly as it is written. I didn’t want to go that route because I don’t think it’s a good practice to “force” the user to adjust to my program's limitations. The users should add file paths as they see it in the file explorer and the conversion to the right format should be done inside the code.

With that in mind, this is the solution I wrote:

First, I save the env var as a string (including brackets, commas, ditto marks (“), and white spaces). Then, I removed the brackets and split the string by the commas. Now I have an array of strings, but they still begin and end with ditto marks. This brings me to my final line of code: Strip each of the strings of any ditto marks and white spaces. In case you wondered, I’m handling white spaces because some people like to write lists with white spaces after the commas.

Note that the paths’ backslashes have converted from \ to \ automatically, in order to avoid those escape characters I mentioned before.

So far we have sent an email, added attachments, and exported the program to .exe… At this point, we can put the .exe file on the desktop and send emails at the click of a button! All we have left to do is to automate that task.

4. Automate a Daily Send

Luckily for us, Windows has a program called Task Scheduler. It’s already on your laptop! It took me a bit of learning to understand how to get it to do what I wanted it to do, but I made it :D

This is what you need to know: First, it is good practice to create a directory for your tasks (“My Tasks”) because other scheduled tasks are running on your computer and you don’t want to mess with them!

Once you are in your new directory -> click the Create Task button.

Create a new task

Now you will see a window with 5 tabs, we will go over them one by one:

General: Give the task a good name, and tick off the “Run whether the user is logged on or not”. This is because we want the task to run daily, regardless of whether you are on your laptop or not.
Triggers: What will initiate our .exe file? The time of the day. Choose your most comfortable hour, set it to “daily”, and don’t forget to tick off the “Stop task if it runs longer than” option. Our program only takes a few seconds and if it’s running longer it means it’s stuck — and we should kill it.

Actions: This is the juicy part! Here you put the path to your .exe file. Let me save you some precious time, and tell you this is where the program, as it is now — will fail.

The issue is that the task scheduler doesn’t run the program from the same directory of the file. This is what I mean: Instead of running the command like this
C:\Users\ Desktop\shoe-maker > py .\shoe-maker.py

Imagine running it like this:
C:\Users> py .\ Desktop\shoe-maker \shoe-maker.py

We coded the env vars file location relatively, and we don’t know where the task scheduler is executing the program from, therefore — we need to add the full path as an argument.

Conditions: It’s important to tick off the “Wake the computer to run this task” option so that the automation doesn’t depend on the user’s laptop usage.

And that’s it! I recommend setting it first to 1 minute instead of 1 day just to test that it works :)

Are we done now? Not exactly.

5. Privacy Gap Discovered

Remember earlier, when I easily made the decision to open a dedicated Gmail account, in order not to compromise any existing Gmail account’s security (due to usage of app passwords)?

It looked like a no-brainer solution, and I didn’t realize that using an external Gmail account meant exposing the data (the precious Excel files) to anyone with access to this account (like me!)

Did you notice how WhatsApp keeps reminding you that your messages are end-to-end encrypted, and your message content stays between you and your conversation partner? I aim to provide the same.

Unfortunately, I couldn’t find a way to do so. This is a compromise I’m not happy about, but I still prefer that over risking an actual active email address that has gathered years of data.

A nice settlement had been setting on Gmail a filter to send every email that is not part of the development (aka not “from me to me”) directly to “Trash”. Messages will sit there for 30 days and then will be automatically deleted. In the real world, this is not a good enough solution, but my 2 customers were okay with the terms and conditions :)

6. Final Touches

Now that we have the functionality, we just need to add some final touches, to make it cool:

Icon: That is the first thing I added, as soon as I set up the .exe export, but maybe for you, this step is coming only now.

Meaningful email title and body:

I started by creating a new env var, to let the user title the email with something meaningful. In addition to that, I created a default title, in case they don’t want to set it up (lines 2,4).
On line 6 I set a fun email body.
On lines 8–16, I used the option to add alternative text, in HTML format.

Additional error handling: First, I considered what errors the user might be creating. Then I caught them, and added instructions:

Once you’ve added the code to print the error, it’s pretty easy to start covering specific cases: Make a mess and copy the error that comes out. Then, add it as a case in the catching :D

Adding a progress bar: It’s nice to keep the user in the loop and show them the progress. Why should we message the user only when things are bad? ;)

7. Developer Troubleshooting

Here is a short list of errors you might encounter that didn’t come up in the blog post flow:

(334, b’UGFzc3dvcmQ6'): This is an SMTP authentication error, and it comes up when the email or password you entered is incorrect. The reason behind it might be a typo in the actual email and password, but it can also be a typo in the env vars variable names for those fields! So watch out for that.
PermissionError: [WinError 5] Access is denied: exe window was open: This one happens when you try to create a new .exe version (which overrides the previous one), but you don’t exit the program. Is this the reason I added the You may exit this program. instruction to my error handling? Maybe :)

Why Shoemaker?

First of all, I’m glad that you made it this far in the blog post! Hopefully, this inspired you to start your own little project. It surely made me :D

I named this project The Shoemaker because of the saying “The shoemaker’s children go barefoot”. I found it fitting to the scenario of the software engineer's parents doing daily tasks that can be automated via code. ;)

Useful links:

If you read this, you’ve reached the end of the blog post, and I would love to hear your thoughts! Here are the ways to contact me:
Facebook: https://www.facebook.com/cupofcode.blog/
Instagram: https://www.instagram.com/cupofcode.blog/
Email: cupofcode.blog@gmail.com

DEV Community