As an avid gamer, I was committed to acquire a next-generation console, the PS5. But, like many consumers, I found myself frustrated trying to find a PS5 on sale.

Walmart seemed like a viable option, but its site was infuriating with posted times and problems adding items to a cart and purchasing. Other retailers such as Amazon, GameStop, and Target did not have advertised times for which the units would go on sale.

Instead of aimlessly trying to search these major retailers sporadically throughout the day, let's create a bot to do it for us. My intention is to use Python, Selenium with Chromedriver, Heroku Scheduler, and Twilio to send my personal phone a message whenever a unit is available.

Prerequisites

• Python
• Git
• Selenium with Chrome Driver
• Twilio
• Heroku

Python

I created a stub repository for this project which anyone could use, given they supply valid keys to an environment file: python-flask-heroku-ps5-notifier. In order to deploy to Heroku, I recommend forking or Rambo cloning this repository.

To download and run the project, run the following commands:

git clone https://github.com/DEV3L/python-flask-heroku-ps5-notifier.git
cd python-flask-heroku-ps5-notifier
cp .env.local .env
python3 -m venv python-flask-heroku-ps5-notifier
source python-flask-heroku-ps5-notifier/bin/activate
python setup.py develop
pytest
python app.py runserver

The result of the pytest command should result in passing tests. With python app.py runserver you should be able to visit http://localhost:5000 in a browser to see a "Hello, world!" message.

If you got a successful result on both of these commands, congratulations, you are ready to move forward! The repository has .env in the .gitignore file, please do not check in any secrets into your own public repository. These secrets will be hidden and managed inside of Heroku.

Selenium with Chrome Driver

This can be a tricky step. In order to run headless on Heroku, we need to make sure our initialization of the Selenium Chrome Web Driver has appropriate path variables. Here is a short YouTube video that explains what we are trying to do, Running ChromeDriver with Python Selenium on Heroku .

On the server, the variables will be as the video describes. But, in order to test locally, we need to make sure the CHROMEDRIVER_PATH and GOOGLE_CHROME_BIN are set appropriately for your unique machine paths. If you are using a mac, BIN file location will not have to change. In the project, an instance of chrome driver for mac is available. Set the path to whatever this value is for you.

# Selenium
CHROMEDRIVER_PATH=/Users/dev3l/workspace/python-flask-heroku-ps5-notifier/drivers/chromedriver
GOOGLE_CHROME_BIN=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
HEADLESS=True
USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36

The USER_AGENT is configurable incase that needs to change in the future to accommodate any bot detection. In addition, if you want to see the magic happen (not headless) just change the value of HEADLESS to False and the browser will display. This helps out with debugging when scraping code has to change.

With your virtual environment active, run the following command:

python scrape_stub.py

If you see the message "Selenium with Chrome configured correctly", then Selenium is appropriately configured for your local machine. Let's move on to setting up Twilio.

Twilio

Setting up a free Twilio account was a bit tricky at first. Visit Twilio and sign up for a trial account. As long as you are sending messages only to verified numbers and ok with the message starting with "Sent from your Twilio trial account - ", you can get away with the free trial.

Create a new project, giving it any name you desire. Then from the All Products and Services menu (... on the left menu while logged in), select Programmable Messaging. Select Try it out and Try SMS..., at this point you should be able to acquire a free SMS number. Using the UI, verify that you can send a message to your validated phone number.

Once completed, update the variables in the test project to your appropriate values:

# Twilio
TWILIO_ACCOUNT_SID=TWILIO_ACCOUNT_SID
TWILIO_AUTH_TOKEN=TWILIO_AUTH_TOKEN
TWILIO_TO=+12316851234
TWILIO_FROM=+15555555555

Given you have updated these variables appropriately, run the following command to verify that Twilio is setup appropriately:

python scripts/twillio-message.py

If you received a "Hello, world!" text message, you are ready to move forward!

Heroku

If you do not have one already, sign up for a free Heroku account. Create a new app and give it a name, mine is ps5-notifications. Once created, go to the "deploy" tab and select GitHub as a Deployment method.

Search for and connect the repository. Be sure to press the button to enable automatic deployments. The project should automatically deploy and be available at your <project-name>.herokuapp.com, for example, mine is available at https://ps5-notifications.herokuapp.com/

Without any modification, you should be able to visit your site URL (Open app button at the top of the Heroku project dashboard). If this works for you, once again you are ready to move forward!

Update Heroku Environment Variables

Without any environment variables set, we are not going to be able to send messages to our target phone. Within the Heroku dashboard for your app, go to Settings and then click on the Reveal Config Vars button. Enter each of your specific four key-value pairs for TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN,
TWILIO_TO, TWILIO_FROM. Once completed, redeploy your app by going to the Deploy tab and select Deploy Branch from the Manual deploy section.

With this done, you should now be able to manually send yourself a text message through a URL in your browser. <project-name>.herokuapp.com/twilio will send a default "Hello, World!" text message, or you can add a message query string parameter to customize this message. I may regret putting this here, but you can hit my example at Send Message Example Link

If you received a text message on your phone, move forward!

Setting up Selenium with Chrome Driver in Heroku

Following the steps in the video linked above, or here, we need to add two build packs and two environment variables.

After this is done you should be able to run the scrape_stub.py file directly on your Heroku instance. If you don't have the CLI tools installed, this can be easily verified through the Heroku dashboard by clicking the More button then selecting Run console. Then, in the dialog box send python scrape_stub.py. If you received the same success message as from running the command locally, then congratulations! We have one more step to set up before our bot is ready to let us know when a PS5 is on sale.

Setting Up Scheduler Task in Heroku

The final step is to hook up the screen scraping code to be called on a regular interval. I assumed that the sites would be easy to bypass using various scraping techniques, but I found that this was not an easy task. Particularly Walmart, it seems that using Selenium and Chromium I could fake it out sometimes, with the assist of solving a captcha manually a few times, but it kept resetting itself so it would have to be checked. Fortunately, Amazon, BestBuy, and GameStop are not this sophisticated.

To set up the Heroku Scheduler, go to the Overview tab and select the Configure Add-ons link. Search for Heroku Scheduler and select Heroku Scheduler and accept the dialog box that comes up. Click on the now available Heroku Scheduler add-on and click the Create job button

Conclusion

With our bot deployed and complete, all there is left to do is wait.

I received a message from Amazon that a PS5 was for sale, but it was from a secondary seller for almost $1800. Still excited to see it working!

Given our totally free solution, we will be notified if Amazon, Best Buy, Game Stop, or Target has an available unit once every ten minutes. In the future, it would not be hard to add more sites or even Xbox Series X.