Introduction
In this blog, we're going to see how we can get horoscopes for all the zodiac signs for different days using BeautifulSoup.
Required Libraries:
- requests: Requests allow you to send HTTP/1.1 requests extremely easily. The module doesn't come pre-installed with Python, we need to install it using the command:
$ pip install requests
- bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. The module doesn't come pre-installed with Python, we need to install it using the command:
$ pip install bs4
Fetch Horoscope
We'll be using requests and bs4 to scrape horoscope data from this website. The requests module allows us to send HTTP requests using Python. The HTTP request then returns a Response Object with all the responses of the webpage mentioned. We then use the BeautifulSoup library to pull data out of HTML and XML files.
If you open the website, you will see a similar webpage opened. If you click on any of the zodiac signs (suppose Capricorn), the URL of the page will be like: https://www.horoscope.com/us/horoscopes/general/horoscope-general-daily-today.aspx?sign=10. The URL has two variables, if you see clearly, sign and today. The value of variable sign will be assigned according to the zodiac sign. The variable today can be replaced with yesterday and tomorrow. If we want to find the horoscope for a custom date, click on the calendar symbol and choose any date before yesterday.
The URL of the webpage will be like this: https://www.horoscope.com/us/horoscopes/general/horoscope-archive.aspx?sign=10&laDate=20211201. We have the same sign variable here too. Instead of today , yesterday or tomorrow , we have a variable laDate which takes the date in YYYYMMDD format.
Let us create a function called find_horoscope to get the horoscope data. This function takes two parameters - zodiac_sign(int) and day(string).
import requests
from bs4 import BeautifulSoup
def get_horoscope(zodiac_sign: int, day: str):
if not "-" in day:
res = requests.get(f"https://www.horoscope.com/us/horoscopes/general/horoscope-general-daily-{day}.aspx?sign={zodiac_sign}")
else:
day = day.replace("-", "")
res = requests.get(f"https://www.horoscope.com/us/horoscopes/general/horoscope-archive.aspx?sign={zodiac_sign}&laDate={day}")
soup = BeautifulSoup(res.content, 'html.parser')
data = soup.find('div', attrs={'class': 'main-horoscope'})
return data.p.text
If you see inside the function, we are using an if condition to check if the day is today , yesterday, or tomorrow. If it is, we make a GET request on https://www.horoscope.com/us/horoscopes/general/horoscope-general-daily-today.aspx?sign=10. Else first, we change the date from YYYY-MM-DD to YYYYMMDD format. Then we make a GET request on https://www.horoscope.com/us/horoscopes/general/horoscope-archive.aspx?sign=10&laDate=20211201. After that, we pull the HTML data from the response content of the page using BeautifulSoup. Now we need to get the horoscope text from this HTML code. If you inspect the code of any of the webpage, you'll find this:
soup.find() function to extract the paragraph text string, and return it.
Main function
As we know, we are using an integer for the zodiac sign. So, first of all, we create a dictionary to store all the zodiac sign’s strings as key and a specific number as their value. Then we ask the user to enter its zodiac sign and day, and then we use the get_horoscope() function to print the horoscope data.
if __name__ == ' __main__':
zodiac_signs = {
"Aries": 1,
"Taurus": 2,
"Gemini": 3,
"Cancer": 4,
"Leo": 5,
"Virgo": 6,
"Libra": 7,
"Scorpio": 8,
"Sagittarius": 9,
"Capricorn": 10,
"Aquarius": 11,
"Pisces": 12
}
print(*zodiac_signs.keys(), sep=", ")
zodiac_symbol = input("Which is your zodiac sign from the above? : ").capitalize()
day = input("For which day do you wish to know the horoscope? For today, write 'today', for yesterday, write 'yesterday', for tomorrow, write 'tomorrow', else write the date in YYYY-MM-DD format : ")
print(f"Getting horoscope for {zodiac_symbol} for {day}\n")
print(get_horoscope(zodiac_signs[zodiac_symbol], day))
When we run the file, we'll get a similar output:
$ python main.py
Aries, Taurus, Gemini, Cancer, Leo, Virgo, Libra, Scorpio, Sagittarius, Capricorn, Aquarius, Pisces
Which is your zodiac sign from the above? : capricorn
For which day do you wish to know the horoscope? For today, write 'today', for yesterday, write 'yesterday', for tomorrow, write 'tomorrow', else write the date in YYYY-MM-DD format : today
Getting horoscope for Capricorn for today
Dec 6, 2021 - Things should be going your way, and you may feel like quite a bit is getting done without your having to lift a finger. At the same time, a nagging voice in the back of your mind is telling you to watch your back. Have all your bases covered. While you may be tempted to go with the most comfortable and familiar, this is a good time to consider other perspectives.
With custom date:
$ python main.py
Aries, Taurus, Gemini, Cancer, Leo, Virgo, Libra, Scorpio, Sagittarius, Capricorn, Aquarius, Pisces
Which is your zodiac sign from the above? : capricorn
For which day do you wish to know the horoscope? For today, write 'today', for yesterday, write 'yesterday', for tomorrow, write 'tomorrow', else write the date in YYYY-MM-DD format : 2021-12-01
Getting horoscope for Capricorn for 2021-12-01
Dec 1, 2021 - You may feel like a racehorse crammed in the starting gate. You're stuck in a small spot, stomping your feet, and anxiously waiting for the starting bell. You may feel powerless since you have no control over when the gate will open. Be patient. Don't waste all your energy fidgeting. The gate will open soon enough and you'll be up and running.
Full Code
Conclusion
In this blog, we learned how we can get horoscopes using BeautifulSoup and Python. You can find the entire code here.
Top comments (0)