Overview of My Submission
The climate change is an important topic we should all care to protect our planet. So, I wanted to create a simple web application to explore how countries produce energy, if they rely on renewable energy or fossil fuels.
To visit my web application click here! Similarly to the TV show Our Planet, I wanted to call my app Our Energy because it is a resource we all should be care of!
Below you can see the homepage:
You can look the amount of MegaWatts of energy produced by a country. To find the countries in the Mongo Database I used Atlas Search service. Below you can see the values for my country, Italy:
Moreover, from the homepage you can explore the top 5 countries producing energy using a particular renewable source. You can choose among solar, wind, waves and tidal, and hydro. Below there is a screenshot for the wind energy:
Data preparation
The original dataset was in .csv
format. Firstly I removed many columns that were not necessary to my scope to reduce the size. Then I converted it to a JSON file which is easy to be uploaded to MongoDB Atlas.
Connecting to Atlas Cloud is as easy as connecting to any local database thank you to this official guide. To upload the dataset onto Mongo Cloud I used literally one function: insert_many()
. You can explore the full code here.
Atlas Search
To search data by country I had to match the input string with the country_long
field of the dataset. I used the Atlas UI to create an index with static mapping for this field. You can refer to this guide to create one.
Aggregation pipeline
The dataset consists of many power plants located in many countries. To retrieve a summary information for a certain country I exploited the power aggregation pipeline feature of MongoDB. Look at this code snippet that I use to find the 5 countries producing the biggest quantity of energy from a particular fuel
(oil, gas, wind, solar, etc..., renewable energy sources are considered fuels as well):
result = collection.aggregate([
{
'$match': {
'primary_fuel': fuel
}
},
{
# Accumulate capacity by fuel type
'$group':
{
'_id': '$country_long',
'totCapacity': {
'$sum': {'$toDecimal': '$capacity_mw'}
}
}
},
{
'$sort': {'totCapacity': -1}
},
{
'$limit': 5
}
])
A pipeline is made of many stages that performs different operations to extract the data we want from the database. The output of a stage is the input of the next stage.
- The first step find all the entries whose field
'primary_fuel'
matches the stringfuel
:
{
'$match': {
'primary_fuel': fuel
}
},
- This step groups the output of the previous step by country (using the field
country_long
, the country name) and accumulate the energy production of a power plant. Up to now, we know the amount of wind energy produced by each country.
{
'$group':
{
'_id': '$country_long',
'totCapacity': {
'$sum': {'$toDecimal': '$capacity_mw'}
}
}
},
- We sort the output in descending order
{
'$sort': {'totCapacity': -1}
},
- Eventually, we limit the output to only 5 results
{
'$limit': 5
}
And in the web app we can see the Top 5 countries producing that kind of energy. For instance, if you want to discover who are the 5 biggest producer of energy from waves and tydal are:
Conclusion
It was exciting to produce a simple web application from scratch to production using Python and MongoDB.
If you want to contribute to my project or propose some improvements, visit the GitHub repository.
Submission Category:
Choose Your Own Adventure
GitHub repository
š OurEnergy
OurEnergy is a web application to explore some meaningful data about the Global Power Plant Dataset. You can explore either how each country is producing energy or little charts comparing the top 5 countries that are distinguishing for producing green energy š±
š§ Tech Stack
- Python Flask
- MongoDB Atlas
- Heroku
š Run Locally
Preparation of the dataset on MongoDB
Environment variables:
export MONGODB_URI='<your connection string to MongoDB'
export GPPDB_PATH='path/to/dataset/csvfile' # Do not write the extension '.csv'
For example my GPPDB_PATH
env variable for the file ./data/global_power_plant_database.csv
is:
export GPPDB_PATH='./data/global_power_plant_database'
Finally you can run two scripts to convert the dataset to JSON and to upload it to MongoDB Atlas:
python db_preparation/convert.py
python db_preparation/upload.py --cloud
Run Flask app
FLASK_ENV=development FLASK_DEBUG=true FLASK_APP=app flask run
Deploy to production
Create a .env
file with the following variables:
export MONGODB_URI='<your
ā¦
Top comments (4)
Thank you! Indeed, my goal was to keep the design simple with no external frameworks. It was the hardest part for me because I almost never design web ui š
Really great and unusual design, and the topic is very relevant š„!
I appreciate it! I'd like to correlate this information with other data such as the global temperature change over the years
Good job !