My Final Project
Finally finished my last semester of my degree, this article is about my final year project (FYP). It is an mobile application that allows users to search for the web crawled e-books from a few websites. All the e-books are free to download. There are also recommendation systems to recommend users some e-books.
How I built it ?
There are 4 main components for this FYP as shown below:
Web Crawler
3 e-books websites are being scraped by using the following Python technologies:
- Scrapy
- Scrapyd
- Pymongo
Mobile Application
The mobile application is being built with Flutter. Here are just a few UI designs.
Back-end for Mobile Application
The back-end is used to handle the actions of the mobile application such as login, register, search for books and etc.
It is built with Node.js + MongoDB.
Recommendation Systems
The recommendation systems are being built with the following Python technologies:
- flask
- pandas
- numpy
- gensim
- sklearn
There are something one might need to take care for is the cold start problem of the recommendation system. In order to solve it, the users will be prompted to select e-books and genres that they like.
There are 2 recommendation systems, the first one is a content-based recommender after users have selected their favourite e-books and genres.
The 2nd one is being used for collaborative filtering (CF) when users have rated a few e-books. The 2nd one is based on k-NN algorithm.
Website
I've also created a website with React to promote this application and for users to download the apk file.
Note: The website may take quite a while to load due to inactivity.
New Updates: The backend server has been shut down to save money, so the application is not working anymore.
Additional Thoughts / Feelings / Stories
In short, I am really happy that I am able to complete my FYP. Hopefully that I could get a really good grade for it.
Thank you for reading !!
Top comments (1)
Wow! Congrats on wrapping up your final year project! Your e-books web crawling app with a recommendation system sounds awesome. Using Scrapy, Scrapyd, and Pymongo shows serious tech skills. If you're up for it, Crawlbase could add some extra oomph to your project with its proxy rotation feature.