It all started about a year ago. I have met with my younger brother Paweł. He had this idea "that would make us rich" and asked if I could implement it. He is really into sports betting, he claimed he has a system for finding when the underdog team is about to score the next goal. The concept was straight forward - the app would take the data from live football events and analyze them. If the stats are right, it sends an email.
Looking at the data he had gathered, I knew that it was not enough to tell how accurate this system is, but I thought "that's a great learning opportunity!" so I wanted to give it a try anyway.
That's how I began working on a non-work related project "requested" by my brother. (If you thought based on the title that I would be doing something that my brother was supposed to do for his school/studies, I am sorry to disappoint you)
I wanted to share some lessons I have learned on the way. The list is very subjective and most of the stuff may sound obvious (but we do not always go with obvious things, do we?). Still, I hope some of you will find it beneficial:
- Requirements WILL change - "what do you mean it's not working right? I have implemented it the way you have described it the docs you have shared" "right... but it was supposed to take the stats only from the second half". So what do you do? I really liked the recommendations given by Uncle Bob in his book "Clean architecture" - divide the app into modules based on their abstraction level and note the flow of control. Our app was scraping the stats from an external betting website. When they changed the design, I needed to change only one class with scraping logic to make it work again.
- Follow good practices very early - by using linter and taking care of high test coverage from the start you save yourself some work.
- Try out a new technology - that was my first experience with Bitbucket pipelines, AWS EC2 and AWS CodeDeploy, as well as Selenium for web scraping. I feel like I have learned a lot.
- Track your time - I used Toggl for measuring the time I have spent on this project. I feel like it helped me focus on the task, but also be a little better at planning and estimating my work.
- When you're scraping - don't get banned! - You probably know that your app should not send many HTTP requests in a short time. I knew that from the start, but then the instance with our app would get banned. Why? The app tried to refresh the external website when the elements are missing. Imagine, what had happened if that website was under heavy load! So be aware of it on every level of your app (Captain obvious strikes again!). There is more to web scraping than just the interval of sending requests: I recommend the artice if you want to learn more.
- Track your budget - it refers to cloud services. Can you use a free trial for your project? When is it going to end? How much a month would cost me? You can also use a pricing calculator when you're not sure. I highly recommend setting up limits/alerts on your account. Oh, and you can always create a new Google Cloud or AWS account when you want to try something new in the free trial.
Our app turned out to be pretty inaccurate in predicting the next team to score (Tossing a coin would give us similar results...), so we have decided to terminate the instance recently. Even though the app did not make us rich, I feel satisfied that I took my time and I was there through the whole lifecycle of this "product".
Since the app is unlikely to be monetized now, I put the code on Github. Feel free to play around if you would like to.
Thanks for reading!
Top comments (0)