In my country, we now have 02 Technical AWS Heroes (Serverless and Machine Learning), and this is really good for the diversity ...But there is a small problem to fix.
During some of our local AWS meetups or even offline, some of our members have been reaching out to me as they are confused about which path to follow, especially when they hear the value proposition of both the Serverless and the Machine Learning paths, and they all look very attractive. They want to know which to choose.
It is for this reason, I decided to write this article, to discuss it openly, because I believe there are many people out there who have these same doubts or would benefit from my opinion about these 02 paths.
Also, I would invite other experts to share their opinion in the comments section.
Now that we are all set, let us start our discussion.
So what is the root of this confusion?
When you look at the roots of Serverless and the movement behind it, you will see it is closely related to a Full Stack development career. If you take the time to analyze most tutorials on Serverless, you would languages like Nodejs, JavaScript, or TypeScript. At the base, the discussions would mostly be around Rest APIs, GraphQL, and Infrastructure as Code. This confirms the leaders of the movement are web developers right?
But is that truly what Serverless is?….. Do not worry. We will get to that later in this article.Now, when we look at Machine Learning and it is closely related to the data careers, the reason you see most of the tutorials in Python. Mostly the learning path here is for data science which employs statistics, data analysis, and modeling.
Is that truly what Machine Learning is ?…. Most probably, even though it depends on the company.
So,on the one hand, you need maths and statistics and a passion to generate insights for the business leaders (data science and machine learning), while on the other hand, you do not need much maths and statistics, but instead need to build highly responsive interactions with your application, having the customers UI/UX in mind(full stack developer).
What then is my opinion about all this?
Before the coming of e-commerce to the mainstream and the rise of AI, you could completely separate these 02 professions of Full Stack (or web development) and Machine Learning(or data science). Also, and for those who have the luxury of working in a large company, the link between these 02 might be irrelevant to you. Because in large companies being highly specialized in one sub-area in any of the 02 paths above is what is required.
But not all have the luxury of working in large companies, the majority of us are only finding opportunities in smaller companies and startups. In addition to that, AI has no signs of slowing down for one bit, with or without eCommerce.
So if we are all on the same page now, let me go with my opinion, and again other experts can always chip in with their comments.
Now the modern ecommerce example again
Let us imagine once more that you are trying to build an eCommerce website this year. If you leave it completely in the hands of core web developers would have a good interactive and responsive application with great UI/UX for the customer, but you would be missing out on harnessing the data customers are generating every second, to help generate you disruptive ideas to help move your entire business forward. This is because your typical web developer is good with the customer experience, but leveraging data does not come easily to them. While customers have been “spoiled” by Tech giant offerings nowadays, their fancy expectations from using smart apps like Amazon will not be met. No smart recommendation systems, no smart natural language processing to help with searches, autocomplete, etc. This is mainly because the system might not have been designed to accommodate these services in the first place since it does not come naturally to the core web developers.
Also, we all can imagine what will happen if we decided to leave this project in the hands of core data scientists or machine learning engineers. It would be highly inefficient right?
With very little satisfactory customer experience. This is because the data scientists would be more focused on generating insight for management. They normally do not have instincts for taking care of APIs, endpoints, and building interesting UI/UX. The first thing will be about how to collect and do some ETL to make the data ready for modeling. We all agree that the business would not survive without pivoting to something else.
Wait!!! Was the issue about Full Stack Vs Machine Learning?
No. The original problem was about Serverless Vs Machine Learning, and not about Full Stack (web development) vs AI (or Machine Learning).
That is actually the point. Even though the Serverless movement is supported by Full Stack or web developers who master Nodejs, Serverless in itself is not a profession as such, but instead a way of thinking. It is about designing your infrastructure to be more efficient in terms of cost, time, and reliability.
Serverless is where FaaS ( Function as a Service ) such as AWS Lambda is at the core of the compute, in the place of EC2 instances.
It is where Cloud native AI solutions like Rekognition are at the front, in favor of bringing your own dockerized model to ECR, and also where Sagemaker Serverless Inference is used to scale inference more flawlessly in favor of manually configuring these endpoints.
Based on the examples above, despite the fact that Serverless seems to be linked to full stack developers alone, we can comfortably talk of Serverless Machine Learning. Where we do machine learning, but with a twist of thinking Serverless in the design of the entire infrastructure. For example, we first think about using cloud-native models like XGBoost in Sagemaker or integrating the APIs for AI Services like Rekognition, Comprehend, etc. Think of substituting EC2 instances with event-driven lambda functions for model inferences. Think of leveraging Sagemaker Serverless Inference with the overall objective of leveraging the benefits of Serverless (such as micro-services design, reduced cost, faster time-to-market, and highly reliable systems), where developers focus more on their application and leave the dirty work to the Cloud.
It is true, that Serverless is not a magic bullet for all Machine Learning workflows, but it is still good to first think Serverless first, before deciding to look the other way.
So what is the conclusion about the path to take?
Since we now know that Serverless is a way of thinking, all this could still lead you to Serverless.
In the beginning, if you have to choose between going the web development route or the machine-learning route, there are many articles around this, but below are my high-level recommendations.
If you love maths, and statistics and are passionate about playing with data, you should start with the data science path, otherwise, if you love building systems that interact with customers and provide attractive UI/UX and responsive interactions then go for the Full Stack route. But never forget that even though Serverless lives best with Full Stack Developers, it can also live comfortably well with data scientists or machine learning engineers working in the cloud like AWS.
Wish you Good Data & Serverless Luck!!!
Top comments (0)