Hello wonderful people!
On the 16th of April, we are running the online Data Love Conference.
Data Love is the conference for Data Engineers, Data Scientists, and everyone who wants to dive into the data-driven world. Register for FREE!
With this event, we hope to build more personal connections between the audience and our lovely speakers. For that, we asked our speakers to share their ideas.
Let’s jump on a wonderful ride of the Data Love world and explore what our speakers have to say.
Our first incredible speaker is Jean-Yves Stephan, CEO, and Co-Founder at Data Mechanics.
JY is the CEO and Co-Founder of Data Mechanics, a hassle-free containerized data platform that abstracts away the complexities of Spark and infrastructure management. Prior to that, he was a software engineer and Spark infrastructure team lead at Databricks, growing their cluster-management capabilities from early days to the scale of launching hundreds of thousands of nodes in the cloud every day. JY is passionate about making distributed data technologies 10x more accessible and resource-efficient through automation.
What are you working on right now? What drew you to your company?
I'm the co-founder at Data Mechanics. We're a YCombinator backed startup building a cloud-native Spark platform for data engineers. It's a more developer-friendly and cost-effective alternative to platforms like EMR, Databricks, Dataproc, and so on. Prior to Data Mechanics, I was working as a software engineer at Databricks.
We created Data Mechanics because we feel like most Spark platforms put way too much burden on their users - choosing cluster sizes, instance types, managing the number of partitions, sizing memory to avoid errors, ... We think a cloud-native data platform should automate all these choices, make smart choices to make data pipelines stable and cost-effective, while letting data scientists and engineers focus on building models and pipelines, rather than the DevOps work.
Building such an ambitious platform from scratch on Kubernetes is full of technical challenges, but we're data geeks and we're having a lot of fun while doing it.
A lot of people are wondering about Data Engineers and Data Scientists, and the differences between them. What’s your favorite part about your role? What are you measured on? What do you expect when working in a tandem with Data Engineers/Scientists?
The role of "data engineer" and "data scientist" actually encompasses various responsibilities. Some people are called data scientists, but they can spend most of their time doing software engineering work - building complex code projects, building data pipelines to build their dataset, managing and monitoring machine learning models in production.
I think that increasingly, companies realize that the bulk of work does not come from building statistical models, but from building the software engineering machine around it, and this is why the software engineering role is catching in popularity recently with the data science role (which remains very popular and needed).
The best part about data engineering and data science is the broad spectrum of technical skills (and responsibility) involved in your work. As a data scientist or data engineer, you almost necessarily become a "full-stack" engineer, learning about different tools and techniques. You often have a bird eye view of what your company is working on and so you can have a huge impact on its business too.
We thank JY for the thoughtful answers!
At the conference he is going to speak on the topic: Cloud-Native Apache Spark - Spark on Kubernetes Is Now Generally Available.
If you want to attend Jean-Yves's talk and to discuss some questions “in person” you can join us on the 16th of April!
The lineup of speakers is incredible. Topics are diverse. Suitable for any level. Interesting Q&A sessions in Spatial Chat. New career opportunities.
Data is all around you.
Data Love@_datalove
.@MySQL is the World’s Most Popular OSS DB. It powers the most innovative companies including @Facebook, @bookingcom , @Uber, and @Tesla. It's available on-premises and as the MySQL DB Service on @OracleCloud. #DataLove is so happy to have support from 🐬 mysql.com15:31 PM - 07 Apr 2021
Register for free
Check out our Website
Join us!
Top comments (0)