DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra
karankumar profile

Architecting High-Performance Data Pipelines with Modern ETL | Spiral Mantra

#dataengineering #cloud #performance #bigdata
Comments
1 min read
🔬Public docker images Trivy scans as duckdb datas on Kaggle
adriens profile

🔬Public docker images Trivy scans as duckdb datas on Kaggle

#cybersecurity #dataengineering #jupyter #datascience
Comments 1
1 min read
Creating a new Airbyte connector from scratch
lgerthal profile

Creating a new Airbyte connector from scratch

#airbyte #data #dataengineering #python
Comments
4 min read
Building an Automated Bitcoin Price ETL Pipeline with Airflow and PostgreSQL
katumo profile

Building an Automated Bitcoin Price ETL Pipeline with Airflow and PostgreSQL

#dataengineering #data #apacheairflow
Comments
3 min read
Apache Airflow for Data Engineering: Best Practices and Real-World Examples
katumo profile

Apache Airflow for Data Engineering: Best Practices and Real-World Examples

#dataengineering #data #apacheairflow
5
Comments 1
7 min read
How to Optimize SQL Queries for Speed and Efficiency
bikodes profile

How to Optimize SQL Queries for Speed and Efficiency

#sql #sqlserver #database #dataengineering
1
Comments
5 min read
Software and Data Engineering: How They Power AI and Big Data
vaishali2 profile

Software and Data Engineering: How They Power AI and Big Data

#aidrivendataengineering #dataengineering #softwareengineering #dataengineeringsolutions
Comments
4 min read
Personal Picks: Data Product News (March 19, 2025)
sagara profile

Personal Picks: Data Product News (March 19, 2025)

#dataengineering
Comments
5 min read
Unlocking the Power of Delta Lake UNIFORM in Databricks
anil_kumarmoka_f93e418d0 profile

Unlocking the Power of Delta Lake UNIFORM in Databricks

#databricks #datamananagement #datasecurity #dataengineering
1
Comments
5 min read
Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments
anil_kumarmoka_f93e418d0 profile

Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments

#databricks #lakehouse #dataengineering #datascience
1
Comments
5 min read
Why Pi-Shaped Teams Matter in This AI Era
kannandreams profile

Why Pi-Shaped Teams Matter in This AI Era

#ai #softwareengineering #leadership #dataengineering
Comments
5 min read
How to Build a Career in Data Science: A Step-by-Step Guide
publish profile

How to Build a Career in Data Science: A Step-by-Step Guide

#science #datascience #career #dataengineering
Comments
3 min read
Data architecture stack
max_verstappen_c26d3a7c99 profile

Data architecture stack

#dataengineering #programming #softwareengineering #database
Comments
1 min read
[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables
sagara profile

[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables

#snowflake #dataengineering
Comments
7 min read
Study Notes 6.11-12: Kafka ksqlDB, Connect & Schema Registry
pizofreude profile

Study Notes 6.11-12: Kafka ksqlDB, Connect & Schema Registry

#dataengineering #dezoomcamp #kafka #sql
Comments
7 min read
Study Notes 6.15: Kafka & Flink Streaming
pizofreude profile

Study Notes 6.15: Kafka & Flink Streaming

#dataengineering #dezoomcamp #kafka #pyflink
Comments
5 min read
Study Notes 6.13-14: Kafka Streaming with Python & PySpark Structured Streaming with Kafka
pizofreude profile

Study Notes 6.13-14: Kafka Streaming with Python & PySpark Structured Streaming with Kafka

#dataengineering #dezoomcamp #kafka #pyspark
Comments
7 min read
Study Notes 6.5-6: Kafka Producer, Consumer & Configuration
pizofreude profile

Study Notes 6.5-6: Kafka Producer, Consumer & Configuration

#dataengineering #dezoomcamp #kafka #streaming
Comments
7 min read
Study Notes 6.7-10: Kafka Stream Basics, JOIN, Testing & Windowing
pizofreude profile

Study Notes 6.7-10: Kafka Stream Basics, JOIN, Testing & Windowing

#dataengineering #dezoomcamp #streaming #kafka
Comments
13 min read
Study Notes 6.3-4: What is Kafka & Confluent Cloud
pizofreude profile

Study Notes 6.3-4: What is Kafka & Confluent Cloud

#dataengineering #dezoomcamp #kafka #cloud
Comments
6 min read
Study Notes 6.1-2: Introduction to Stream Processing
pizofreude profile

Study Notes 6.1-2: Introduction to Stream Processing

#dataengineering #dezoomcamp #streamprocessing #kafka
Comments
3 min read
Real-Time Data Processing with PyFlink and Redpanda
grokker_f9bf83d79cb9beb6f profile

Real-Time Data Processing with PyFlink and Redpanda

#dezoomcamp #dataengineering
Comments
1 min read
🚀 SQL Query Optimization in 2025: 7 Techniques for Faster Performance
aleksei_aleinikov profile

🚀 SQL Query Optimization in 2025: 7 Techniques for Faster Performance

#sql #mariadb #database #dataengineering
Comments
1 min read
From DWH to Data Mesh: How Data Architectures Evolved to Meet Business Demands
aleksei_aleinikov profile

From DWH to Data Mesh: How Data Architectures Evolved to Meet Business Demands

#dataengineering #database #bigdata #datastructures
Comments
1 min read
Star Schema: The Data Warehouse Hack So Simple, Experts Hate It
spkibe profile

Star Schema: The Data Warehouse Hack So Simple, Experts Hate It

#datawarehouse #datamodelling #dataengineering
Comments
2 min read
loading...