DEV Community

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
"Shannon Was Right, But We Can Be Smarter: How ALEC Achieves 22x Compression on IoT Data"
zeekmartin profile

"Shannon Was Right, But We Can Be Smarter: How ALEC Achieves 22x Compression on IoT Data"

#iot #rust #dataengineering #shannon
Comments
3 min read
Apache Data Lakehouse Weekly: January 15–22, 2026
alexmercedcoder profile

Apache Data Lakehouse Weekly: January 15–22, 2026

#news #architecture #dataengineering #opensource
Comments
5 min read
Medallion Architecture 101: Building Data Pipelines That Don't Fall Apart
aawiegel profile

Medallion Architecture 101: Building Data Pipelines That Don't Fall Apart

#dataengineering #database #python #sql
Comments
11 min read
My Friday "Sanity Savers" (Software, Data & DevOps edition) 🛠️
bhoyee profile

My Friday "Sanity Savers" (Software, Data & DevOps edition) 🛠️

#devops #dataengineering #python #productivity
Comments 1
1 min read
Introduction to Linux for Data Engineers, Including Practical Use of Vi and Nano.
karen_langat_299784e2c330 profile

Introduction to Linux for Data Engineers, Including Practical Use of Vi and Nano.

#beginners #dataengineering #linux #tutorial
Comments
3 min read
Introducing `everyrow.io/dedupe`: An LLM-based approach to semantic deduplication
rafael_poyiadzi_85c2eed09 profile

Introducing `everyrow.io/dedupe`: An LLM-based approach to semantic deduplication

#showdev #data #dataengineering #llm
Comments
6 min read
Streaming Crypto Changes: A Practical Guide to Real-Time Data Pipelines with Debezium CDC
j_m47 profile
J M

Streaming Crypto Changes: A Practical Guide to Real-Time Data Pipelines with Debezium CDC

#dataengineering #data #docker
Comments
3 min read
Lakehouse? More Like a Lake + Warehouse Parking Lot
esproc_spl profile
Judy

Lakehouse? More Like a Lake + Warehouse Parking Lot

#architecture #data #database #dataengineering
5
Comments
10 min read
Why AI Models Fail in Production — Even When Accuracy Looks High
naanhe_gujral_c001233100f profile

Why AI Models Fail in Production — Even When Accuracy Looks High

#ai #machinelearning #devops #dataengineering
Comments
1 min read
🕸️ I Just Deleted My Scraper Boilerplate: Meet the "One-Liner" Crawler
siddhesh_surve profile

🕸️ I Just Deleted My Scraper Boilerplate: Meet the "One-Liner" Crawler

#python #opensource #productivity #dataengineering
Comments
3 min read
From Splicing Fibers to Scaling Clouds: My Journey to the AWS Community
maureen_chepkirui_03c48a2 profile

From Splicing Fibers to Scaling Clouds: My Journey to the AWS Community

#aws #cloudcomputing #dataengineering #fiberoptics
Comments
2 min read
Building Production-Grade Data Analytics Pipelines: A Real-World Case Study in Government Data
rosewabere profile

Building Production-Grade Data Analytics Pipelines: A Real-World Case Study in Government Data

#analytics #dataengineering #datascience #privacy
Comments
9 min read
Building an Automated Data Pipeline
maureen_chepkirui_03c48a2 profile

Building an Automated Data Pipeline

#aws #dataengineering #python #cloud
Comments
2 min read
SQL - PostgreSQL: Execution Order
yurilbrok profile

SQL - PostgreSQL: Execution Order

#sql #postgres #dataengineering #database
2
Comments
5 min read
The Three Phases of Data Pipelines
thanh_truong_a99577c6b879 profile

The Three Phases of Data Pipelines

#technology #dataengineering #systemdesign
Comments
4 min read
Architecture of a 6TB Media Pipeline: Engineering Real-Time Content at Bharat Drone Shakti
pranjal_csmedia profile

Architecture of a 6TB Media Pipeline: Engineering Real-Time Content at Bharat Drone Shakti

#architecture #dataengineering #performance #systemdesign
Comments
6 min read
Why Columnar Storage Makes Analytics Faster
quame_jnr1 profile

Why Columnar Storage Makes Analytics Faster

#analytics #database #dataengineering #performance
1
Comments
1 min read
Are Wide Tables Fast or Slow?
esproc_spl profile
Judy

Are Wide Tables Fast or Slow?

#database #dataengineering #performance #sql
5
Comments
4 min read
HOW TO GIT IT
shabex profile

HOW TO GIT IT

#luxdev #git #onboarding #dataengineering
Comments
3 min read
Tableau + Databricks at Scale: A Technical Guide for Managing 10,000+ Databases
ridwan_sassman_3d07 profile

Tableau + Databricks at Scale: A Technical Guide for Managing 10,000+ Databases

#analytics #architecture #database #dataengineering
Comments
5 min read
Pipelines, ETL, and Warehouses: The DNA of Data Engineering
qvfagundes profile

Pipelines, ETL, and Warehouses: The DNA of Data Engineering

#dataengineering #datascience #beginners #career
4
Comments
4 min read
How to Set Up GPG Keys for an Existing GitHub Account (Step-by-Step)
rose1845 profile

How to Set Up GPG Keys for an Existing GitHub Account (Step-by-Step)

#git #github #dataengineering #softwareengineering
Comments
2 min read
Making AI Data Flows Visible: Building an Open-Source Tool to Understand SaaS & LLM Data Risk
harris_bashir_ac60ba5fff3 profile

Making AI Data Flows Visible: Building an Open-Source Tool to Understand SaaS & LLM Data Risk

#ai #dataengineering #privacy #opensource
1
Comments
3 min read
An Introduction to Git: Concepts, Commands, and Workflows
edmund_eryuba profile

An Introduction to Git: Concepts, Commands, and Workflows

#github #git #dataengineering #githubactions
Comments
4 min read
Apache Iceberg & the Open Data Stack: Why the Lakehouse is Real in 2026
dataformathub profile

Apache Iceberg & the Open Data Stack: Why the Lakehouse is Real in 2026

#news #dataengineering #lakehouse #iceberg
Comments
8 min read
loading...