DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Why Using FLOAT for Financial Pipelines is a Silent $100k Trap (and How PostgreSQL NUMERIC Saves Your Ledger)
arpitmbangre profile

Why Using FLOAT for Financial Pipelines is a Silent $100k Trap (and How PostgreSQL NUMERIC Saves Your Ledger)

#sql #datascience #database #dataengineering
Comments
2 min read
PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere
nbaubek profile

PyArrow, Explained: How Zero-Copy Actually Works, and Why It's Everywhere

#dataengineering #python #bigdata #pyarrow
Comments
10 min read
Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases.
gacherimutua profile

Understanding Apache Airflow 3.3: Key Improvements, Features Every Data Engineer Should Know, and Real-World Use Cases.

#dataengineering #airflow
Comments
3 min read
Mask, Hash, Tokenize, or Encrypt? Choosing the Right PII Protection for Your Pipeline
nbaubek profile

Mask, Hash, Tokenize, or Encrypt? Choosing the Right PII Protection for Your Pipeline

#dataengineering #security #dataprivacy #gdpr
Comments
6 min read
Snowflake Multi-Snowflake Multi-Environment Account Architecture: RBAC, Cost Governance, Identity Isolation & PII Masking Design
krish0502 profile

Snowflake Multi-Snowflake Multi-Environment Account Architecture: RBAC, Cost Governance, Identity Isolation & PII Masking Design

#snowflake #dataengineering #rbac #accesscontrol
Comments
17 min read
From Detection to Production: A PII-Safe Pipeline in Python and DuckDB
nbaubek profile

From Detection to Production: A PII-Safe Pipeline in Python and DuckDB

#dataengineering #duckdb #dataprivacy #security
Comments
6 min read
You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines
nbaubek profile

You Can't Protect What You Can't Find: Detecting and Classifying PII in Data Pipelines

#dataengineering #python #dataprivacy #gdpr
Comments
8 min read
Avro schemas for Kafka developers: the 10-minute practical guide
yuus_company profile

Avro schemas for Kafka developers: the 10-minute practical guide

#kafka #avro #dataengineering #tutorial
Comments
2 min read
AWS Has Two Iceberg REST Catalogs: What Each One Actually Serves
xbill profile

AWS Has Two Iceberg REST Catalogs: What Each One Actually Serves

#aws #iceberg #lakehouse #dataengineering
Comments
8 min read
The hardest label in our dataset is the one where nothing happened
inve_money_e95961f8489d28 profile

The hardest label in our dataset is the one where nothing happened

#datascience #dataengineering #analytics #sql
Comments
14 min read
Recursive CTEs, explained from first principles
sophiemwangi profile

Recursive CTEs, explained from first principles

#sql #cte #dataengineering #datascience
3
Comments 2
5 min read
I Analyzed 2,061 Public n8n Workflows and Found 92% of the AI Nodes Had Been Silently Replaced
adix65 profile

I Analyzed 2,061 Public n8n Workflows and Found 92% of the AI Nodes Had Been Silently Replaced

#ai #opensource #dataengineering #automation
Comments
4 min read
I built a hockey analytics pipeline that scrapes, models, and publishes itself every week — no server, no database
agesatony profile

I built a hockey analytics pipeline that scrapes, models, and publishes itself every week — no server, no database

#showdev #python #dataengineering #githubactions
Comments
4 min read
Azure Synapse Databricks: A Migration Playbook That Survives Production
firfircelik profile

Azure Synapse Databricks: A Migration Playbook That Survives Production

#databricks #azure #dataengineering #etl
Comments
9 min read
Best Dimensional Data Modeling Tools: erwin Data Modeler Features & Alternatives
rutadbm profile

Best Dimensional Data Modeling Tools: erwin Data Modeler Features & Alternatives

#data #database #dataengineering #tools
Comments
11 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.