DEV Community

# etl

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Why Is Spark Slow??

Why Is Spark Slow??

Comments
3 min read
Data Pipeline Filters 101: Choosing Between Static and Dynamic Approaches

Data Pipeline Filters 101: Choosing Between Static and Dynamic Approaches

Comments
1 min read
Ensuring Deployment Accuracy with air sandbox diff in AbInitio

Ensuring Deployment Accuracy with air sandbox diff in AbInitio

Comments
3 min read
AbInitio Automation: How We Reduced 80% of Incidents Due to Connection Failures

AbInitio Automation: How We Reduced 80% of Incidents Due to Connection Failures

Comments
3 min read
*Mastering Informatica Intelligent Cloud Services (IICS) for Cloud Data Integration*

*Mastering Informatica Intelligent Cloud Services (IICS) for Cloud Data Integration*

1
Comments
3 min read
Unlocking Data Potential: My Journey with .NET ETLBox

Unlocking Data Potential: My Journey with .NET ETLBox

11
Comments
2 min read
Presenting at DataEngBytes 2024 Sydney: Building a Transactional Data Lakehouse on AWS with Apache Iceberg

Presenting at DataEngBytes 2024 Sydney: Building a Transactional Data Lakehouse on AWS with Apache Iceberg

Comments
4 min read
Building Scalable data pipelines ;Best practices for Modern Data Engineers

Building Scalable data pipelines ;Best practices for Modern Data Engineers

2
Comments
14 min read
5 Best ETL Tools: A Comprehensive Comparison Guide

5 Best ETL Tools: A Comprehensive Comparison Guide

1
Comments
3 min read
Move Data from DynamoDB to Redshift Using Estuary

Move Data from DynamoDB to Redshift Using Estuary

Comments
4 min read
Understanding the ETL Process with Real-Time Data: Extraction, Transformation, Loading, and Visualization

Understanding the ETL Process with Real-Time Data: Extraction, Transformation, Loading, and Visualization

Comments
3 min read
Scalable ETL pipeline for Google Merchant XML Feed and RDS with AWS Glue

Scalable ETL pipeline for Google Merchant XML Feed and RDS with AWS Glue

Comments
4 min read
Ab Initio Automation: How We Reduced 80% of Incidents Due to Connection Failures

Ab Initio Automation: How We Reduced 80% of Incidents Due to Connection Failures

Comments
3 min read
Cogumelos Mágicos: explorando e tratando dados nulos com Mage

Cogumelos Mágicos: explorando e tratando dados nulos com Mage

Comments
6 min read
Optimize ETL Processes with Apache Iceberg: A Game Changer

Optimize ETL Processes with Apache Iceberg: A Game Changer

Comments
4 min read
Reduce ETL Time by Converting Sequential Code to Parallel AWS Lambda Execution

Reduce ETL Time by Converting Sequential Code to Parallel AWS Lambda Execution

2
Comments
2 min read
Mastering Data Routing in Apache Camel: Leveraging the Splitter Pattern

Mastering Data Routing in Apache Camel: Leveraging the Splitter Pattern

1
Comments
7 min read
Exploring Core Features and Components of Apache Camel

Exploring Core Features and Components of Apache Camel

2
Comments
8 min read
Practical Guide to Apache Camel with Quarkus: Building an ETL Application

Practical Guide to Apache Camel with Quarkus: Building an ETL Application

4
Comments
6 min read
Speeding Up Data on AWS: From Ingestion to Insights

Speeding Up Data on AWS: From Ingestion to Insights

4
Comments
11 min read
How I contributed my first data pipeline to the open source.

How I contributed my first data pipeline to the open source.

1
Comments
3 min read
On Orchestrators: You Are All Right, But You Are All Wrong Too

On Orchestrators: You Are All Right, But You Are All Wrong Too

1
Comments
10 min read
What is the REST API Source toolkit?

What is the REST API Source toolkit?

1
Comments
7 min read
MongoDB to SQL Server Migration in 5 Steps

MongoDB to SQL Server Migration in 5 Steps

3
Comments
3 min read
Ways to load data in DW from External Data Source

Ways to load data in DW from External Data Source

1
Comments
6 min read
Fivetran vs Airbyte vs Estuary: Data Integration Tools Showdown

Fivetran vs Airbyte vs Estuary: Data Integration Tools Showdown

1
Comments
3 min read
Mastering Database Merging: Comparing Different Approaches

Mastering Database Merging: Comparing Different Approaches

4
Comments
14 min read
Optimizing ETL Processes for Efficient Data Loading in EDWs

Optimizing ETL Processes for Efficient Data Loading in EDWs

Comments
4 min read
How Data Integration Is Evolving Beyond ETL

How Data Integration Is Evolving Beyond ETL

Comments
16 min read
Unlock the Power of C# in Polyglot Notebooks

Unlock the Power of C# in Polyglot Notebooks

5
Comments
7 min read
Simplifying SDMX Data Integration with Python

Simplifying SDMX Data Integration with Python

2
Comments
3 min read
A Comprehensive Guide to Extracting Data from MySQL Using Singer ETL

A Comprehensive Guide to Extracting Data from MySQL Using Singer ETL

Comments
2 min read
Practical Way to Use AWS Glue with Postgresql

Practical Way to Use AWS Glue with Postgresql

10
Comments
2 min read
Reverse ETL in Healthcare: Enhancing Patient Data Management

Reverse ETL in Healthcare: Enhancing Patient Data Management

Comments 1
4 min read
4 Types of ETL tools: Description, Pros & Cons, and Use Cases

4 Types of ETL tools: Description, Pros & Cons, and Use Cases

Comments
7 min read
From ETL to Modern Integration Platforms

From ETL to Modern Integration Platforms

Comments
4 min read
Demystifying:Azure Data Factory

Demystifying:Azure Data Factory

Comments
1 min read
Open Source High-Scale Data Pipeline Platform for Enterprise Data, Analytics, and Machine Learning Applications

Open Source High-Scale Data Pipeline Platform for Enterprise Data, Analytics, and Machine Learning Applications

Comments
2 min read
Best Practices for Designing an Efficient ETL Pipeline

Best Practices for Designing an Efficient ETL Pipeline

4
Comments
4 min read
Supercharge Data Insights: Harnessing AWS Glue for Advanced ETL in Healthcare and Life Sciences

Supercharge Data Insights: Harnessing AWS Glue for Advanced ETL in Healthcare and Life Sciences

3
Comments
3 min read
Top 5 Data Integration Tools for Modern Data Pipelines

Top 5 Data Integration Tools for Modern Data Pipelines

1
Comments
3 min read
Cómo Crear tu Primer Data Warehouse: Una Guía para Principiantes

Cómo Crear tu Primer Data Warehouse: Una Guía para Principiantes

2
Comments
3 min read
Cost-Effective GPT API Usage with Datapipe

Cost-Effective GPT API Usage with Datapipe

Comments
3 min read
Embracing Zero ETL: Unveiling the Benefits

Embracing Zero ETL: Unveiling the Benefits

Comments
6 min read
Data Engineering Projects

Data Engineering Projects

Comments
1 min read
Streamlined Data Processing: A Guide to Cost-Effective ELT Implementation

Streamlined Data Processing: A Guide to Cost-Effective ELT Implementation

Comments
7 min read
The easiest way to navigate through MongoDB, PySpark, and Jupyter Notebook

The easiest way to navigate through MongoDB, PySpark, and Jupyter Notebook

9
Comments
3 min read
Building a Data Warehouse with ETLBox: A .NET Developer's Guide

Building a Data Warehouse with ETLBox: A .NET Developer's Guide

Comments
18 min read
Haunted data pipeline

Haunted data pipeline

1
Comments
1 min read
Redefining ETL: Data Flows Powered by C# (Part III)

Redefining ETL: Data Flows Powered by C# (Part III)

Comments
12 min read
Redefining ETL: Data Flows Powered by C# (Part I)

Redefining ETL: Data Flows Powered by C# (Part I)

11
Comments
11 min read
Data Processing with Elixir (Part 2)

Data Processing with Elixir (Part 2)

6
Comments 3
3 min read
Data processing with Elixir (Part 1)

Data processing with Elixir (Part 1)

9
Comments 5
4 min read
A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes

A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes

6
Comments
9 min read
How to check for quality? Evaluate data with AWS Glue Data Quality

How to check for quality? Evaluate data with AWS Glue Data Quality

8
Comments 1
10 min read
Data Engineering (Part 02)

Data Engineering (Part 02)

6
Comments
3 min read
Improving ETL jobs on AWS with sparksnake

Improving ETL jobs on AWS with sparksnake

4
Comments 1
4 min read
How I Decreased ETL Cost by Leveraging the Apache Arrow Ecosystem

How I Decreased ETL Cost by Leveraging the Apache Arrow Ecosystem

Comments
6 min read
Moving data from MongoDB to PostgreSQL using AWS Glue: A Guide

Moving data from MongoDB to PostgreSQL using AWS Glue: A Guide

2
Comments
2 min read
Data Masking

Data Masking

5
Comments
1 min read
loading...