DEV Community

Tankala Ashok
Tankala Ashok

Posted on

My First Billion (of Rows) in DuckDB | By João Pedro

When you want to process 450Gb/1billion rows of data we think in all the directions like PySpark, Bigquery and etc. If someone says it can be processed with one Python package(DuckDB) without using/installing any fancy tools can you believe it? That’s what João Pedro did and explained in this article.

My First Billion (of Rows) in DuckDB | by João Pedro | May, 2024 | Towards Data Science

First Impressions of DuckDB handling 450Gb in a real project

favicon towardsdatascience.com

Top comments (0)