InIntellytics SolutionsbyBrijesh Dave·2d agoSpark vs. DuckDB vs. DataFusion: A 100GB Transformation BenchmarkThe 100GB Challenge: Why Many Pipelines Are Over-ProvisionedA response icon1A response icon1
Vinicius Freitas·Sep 23Como o DataFusion processa dezenas de milhões de linhas em memória vetorizada?Neste Artigo irei apresentar o Apache DataFusion, uma introdução do uso da ferramenta para processar dados e a integração com o formato…
Alex Merced·Sep 11Apache Data Lakehouse Weekly: September 3–9, 2026Three things ran through the dev lists this week. Projects kept redrawing their own boundaries, with the Iceberg Rust DataFusion…
Arda Avci·Jul 19From DataFusion to Ballista: Making a Custom Data Source Survive the NetworkImplementing LogicalExtensionCodec and PhysicalExtensionCodec so a custom TableProvider can be rebuilt on a remote executorA response icon1A response icon1
Alex Merced·Jul 3Apache Data Lakehouse Weekly: June 24 to July 1, 2026The open lakehouse runs on a small stack of Apache projects, and this was a week where those projects spent most of their energy on the…
Alex Merced·Jun 25Apache Data Lakehouse Weekly: June 17 to June 24, 2026This week the lakehouse community spent most of its energy on two questions. The first is how the catalog grows from a table directory into…
Alex Merced·Jun 16Apache Data Lakehouse Weekly: June 9 to 16, 2026This was a week of votes that passed and arguments that opened. Iceberg shipped three formal decisions while debating how to cut read…
Shashidhar Reddy Sudi·Jun 13Apache Spark vs Apache DataFusion: The Modern Data Engineer’s Complete GuideWhen to reach for which engine — and how to combine both for maximum performance
Alex Merced·Jun 4Apache Data Lakehouse Weekly: May 28 — June 4, 2026The lakehouse projects spent this week doing two things at once. They lined up a remarkable stack of releases, with DataFusion 54.0.0 in…
Alex Merced·May 27Single-Node Data Engineering: DuckDB, DataFusion, Polars, and LakeSailFor the past decade, data engineering was synonymous with distributed clusters. If your dataset exceeded a few gigabytes, standard practice…
