InTowards AIbychakshu_salgotra·Oct 20Part XV — Adaptive Query Execution: What AQE Actually Rewrites at RuntimeSpark plans your query before it has seen your data. AQE is the engine’s admission that the planner was guessing — and the machinery that…A response icon1
DataPassion·Sep 9Apache Spark Interview Quickie #1 — Adaptive Query Execution (AQE)Crack FAANG-style questions!!A response icon1
The Data Forge·Jul 29Spark Tuning: Partitions, Memory, and Why Liquid Clustering Changes the Optimization GameDataForge Series: Databricks Data Engineer Associate - Part 8 of 16
Sumit Mittal·Jun 30From 45 Minutes to 21 Minutes: How Adaptive Query Execution (AQE) Saves Your Spark Jobs4 billion records with 10% skewed data and 3 cores sitting idle with 24 GB spill. Sort Merge Join shuffled 57.5 GB data. 2000 partitions…A response icon2
InTowards Data EngineeringbySriw World of Coding·Apr 9Data Engineer Interview Scenario Based Question : How to Optimize SQL Queries in DatabricksYou’ve written a “perfect” SQL query. You run it in Databricks… and it takes 3 minutes instead of 3 seconds.
