Press "Enter" to skip to content

Improvements in Spark 3.0

Alex Woodie covers some of the improvements available to us in Apache Spark 3.0:

The addition of join hints further enhances the accuracy of the compiler when the built-in algorithms deliver a suboptimal plan. “When the compiler is unable to make the best choice, users can use join hints to influence the optimizer to choose a better plan.”

Ah, join hints—the double-edged sword.