Anubhav Tarar shows how to get an execution plan for a Spark job:
There are three types of logical plans:
- Parsed logical plan.
- Analyzed logical plan.
- Optimized logical plan.
Analyzed logical plans go through a series of rules to resolve. Then, the optimized logical plan is produced. The optimized logical plan normally allows Spark to plug in a set of optimization rules. You can plug in your own rules for the optimized logical plan.
Click through for the details.