Daniel Ciocirlan has a primer on query plans in Apache Spark:
Let’s go over some examples of query plans and how to read them. Let’s go back to the one we’ve just shown:
== Physical Plan == *(1) Project [(id#0L * 5) AS id#2L]
+- *(1) Range (1, 1000000, step=1, splits=6)We read this plan backwards, bottom to top:
Spark does have some UI components which make this a bit easier, but you’ll probably end up in a situation where you need to read it in this format.