The Hadoop in Real World team walks us through resource allocation for Spark applications:
In this post we will look at how to calculate resource allocation for Spark applications. Figuring out how to allocate resources for a Spark application requires a good understanding of resource allocation properties in YARN and also resource related properties in Spark. Let’s look at both.
This post covers the properties you want to keep an eye on when running Spark applications.