Andrew Wang and Ray ChaingĀ note thatĀ a new Hadoop 3 alpha is now available:
YARN introduces the notion of opportunistic containers in addition to the current guaranteed containers. An opportunistic container is queued at the NodeManager waiting for resources to become available, and run opportunistically so long as resources are available. They are preempted, if and when needed, to make room for guaranteed containers. Running opportunistic containers between the completion of a guaranteed container and the allocation of a new one should improve cluster utilization.
There are a couple other new features, including support for Azure Data Lake Store.