Matt Collins shares some recommendations:
You can use Databricks for a vast range of applications these days. From handling streaming datasets, running Deep Learning models to populating data model fact tables with complex transformations, choosing the correct compute option can seem a lot like a stab in the dark followed by (potentially expensive) trial end error.
You can choose from an incredible range of configurations in the Databricks Compute User Interface.
This variety is comprised of the Virtual Machine (VM) category, size, availability and access mode (also referred to as family).
Determining the right compute choice for you could be answered by the classic answer “it depends”, but this guide aim to inform the decision-making process — both in terms of cost and performance.
Read on for more information.