When getting started with Azure Databricks for data processing and analytics, you need to create at least one cluster to get started. Check out the video for a quick overview of how to do this from the Azure Portal. I include a quick description of the options you have and an overview of what cluster management tabs are available after cluster creation.
The requirements to follow along in your own Azure account are:
- An Azure Account
- An Azure Databricks Workspace (14-day trial will work)
Here are the basic setting and I recommend for a test cluster (see video for explanations of all the UI options).
- Cluster Mode = Standard
- Pool = None
- Databricks Runtime Version = 6.3 (or latest)
- Enable Autoscaling = No
- Terminate After = 120 minutes (default)
- Worker Type = Standard_DS3_v2 (default)
- Workers = 2
- Driver Type = Same as worker
In future posts I’ll share how to create clusters from the command line or using a Python script and show a few more options that are not included in this video.