System Requirements#
Apache Spark cluster on
Azure/AWS Databricks (Preferred)
Amazon EMR
Cloudera/Hortonworks
Azure HDInsight
Google Dataproc
On-Prem Infrastructure
BDT 3.0 Supported Versions |
BDT 3.1 Supported Versions |
BDT 3.2 Supported versions |
BDT 3.3 Supported versions |
BDT 3.4 Supported versions |
|
|---|---|---|---|---|---|
Spark |
3.2.1 (Scala 2.12) |
3.3.0 (Scala 2.12) |
3.4.1 (Scala 2.12) |
3.5.x (Scala 2.12) |
3.5.x (Scala 2.12) |
Databricks |
DBR 10.4 LTS |
DBR 11.3 LTS |
DBR 12.2 LTS, DBR 13.3 LTS |
DBR 14.3 LTS |
DBR 14.3 LTS, DBR 15.4 LTS, DBR 16.4 LTS (Scala 2.12) |
EMR |
emr-6.6.0, emr-6.7.0 |
emr-6.8.0, emr 6.9.0 |
emr-6.12.0, emr-6.15.0 |
emr-7.0.0, emr-7.1.0 |
emr-7.6.0 |
BDT 3 Supported Databricks Runtime Versions
BDT 3.0 |
BDT 3.1 |
BDT 3.2 |
BDT 3.3 |
BDT 3.4 |
|
|---|---|---|---|---|---|
DBR 10.4 LTS |
☑ |
☐ |
☐ |
☐ |
☐ |
DBR 11.3 LTS |
☐ |
☑ |
☐ |
☐ |
☐ |
DBR 12.2 LTS |
☐ |
☐ |
☑ |
☐ |
☐ |
DBR 13.3 LTS |
☐ |
☐ |
☑ |
☐ |
☐ |
DBR 14.3 LTS |
☐ |
☐ |
☐ |
☑ |
☑ |
DBR 15.4 LTS |
☐ |
☐ |
☐ |
☐ |
☑ |
DBR 16.4 LTS (Scala 2.12) |
☐ |
☐ |
☐ |
☐ |
☑ |
BDT 3 Supported Databricks Default Policies and Access Modes
Unrestricted |
Power User Compute |
Personal Compute |
Shared Compute |
Legacy Shared Compute |
|
|---|---|---|---|---|---|
Single User |
☑ |
☑ |
☑ |
N/A |
N/A |
Shared |
☐ |
☐ |
N/A |
☐ |
☐ |
No Isolation Shared |
☑ |
☑ |
N/A |
N/A |
N/A |
Note: This table only covers the default policies available in Databricks. In general, shared access mode and any policy that forces the user to use shared access mode is not supported by BDT at this time. For more on policies and access modes see the Databricks documentation.
