Databricks
Creating a connection
To connect a Databricks table to Telmai, you'll need to gather specific information from your Databricks instance. Follow the steps below:
Capture Connection Details.
You can find the JDBC connectivity details in the Databricks workspace console:
Navigate to Compute -> Cluster -> Cluster Name -> Configuration -> Advanced Options -> JDBC/ODBC.
Capture the following details:
Server Hostname
Port
HTTP Path
For more information, refer to the : Databricks ODBC and JDBC Drivers
Generate a Security Token
A security token is required to connect to the cluster remotely. Create this token from the Databricks workspace console:
Go to the top right corner and click on your User Name -> User Settings -> Access Token -> Generate New Token.
Capture the token created.
For detailed instructions, see theAuthentication for Databricks tools and APIs
Create a connection in Telmai
Once you have the necessary information, navigate to the "Connections" page, click +New Connection, select databricks, and enter the following properties:
Host: Cluster hostname
Port: Cluster port
HTTP path: HTTP path of the cluster
Token: API token
Read from UnityCatalog: A flag that enables spark job to read from UnityCatalog directly. This option will only work in case of using Databricks for Telmai-Spark jobs.
Only use "Read from UnityCatalog" if running Telmai jobs in Databricks and provide access to Telmai service account to the data being accessed. Otherwise, data scans will fail.

Connecting an asset
Once a connection is defined, you can start using it to create assets. To create assets, you will need:
Schema: Schema name (i.e., database name)
Catalog (Optional): Unity catalog name
Alternatively, you can run a SQL query

Last updated