Databricks

Visier can retrieve data through this connector if the following requirements are met.

Note: Beta This is a beta connector. If you are interested in using this connector, please contact your Customer/Partner Success Manager.

Prerequisites

  • Set up Delta Sharing in Databricks

Set up Delta Sharing

Create a share in Databricks so that Visier can retrieve your source data.

  1. In Databricks, go to the Delta Sharing page.
  2. Click Share data in the Shared by me tab.
  3. In the Share data dialog:
    1. Enter a name for the share. This name will appear in Visier.

    2. Add data assets by selecting the tables to share. The selected tables will be loaded into Visier.

      Note: Notebooks cannot be loaded into Visier.

    3. Create a new recipient for Visier with the following settings:
      • Recipient type: Open
      • How will the recipient be authenticated: Token
    4. Click Create.

      Result: An activation link is created. Go to the activation link and download the credential file as you will need to provide the credentials in Visier.

Connector credentials

Create a connector credential in Visier to authenticate and authorize access to your source system. Connector credentials can be created in Studio or through an API. For more information, see Set Up Data Connectors and "Create a connector credential" in API Reference. You will need to provide the following credentials in your connector credential configuration.

The generated file contains your credentials in JSON format. Upload the file to automatically populate the connector credentials, or open it in a text editor to manually enter the required information.

Copy
Credential file example
{
    "bearerToken": "test_token",
    "endpoint": "https://oregon.cloud.databricks.com/api/2.0/delta-sharing/.../",
    "expirationTime": "2025-05-29T22:00:00.000Z",
    "shareCredentialsVersion": 1
}

Credential

Description

Example

Share Credentials Version

The version of the credential file format used by Databricks in the open Delta Sharing protocol.

1

Bearer Token

The token used to authorize the recipient.

test_token

Endpoint

The endpoint for the share to be used by the recipient.

https://oregon.cloud.databricks.com/api/2.0/delta-sharing/.../"

Sync data

After the connector credential is created, you can configure the connector to bring data into Visier.For more information, see Sync Databricks Data.

How the connector loads data

Initially, data connectors retrieve a full history for each subject to generate an initial history of events. In each subsequent data retrieval, each connector handles the data load differently.

  • Full load: All data from the source system is loaded into Visier.
  • Delta load: Only changed data is loaded into Visier. A delta load may take either the entire history or a specific number of years of history for each changed record.
  • Snapshot load: All data at a particular point in time. A snapshot load does not contain historical records or show changes over time.
  • Partial load: A subset of data is loaded into Visier.