Databricks Data Export Connector

Visier can send data through this connector if the following requirements are met.

Overview

This connector uses Iceberg REST to send data to Databricks. For a guide that walks you through Databricks set up, see Push Data From Visier to Databricks.

Service account

Create a service account in your target system and assign it the appropriate security access so that Visier can send data through this user.

The service account must use OAuth 2.0 client ID and client secret authentication.

1. Set up a service principal for OAuth 2.0 machine-to-machine (M2M). For more information, see Create service principal.
2. Grant the necessary privileges to the service principal. For more information, see Grant a principal Unity Catalog privileges.

1
2
3
4
5
6
7
8
9
10
-- Example service principal ID: 123e4567-e89b-12d3-a456-426614174000

-- Allow service principal to create and write data to tables in a provided catalog+schema
GRANT EXTERNAL USE SCHEMA ON SCHEMA visier_catalog.visier_schema TO `123e4567-e89b-12d3-a456-426614174000`
GRANT USE CATALOG ON CATALOG visier_catalog TO `123e4567-e89b-12d3-a456-426614174000`;
GRANT CREATE TABLE ON SCHEMA visier_catalog.visier_schema TO `123e4567-e89b-12d3-a456-426614174000`;
GRANT USE SCHEMA ON SCHEMA visier_catalog.visier_schema TO `123e4567-e89b-12d3-a456-426614174000`;

-- Allow service principal abillity to create external metadata for lineage
GRANT CREATE EXTERNAL METADATA ON METASTORE TO `123e4567-e89b-12d3-a456-426614174000`;

Connector credentials

Provide the following credentials in Visier. For more information, see Add data export connector credentials in Visier.