ABFS Connection
Connection to a container in Azure Blob File System using ABFS protocol.
Examples​
- abfs_connection_shared_key.yaml
- abfs_connection_service_principal.yaml
connection:
abfs:
root: "abfs://my-container@myaccount.dfs.core.windows.net/"
shared_key: "${vaults.my_vault_name.shared_key}"
connection:
abfs:
root: "abfs://my-container@myaccount.dfs.core.windows.net/"
service_principal:
client_id: "${vaults.my_vault_name.client_id}"
client_secret: "${vaults.my_vault_name.client_secret}"
tenant_id: "${vaults.my_vault_name.tenant_id}"
AbfsConnection​
info
AbfsConnection
is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the AbfsConnection
. Each property links to the specific details section further down in this page.
Property | Default | Type | Required | Description |
---|---|---|---|---|
description | string | No | A brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
abfs | Yes | The ABFS connection options. |
Property Details​
Connection​
A connection to a data source/sink.
Property | Default | Type | Required | Description |
---|---|---|---|---|
connection | One of:   S3Connection   GcsConnection   AbfsConnection   LocalFileConnection   SnowflakeConnection   BigQueryConnection   MSSQLConnection   MySQLConnection   OracleConnection   PostgresConnection   HttpConnection   DuckDBConnection   SFTPConnection   DatabricksConnection | Yes | The connection to the data system. |
AbfsConnectionOptions​
Azure Blob File System connection options.
Property | Default | Type | Required | Description |
---|---|---|---|---|
account | string | No | The Azure Blob File System account name to connect to. | |
root | string | Yes | The abfs[s] URL for the root prefix. | |
shared_key | string | No | The Azure Blob File System shared key to use for the ABFS connection. | |
service_principal | No | The Azure Blob File System service principal in JSON to use for the ABFS connection. The JSON should include a key named 'client_id' for the client ID, a key named 'client_secret' for the client secret, and a key named 'tenant_id' for the tenant ID. |
AbfsServicePrincipal​
Property | Default | Type | Required | Description |
---|---|---|---|---|
client_id | string | Yes | The client ID for the service principal. | |
client_secret | string | Yes | The client secret for the service principal. | |
tenant_id | string | Yes | The tenant ID for the service principal. |
ResourceMetadata​
Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
Property | Default | Type | Required | Description |
---|---|---|---|---|
source | No | The origin or source information for the resource. | ||
source_event_uuid | string | No | UUID of the event that is associated with creation of this resource. |
ResourceLocation​
The origin or source information for the resource.
Property | Default | Type | Required | Description |
---|---|---|---|---|
path | string | Yes | Path within repository files where the resource is defined. | |
first_line_number | integer | No | First line number within path file where the resource is defined. |