ABFS Connection

InlineAbfsConnection

info

InlineAbfsConnection is defined beneath the following ancestor nodes in the YAML structure:

Below are the properties for the InlineAbfsConnection. Each property links to the specific details section further down in this page.

Property	Default	Type	Required	Description
abfs			Yes	The ABFS connection options.

A connection to a data source/sink.

Property	Default	Type	Required	Description
connection		One of: S3Connection GcsConnection AbfsConnection LocalFileConnection SnowflakeConnection BigQueryConnection MSSQLConnection MySQLConnection OracleConnection PostgresConnection HttpConnection DuckDBConnection SFTPConnection DatabricksConnection	Yes	The connection to the data system.

Connection to a DuckDB database.

Property	Type	Required	Description
description	string	No	A brief description of what the model does.
metadata		No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name	string	Yes	The name of the model
duckdb		Yes	The DuckDB connection options.

DuckDB connection options.

Property	Type	Required	Description
max_query_length	integer	No	The maximum query length to allow.
max_concurrent_queries	integer	No	The maximum number of concurrent queries to allow.
max_combined_sql_statements	integer	No	The maximum number of combined SQL statements to allow.
path	string	No	The path to the DuckDB database file. Use ':memory:' for in-memory databases. This setting is ignored when ducklake configuration is present.
memory_limit	integer	No	The memory limit to use for the DuckDB connection.
ducklake		No	Configuration for using DuckLake with this DuckDB connection.
schema	string	No	The schema to use for the DuckDB connection. If left empty, the flow name will be used as the schema name at runtime
init_sql	string	No	SQL to run when the DuckLake connection is initialized.

DuckLake configuration options.

Property	Default	Type	Required	Description
metadata_connection_name		string	No	The name of the Ascend Connection to use for DuckLake metadata storage (Postgres connections only).
data_connection_name		string	No	The name of the Ascend Connection to use for DuckLake data storage (Local Files, GCS, S3, or ABFS connections supported).
metadata_connection		Any of: InlinePostgresConnection ASCEND_MANAGED	No	Inline metadata connection configuration (Postgres connections only).
data_connection		Any of: One of: InlineLocalFileConnection InlineS3Connection InlineGcsConnection InlineAbfsConnection ASCEND_MANAGED	No	Inline data connection configuration (Local Files, GCS, S3, or ABFS connections supported).
metadata_schema	default	string	No	The schema name within the Postgres database to use for DuckLake metadata storage. If left as the default value of 'default', the schema name will be replaced with the profile name at runtime.
data_path	ascendlake/data	string	No	Path within the data connection root where DuckLake data files will be stored.

Azure Blob File System connection options.

Property	Default	Type	Required	Description
account		string	No	The Azure Blob File System account name to connect to.
root		string	Yes	The abfs[s] URL for the root prefix.
shared_key		string	No	The Azure Blob File System shared key to use for the ABFS connection.
service_principal			No	The Azure Blob File System service principal in JSON to use for the ABFS connection. The JSON should include a key named 'client_id' for the client ID, a key named 'client_secret' for the client secret, and a key named 'tenant_id' for the tenant ID.
enable_default_credential	False	boolean	No	If True, enables use of the default credential for the ABFS connection if no other credentials are provided.

Property	Type	Required	Description
client_id	string	Yes	The client ID for the service principal.
client_secret	string	Yes	The client secret for the service principal.
tenant_id	string	Yes	The tenant ID for the service principal.