Synapse Connection
SynapseConnection
info
SynapseConnection
is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the SynapseConnection
. Each property links to the specific details section further down in this page.
Property | Default | Type | Required | Description |
---|---|---|---|---|
name | string | No | The name of the model | |
description | string | No | A brief description of what the model does. | |
metadata | ResourceMetadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | |
synapse | SynapseConnectionOptions | Yes | The Synapse connection options. |
Property Details
Connection
A connection to a data source/sink.
Property | Default | Type | Required | Description |
---|---|---|---|---|
connection | One of: S3Connection GcsConnection AbfsConnection LocalFileConnection SnowflakeConnection BigQueryConnection MSSQLConnection MySQLConnection OracleConnection PostgresConnection HttpConnection DuckDBConnection CustomPythonConnection SynapseConnection | Yes | The connection to the data system. |
SynapseConnectionOptions
Property | Default | Type | Required | Description |
---|---|---|---|---|
max_query_length | integer | No | The maximum query length to allow. | |
max_concurrent_queries | integer | No | The maximum number of concurrent queries to allow. | |
max_combined_sql_statements | integer | No | The maximum number of combined SQL statements to allow. | |
host | string | Yes | The MSSQL server host to connect to. | |
user | string | Yes | The MSSQL Server user to connect as. | |
password | string | Yes | The MSSQL Server password to use for the connection. | |
database | string | No | The MSSQL Server database to connect to. | |
schema | dbo | string | No | The MSSQL schema to use. |
port | integer | No | The MS SQL Server port to connect to. | |
driver | string ("ODBC Driver 18 for SQL Server", "FreeTDS") | Yes | The driver to use for the MSSQL connection. | |
staging | SynapseExternalDataSource | Yes | The external data source to use for staging data for synapse to load from. | |
create_schema_if_not_exists | boolean | No | A boolean flag indicating whether to create the schema if it does not exist. | |
table_index | string ("heap", "clustered_columnstore") | No | The type of index to use for the table. | |
distribution | string ("ROUND_ROBIN", "REPLICATE") | No | The type of distribution to use for the table. |
SynapseExternalDataSource
Property | Default | Type | Required | Description |
---|---|---|---|---|
name | string | Yes | The unique name of the external data source. | |
abfs | AbfsConnectionOptions | Yes | The ABFS connection to use for the external data source. |
AbfsConnectionOptions
Azure Blob File System connection options.
Property | Default | Type | Required | Description |
---|---|---|---|---|
account | string | No | The Azure Blob File System account name to connect to. | |
root | string | Yes | The abfs[s] URL for the root prefix. | |
shared_key | string | No | The Azure Blob File System shared key to use for the ABFS connection. | |
service_principal | AbfsServicePrincipal | No | The Azure Blob File System service principal in JSON to use for the ABFS connection. The JSON should include a key named 'client_id' for the client ID, a key named 'client_secret' for the client secret, and a key named 'tenant_id' for the tenant ID. |
AbfsServicePrincipal
Property | Default | Type | Required | Description |
---|---|---|---|---|
client_id | string | Yes | The client ID for the service principal. | |
client_secret | string | Yes | The client secret for the service principal. | |
tenant_id | string | Yes | The tenant ID for the service principal. |
ResourceMetadata
Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
Property | Default | Type | Required | Description |
---|---|---|---|---|
source | ResourceLocation | No | The origin or source information for the resource. | |
source_event_uuid | string | No | UUID of the event that is associated with creation of this resource. |
ResourceLocation
The origin or source information for the resource.
Property | Default | Type | Required | Description |
---|---|---|---|---|
path | string | Yes | Path within repository files where the resource is defined. | |
first_line_number | integer | No | First line number within path file where the resource is defined. |