Synapse Connection
SynapseConnection
info
SynapseConnection is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the SynapseConnection. Each property links to the specific details section further down in this page.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| description | string | No | Brief description of what the model does. | |
| metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
| name | string | Yes | The name of the model | |
| synapse | Yes | Synapse Connection options. |
Property Details
Connection
Data source/sink Connection.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| connection | One of: S3Connection GcsConnection AbfsConnection LocalFileConnection SnowflakeConnection BigQueryConnection MSSQLConnection MySQLConnection OracleConnection PostgresConnection HttpConnection DuckDBConnection SFTPConnection DatabricksConnection | Yes | Data system Connection. |
SynapseConnectionOptions
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| max_query_length | integer | No | Maximum combined query length permitted during query batching. | |
| max_concurrent_queries | integer | No | Maximum number of concurrent queries permitted. | |
| max_combined_sql_statements | integer | No | Maximum number of combined SQL statements permitted during query batching. | |
| host | string | Yes | Microsoft SQL Server host. | |
| user | string | No | Microsoft SQL Server user. | |
| password | string | No | Microsoft SQL Server user password. | |
| schema | dbo | string | No | Microsoft SQL Server schema. |
| port | integer | No | Microsoft SQL Server port. | |
| driver | string ("ODBC Driver 18 for SQL Server", "FreeTDS") | Yes | Driver to use for the Microsoft SQL Server Connection. | |
| trust_server_certificate | boolean | No | ||
| database | string | Yes | Microsoft SQL Server database. | |
| staging | Yes | External data source for staging data for Synapse to load from. | ||
| create_schema_if_not_exists | boolean | No | Boolean flag indicating whether to create the schema if it does not exist. | |
| table_index | string ("heap", "clustered_columnstore") | No | Table index type. | |
| distribution | string ("ROUND_ROBIN", "REPLICATE") | No | Table distribution type. | |
| spark | No | Optional settings to enable Spark-based operations. |
SynapseAnalyticsSparkOptions
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| endpoint | string | No | The development endpoint of the Synapse Analytics workspace | |
| service_principal | No | Azure service principal for authenticating against the Spark Livy API endpoint. | ||
| sql_user | string | No | SQL user for the Synapse Analytics Connection | |
| sql_password | string | No | SQL password for the Synapse Analytics Connection | |
| sql_server | string | No | SQL server for the Synapse Analytics Connection | |
| staging | No | External data source for staging data for Synapse to load from. |
SynapseExternalDataSource
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| name | string | Yes | Unique name of the external data source. | |
| abfs | Yes | ABFS Connection for the external data source. |
AbfsConnectionOptions
Azure Blob File System Connection options.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| account | string | No | Azure Blob File System account name to connect to. | |
| root | string | Yes | abfs[s] URL for the root prefix. | |
| shared_key | string | No | Azure Blob File System shared key to use for the ABFS Connection. | |
| service_principal | No | Azure Blob File System service principal in JSON to use for the ABFS connection. The JSON should include a key named 'client_id' for the client ID, a key named 'client_secret' for the client secret, and a key named 'tenant_id' for the tenant ID. | ||
| enable_default_credential | False | boolean | No | If True, enables use of the default credential for the ABFS connection if no other credentials are provided. |
AbfsServicePrincipal
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| client_id | string | Yes | Client ID for the service principal. | |
| client_secret | string | Yes | Client secret for the service principal. | |
| tenant_id | string | Yes | Tenant ID for the service principal. |
ResourceMetadata
Meta information of a resource. In most cases, it doesn't affect the system behavior but may be helpful to analyze Project resources.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| source | No | The origin or source information for the resource. | ||
| source_event_uuid | string | No | Event UUID associated with creation of this resource. |
ResourceLocation
The origin or source information for the resource.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| path | string | Yes | Path within repository files where the resource is defined. | |
| first_line_number | integer | No | First line number within path file where the resource is defined. |