Skip to main content

Synapse Connection

SynapseConnection​

info

SynapseConnection is defined beneath the following ancestor nodes in the YAML structure:

Below are the properties for the SynapseConnection. Each property links to the specific details section further down in this page.

PropertyDefaultTypeRequiredDescription
descriptionstring
NoA brief description of what the model does.
metadataNoMeta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
namestringYesThe name of the model
synapseYesSynapse Connection options.

Property Details​

Connection​

Data source/sink Connection.

PropertyDefaultTypeRequiredDescription
connectionOne of:
  S3Connection
  GcsConnection
  AbfsConnection
  LocalFileConnection
  SnowflakeConnection
  BigQueryConnection
  MSSQLConnection
  MySQLConnection
  OracleConnection
  PostgresConnection
  HttpConnection
  DuckDBConnection
  SFTPConnection
  DatabricksConnection
YesData system Connection.

SynapseConnectionOptions​

PropertyDefaultTypeRequiredDescription
max_query_lengthinteger
NoMaximum query length to allow.
max_concurrent_queriesinteger
NoMaximum number of concurrent queries to allow.
max_combined_sql_statementsinteger
NoMaximum number of combined SQL statements to allow.
hoststringYesMSSQL server host to connect to.
userstring
NoMSSQL Server user to connect as.
passwordstring
NoMSSQL Server user password.
schemadbostringNoMSSQL schema to use.
portinteger
NoMSSQL Server port to connect to.
driverstring ("ODBC Driver 18 for SQL Server", "FreeTDS")YesDriver to use for the MSSQL Connection.
trust_server_certificateboolean
No
databasestringYesMSSQL Server database to connect to.
stagingYesExternal data source to use for staging data for synapse to load from.
create_schema_if_not_existsboolean
NoBoolean flag indicating whether to create the schema if it does not exist.
table_indexstring ("heap", "clustered_columnstore")
NoIndex type to use for the table.
distributionstring ("ROUND_ROBIN", "REPLICATE")
NoDistribution type to use for the table.
sparkNoOptional settings to enable Spark-based operations.

SynapseAnalyticsSparkOptions​

PropertyDefaultTypeRequiredDescription
endpointstringNoThe development endpoint of the Synapse Analytics workspace
service_principalNoAzure service principal to use for authenticating against the Spark Livy API endpoint.
sql_userstringNoSQL user to use for the Synapse Analytics Connection
sql_passwordstringNoSQL password to use for the Synapse Analytics Connection
sql_serverstringNoSQL server to use for the Synapse Analytics Connection
stagingNoThe external data source to use for staging data for synapse to load from.

SynapseExternalDataSource​

PropertyDefaultTypeRequiredDescription
namestringYesUnique name of the external data source.
abfsYesABFS Connection to use for the external data source.

AbfsConnectionOptions​

Azure Blob File System Connection options.

PropertyDefaultTypeRequiredDescription
accountstring
NoAzure Blob File System account name to connect to.
rootstringYesabfs[s] URL for the root prefix.
shared_keystring
NoAzure Blob File System shared key to use for the ABFS Connection.
service_principalNoAzure Blob File System service principal in JSON to use for the ABFS connection. The JSON should include a key named 'client_id' for the client ID, a key named 'client_secret' for the client secret, and a key named 'tenant_id' for the tenant ID.
enable_default_credentialFalsebooleanNoIf True, enables use of the default credential for the ABFS connection if no other credentials are provided.

AbfsServicePrincipal​

PropertyDefaultTypeRequiredDescription
client_idstringYesClient ID for the service principal.
client_secretstringYesClient secret for the service principal.
tenant_idstringYesTenant ID for the service principal.

ResourceMetadata​

Meta information of a resource. In most cases, it doesn't affect the system behavior but may be helpful to analyze Project resources.

PropertyDefaultTypeRequiredDescription
sourceNoThe origin or source information for the resource.
source_event_uuidstring
NoUUID of the event that is associated with creation of this resource.

ResourceLocation​

The origin or source information for the resource.

PropertyDefaultTypeRequiredDescription
pathstringYesPath within repository files where the resource is defined.
first_line_numberinteger
NoFirst line number within path file where the resource is defined.