Dataplane

Ascend is flexible in how you setup your Dataplane and Instance Store, however the patterns shown below will be suitable for most use cases.

Snowflake

Ascend only requires a single Snowflake account, and separates environments and instances using database and schema.

The Instance Store is defined on top of a Snowflake database and schema, and is used to store instance level data such as flow runs, tasks, and logs.

Environments will be scoped to a database, with Ascend Projects and flows instantiating schemas and tables within the environment database.

BigQuery

There are two patterns for structuring your Dataplane when using BigQuery as your Dataplane.

1. When using BigQuery as your instance store, you will need a dedicated project and BigQuery dataset for storing instance data. However, when using BigQuery as a dataplane, how you structure your environment is up to you. Most users will benefit from structuring their instances and environment in the following way.

Best practice dictates that when you grant the instance service account access to BigQuery, it is restricted only to the instance dataset, likewise the environment service accounts are to be restricted to operating only on their respective datasets.

2. Some users might be restricted to operating Ascend from a single project, in which case the following pattern would more applicable.

In this second case, you will need to ensure that each environment and instance service account is restricted to accessing only its respective datasets by adding a condition to the policy binding command.