Environment Dataplane

Before you begin

• You will need to make sure you have already set up your Instance Vault and Instance Store.
• You will need to make sure you have already set up your Environment Vault.

Set up your Dataplane

Before you can run your first flow, you first need to define where your flows will run and where your data will be stored. You will define these values by setting up your Data Plane, creating a connection in Ascend to your Data Plane, and then configuring your Profile to use this connection.

Snowflake Dataplane

1. Log in to your Snowflake account and open a new worksheet.
2. Copy and run the following SQL script below, which will create your DB, a warehouse, a user, and a role for your Snowflake Dataplane, as well as grant the necessary permissions. Names for the Database, Warehouse, User and Role have already been provided for you, but can be changed if needed.

info

You will also need to update the Snowflake user's password in the SQL script below, where it says <CHANGE_ME>. You will need this password later to add to your Environment Vault in the next step.

USE ROLE SYSADMIN;
SET ENV_PROD_DB='ASCEND_ENV_PROD';
SET WAREHOUSE='ASCEND_ENV_PROD';
CREATE DATABASE identifier($ENV_PROD_DB);
CREATE WAREHOUSE identifier($WAREHOUSE) WAREHOUSE_SIZE='XSMALL' WAREHOUSE_TYPE='STANDARD' AUTO_SUSPEND=60 AUTO_RESUME=TRUE INITIALLY_SUSPENDED=TRUE;

USE ROLE SECURITYADMIN;
SET ENV_PROD_USER='ASCEND_ENV_PROD';
SET ENV_PROD_ROLE='ASCEND_ENV_PROD';
CREATE ROLE identifier($ENV_PROD_ROLE);
CREATE USER identifier($ENV_PROD_USER) PASSWORD='<CHANGE_ME>' DEFAULT_ROLE=$ENV_PROD_ROLE; -- MUST_CHANGE_PASSWORD=TRUE;
/* Store this password in your environment vault under the key "snowflake_warehouse_password" *
You will need this key later when configuring your connection in Ascend */
GRANT ROLE identifier($ENV_PROD_ROLE) TO USER identifier($ENV_PROD_USER);
GRANT ALL ON WAREHOUSE identifier($WAREHOUSE) TO ROLE identifier($ENV_PROD_ROLE);
GRANT ALL ON DATABASE identifier($ENV_PROD_DB) TO ROLE identifier($ENV_PROD_ROLE);

BigQuery Dataplane

1. Ensure your have the Google Cloud CLI installed on your local machine.
2. Run the following commands to create a service account and store the key in your vault, replacing the placeholders with your own values:

• Set DATAPLANE_PROJECT_ID to the project ID where your BigQuery Dataplane will be stored.
• Set DATAPLANE_SERVICE_ACCOUNT_NAME to the desired service account name.

DATAPLANE_PROJECT_ID=<Replace with your Project ID>
DATAPLANE_SERVICE_ACCOUNT_NAME=ascend-io-prod-bigquery # Replace with desired service account name

# Enable BQ API on your GCP project
gcloud services enable bigquery.googleapis.com --project $DATAPLANE_PROJECT_ID

# Create a service account
gcloud iam service-accounts create $DATAPLANE_SERVICE_ACCOUNT_NAME --project $DATAPLANE_PROJECT_ID

# Grant the service account the necessary permissions
gcloud projects add-iam-policy-binding $DATAPLANE_PROJECT_ID \
  --member serviceAccount:$DATAPLANE_SERVICE_ACCOUNT_NAME@$DATAPLANE_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/bigquery.admin \
  --condition=None

# Create a key for the service account. NB: dataplane_secret.txt is actually a json service account key.
gcloud iam service-accounts keys create dataplane_secret.txt \
  --iam-account $DATAPLANE_SERVICE_ACCOUNT_NAME@$DATAPLANE_PROJECT_ID.iam.gserviceaccount.com

Azure Key Vault

1. If you don't already have write access to your vault, you can grant it by running the following command in your terminal:

az role assignment create \
  --role "Key Vault Secrets Officer" \
  --assignee "$(az ad signed-in-user show --query id -o tsv)" \
  --scope "/subscriptions/<your-instance-subscription-id>/resourcegroups/<your-instance-resource-group>/providers/Microsoft.KeyVault/vaults/<your-environment-vault-name>";

2. Run the following command in your terminal, replacing the necessary values:
   • Set ENV_VAULT_NAME to the name of your Environment Key Vault.
   • Set ENV_VAULT_SECRET_NAME to the desired secret name.
   • Replace <CHANGE_ME> with the password assigned to your Snowflake user.

ENV_VAULT_NAME=<your-environment-vault-azure-key-vault-name>
ENV_VAULT_SECRET_NAME=ascend-env-prod-snowflake-password # Replace with desired secret name
az keyvault secret set --vault-name $ENV_VAULT_NAME --name $ENV_VAULT_SECRET_NAME --value <CHANGE_ME>

Google Secret Manager

• Set ENV_VAULT_PROJECT_ID to the project ID where your Environment Secrets are stored.
• Set ENV_VAULT_SECRET_NAME to the desired secret name. NOTE: If you are using prefixes to separate Environment secrets from Instance secrets, ensure the secret name has the correct prefix.

ENV_VAULT_PROJECT_ID=<your-environment-vault-project-id>
ENV_VAULT_SECRET_NAME=ascend-env-prod-data-plane # Replace with desired secret name
gcloud secrets create $ENV_VAULT_SECRET_NAME --replication-policy automatic --project $ENV_VAULT_PROJECT_ID --data-file=dataplane_secret.txt

Next Steps

Now that you have completed setup for you first Environment, you can move on to connecting your Git Repository to Ascend.