Quickstart for Ascend on Snowflake
Introduction​
In this quickstart guide, you'll learn how to use Ascend with Snowflake. You'll be guided through these processes:
- Connecting Ascend to Snowflake
- Creating required Snowflake resources
- Setting up your Instance Vault
- Setting up your Instance Store
- Setting up your Environment Vault(s)
- Creating an Ascend Project using Snowflake as the Data Plane
- Setting up a Repository
- Creating your developer Workspace
- Setting up your Profile and Connection to use Snowflake
- Building & running your first Flow
This quickstart grants elevated permissions for the sake of simplicity. We recommend isolating your Ascend Instance and each Environment's resources in separate Snowflake resources based on your security requirements.
Prerequisites​
To complete this quickstart, you'll need:
- An Ascend Instance
- An Snowflake account
- The ability to create Snowflake resources (may require elevated permissions for some steps)
Set up Snowflake as an Ascend Instance Store​
You can leverage existing Snowflake resources, but if you're starting with a new Snowflake account, this section will guide you through the setup process step by step.
Follow along to use the provided scripts, which will automatically create the necessary resources for you.
You need to create a set of resources for the Ascend Instance Store. In Snowflake, you'll create the following:
- a User
- a Database
- a Schema
- a Warehouse
- a Role
Start in the Snowflake UI and sign into the account you will use with Ascend. Click Create toward the top left of the screen and select SQL Worksheet. You'll run SQL commands to create the required Snowflake resources for Ascend.
Now, navigate to the Ascend UI and click into Settings toward the top right of the screen. Navigate to the Instance settings. You'll need to store the password for the Snowflake user in the Instance Vault to allow Ascend to access Snowflake resources. Click Manage Secrets under the Instance Vault section and add a secret with the Snowflake user password.
For this quickstart, we'll use the secret name snowflake-password
, although you can feel free to use any name you like.
Store the password you use here. You will need it to set up the Snowflake User in the next step.
Use a password manager to generate a secure password. On MacOS (or Linux) you can use openssl
to generate a random password in your terminal as follows:
openssl rand -base64 32
Head back to the Instance settings in the Ascend UI and scroll toward the bottom to Edit Instance Store. Select Snowflake and fill out the details. Then, click Get Setup Script. This will populate a SQL script using the values you entered.
Paste the script into the Snowflake SQL Worksheet, and make sure to update the password to match the value in your Instance Vault.
The password is not filled in for you in the script. After copying it to the Snowflake UI, ensure you update the password with the secret value you used above.
You can run the entire script all at once or execute each line individually to better understand the purpose of each command.
Now that you've created the necessary resources, return to the Ascend UI and click Check and Update Instance Store.
Congratulations! You successfully set up Snowflake as an Ascend Instance Store.
Set Up Snowflake as an Ascend Data Plane​
To use Snowflake as an Ascend Data Plane, you'll need to create specific resources. Ensure that each Environment has its own dedicated Data Plane setup by creating the following for each one:
- a User
- a Database
- a Schema
- a Warehouse
- a Role
Note that these resources should be separate from the Instance Store resources. You should also create isolated resources for each Environment you use. The setup will be similar to the above, but with different values for the resources.
Change the password in the script below. Do not use the same password as the Instance Store user.
USE ROLE SYSADMIN;
SET ENV_DEV_DB='ASCEND_ENV_DEV';
SET WAREHOUSE='ASCEND_ENV_DEV';
CREATE DATABASE identifier($ENV_DEV_DB);
CREATE WAREHOUSE identifier($WAREHOUSE) WAREHOUSE_SIZE='XSMALL' WAREHOUSE_TYPE='STANDARD' AUTO_SUSPEND=60 AUTO_RESUME=TRUE INITIALLY_SUSPENDED=TRUE;
USE ROLE SECURITYADMIN;
SET ENV_DEV_USER='ASCEND_ENV_DEV';
SET ENV_DEV_ROLE='ASCEND_ENV_DEV';
CREATE ROLE identifier($ENV_DEV_ROLE);
CREATE USER identifier($ENV_DEV_USER) PASSWORD='<CHANGE_ME>' DEFAULT_ROLE=$ENV_DEV_ROLE; -- MUST_CHANGE_PASSWORD=TRUE;
/* Store this password in your environment vault under the key "snowflake-password" *
You will need this key later when configuring your connection in Ascend */
GRANT ROLE identifier($ENV_DEV_ROLE) TO USER identifier($ENV_DEV_USER);
GRANT ALL ON WAREHOUSE identifier($WAREHOUSE) TO ROLE identifier($ENV_DEV_ROLE);
GRANT ALL ON DATABASE identifier($ENV_DEV_DB) TO ROLE identifier($ENV_DEV_ROLE);
Run this SQL script in a Snowflake SQL worksheet after updating the password, similarly to the previous step.
Now, you'll need to add the password as a secret in your Environment Vault. In the Ascend UI, navigate to the Environments settings and edit the Dev environment.
Click Manage Secrets and add a secret containing the Snowflake user password.
Using the same password as in the SQL script above, create a secret in the Environment Vault (Dev). You can choose any name for the secret, but for this quickstart, we recommend using snowflake-password to align with the default values in the code for running your first Flow below.
This secret name matches the one in the Instance Vault, which is fine since the Instance Vault and Environment Vault (Dev) are separate and serve different purposes.
Congratulations! You're ready to use Snowflake as an Ascend Data Plane.
Set up a Git Repository​
When you develop in Ascend, you will need to provide a Git repository in which to store your code. This section walks you through setting up a Git repository and integrating it with your Ascend Instance.
-
Create a new repository
- GitHub
- Navigate to GitHub's New Repository page
- Set Owner to either your personal GitHub account, or your organization.
tip
If you have the appropriate permissions to your company's GitHub Organization, you can create a new repository within your organization. If you do not have admin access, you can create a new repository within your personal account, which will show up in the list as your GitHub name.
- Set Repository name to
ascend-quickstart-snowflake
(or your preferred name) - Set Description to
Ascend Quickstart Snowflake
- Set Public/Private to Private
- Click Create repository
-
Next, clone the Ascend Community Repository and push it to your new repository.
# Set the variables
GITHUB_ORG="<your-github-org>" # If you are using your personal account, set this to your username
GITHUB_REPO="ascend-quickstart-snowflake"# Clone the Ascend Community Repository
git clone git@github.com:ascend-io/ascend-community.git ${GITHUB_REPO}
cd ${GITHUB_REPO}
# Set the remote URL to your new repository
git remote set-url origin git@github.com:${GITHUB_ORG}/${GITHUB_REPO}.git
# Push the code to your new repository
git push -u origin main
Connect Ascend to your Git Repository​
Create the SSH key pair​
- Linux
- Mac
- Windows
-
On your local machine, open a terminal and run the following command:
mkdir -p ~/.ssh && ssh-keygen -t ed25519 -f ~/.ssh/ascend_quickstart_repo_key -q -N ""
-
This command will generate a new SSH key pair in the
~/.ssh
directory. -
The public key will be saved in
~/.ssh/ascend_quickstart_repo_key.pub
-
The private key will be saved in
~/.ssh/ascend_quickstart_repo_key
.tipYou can copy the contents of a file (eg: the public key) to your clipboard by running:
cat ~/.ssh/ascend_quickstart_repo_key.pub | xclip -selection clipboard
Or if xclip is not installed:
cat ~/.ssh/ascend_quickstart_repo_key.pub
and manually copy the output.
- On your local machine, open a terminal and run the following command:
mkdir -p ~/.ssh && ssh-keygen -t ed25519 -f ~/.ssh/ascend_quickstart_repo_key -q -N ""
- This command will generate a new SSH key pair in the
~/.ssh
directory. - The public key will be saved in
~/.ssh/ascend_quickstart_repo_key.pub
- The private key will be saved in
~/.ssh/ascend_quickstart_repo_key
.tipYou can copy the contents of the public key to your clipboard by running:
cat ~/.ssh/ascend_quickstart_repo_key.pub | pbcopy
- On your local machine, open PowerShell and run the following commands:
# Create .ssh directory if it doesn't exist
New-Item -ItemType Directory -Force -Path "$env:USERPROFILE\.ssh"
# Generate SSH key
ssh-keygen -t ed25519 -f "$env:USERPROFILE\.ssh\ascend_quickstart_repo_key" -N '""' - This command will generate a new SSH key pair in the
.ssh
directory within your user profile folder. - The public key will be saved in
%USERPROFILE%\.ssh\ascend_quickstart_repo_key.pub
- The private key will be saved in
%USERPROFILE%\.ssh\ascend_quickstart_repo_key
.tipYou can copy the contents of the public key to your clipboard by running:
Get-Content "$env:USERPROFILE\.ssh\ascend_quickstart_repo_key.pub" | Set-Clipboard
Add the public key to your repository​
- Bitbucket
- GitHub
- GitLab
To allow for Ascend to have write access to your Bitbucket repository, you will need to add your public key to a Bitbucket account with write access, not to the repository itself.
Please follow the instructions under Provide Bitbucket Cloud with your public key to add the public ssh key to a Bit Bucket user.
Please follow the instructions here to setup the public ssh key in your Github repository. You will need to allow write access.
Please follow the instructions here to setup the public ssh key in your Gitlab repository. You will to need to grant write permissions to the key.
Add the private key to Ascend​
- In the Ascend UI, navigate to the instance settings by clicking on the button in the top right with your instance name and your user icon.
- Click on the Repositories settings tab.
- Click the Add Repository button.
- Set the Repository URI in this format:
git@github.com:<github-org>/<github-repo>.git
. - Set SSH Private Key to the private key you generated in the previous step.
Make sure to copy the entire private key, including the lines that indicate the beginning and end of the key.
- Click Check and Create.
Create an Ascend Project​
With your Git Repository connected, you can now create an Ascend Project.
- In the Ascend UI, navigate to the instance settings by clicking on the button in the top right with your instance name and your user icon.
- Click on the Projects settings tab.
- Click the Add Project button.
- Set Title to
Snowflake Quickstart
- Select Repository from the pulldown options.
- Set Project Root to quickstarts/snowflake
- Click Save.
Create your developer Workspace​
Now, let's create a workspace where we can develop.
- In the Ascend UI, navigate to the instance settings by clicking on the button in the top right with your instance name and your user icon.
- Click on the Workspaces settings tab.
- Click the Add Workspace button.
- Set Title to
My Workspace
- Select Environment as
Dev
- Set Project to the project you created above (
Snowflake Quickstart
) - Set Git Branch to
my-first-branch
(You can type the branch name and click "New Branch:my-first-branch" to create it if it doesn't already exist) - Set Profile as
dev
- Select Size as
X-Small
- Set Storage to
8GB
- Click Save.
Build & run your first flow​
-
In the Ascend UI, navigate to the workspace you created above and open the Files view toward the top left of the screen. Edit the
profiles/dev.yaml
file and update the following parameters:- snowflake:
- account:
<your-snowflake-account>
(the Snowflake account for the Data Plane) - any other values if you used different names from the quickstart above
- account:
- snowflake:
-
If you used secrets names in your Environment Vault other than
snowflake-password
, update theconnections/snowflake_data_plane.yaml
file with the correct secret name. -
Finally, go into the quickstart flow and hit Run Flow!