Set up a BigQuery Instance Store

This guide outlines the steps to set up an Ascend Instance Store with BigQuery.

Prerequisites

• Ascend Instance
• Google account
• Access to a GCP project with either:
  • The Owner role, or
  • The following specific permissions:
    • serviceusage.serviceUsageAdmin
    • bigquery.admin
    • resourcemanager.projectIamAdmin
• Ability to run commands in your terminal to interact with GCP resources (See Google Cloud CLI Docs here)

Create a new GCP Project

BigQuery CLI

1. Set the following variables:

   # Create a new ID for your GCP project
   PROJECT_ID="<project-id>"
   
   # If you don't know your billing account, run `gcloud billing accounts list`
   BILLING_ACCOUNT_ID="<billing-account-id>"

2. Run the following commands:

   # Make sure you're logged in and have access
   gcloud auth login
   
   # Create your project
   gcloud projects create \
     --name="Ascend Quickstart on BigQuery" \
     ${PROJECT_ID}
   
   # Set your billing account... you'll need this to create a project with API access
   gcloud billing projects link \
     --billing-account=${BILLING_ACCOUNT_ID} \
     ${PROJECT_ID}
   
   # Set the project
   gcloud config set project ${PROJECT_ID}

BigQuery UI

1. Log in to your Google account.
2. Create a new GCP project. For more information on creating a GCP project, see Google Cloud documentation.
   • Project Name: We recommend changing the project name to something more description, such as Ascend Quickstart on BigQuery.
   • (Optional) Project ID: If you want to specify a project ID, click Edit and enter it in the box that appears. If you leave this blank, the project ID will be generated for you.
   • Copy the project ID to your clipboard.
   • Billing Account: Choose the appropriate billing account for your project.
   • Click Create

GCP Projects & Resource Isolation

It's generally recommend to isolate your Ascend workloads in separate GCP projects. If you cannot create a new project or already have one you'd like to use, you can skip this step.

Connect your Ascend Instance to BigQuery

Bigquery CLI

1. In the Ascend UI, navigate to the Settings page by clicking on the instance name and your icon in the top right corner of the screen, then click on the Settings button.

2. Click on the Instance settings tab.

3. Copy the value for GCP Service Account Email listed under Instance Identity Details. Then set the ASCEND_INSTANCE_SERVICE_ACCOUNT variable by running the command below in your terminal. (Remember to replace <your-ascend-instance-gcp-service-account-email> with the email address you copied.) This is the email address of the service account that your Ascend Instance will use to access your BigQuery Instance Store.

   ASCEND_INSTANCE_SERVICE_ACCOUNT=<your-ascend-instance-gcp-service-account-email>

4. Update your Instance Store settings

   • GCP Project ID: The ID of your GCP project.
   • Dataset: The name of the dataset you want to use for your Instance Store. We recommend using ASCEND_INSTANCE as the name.

5. Click Get Setup Script and copy & paste the commands into your terminal. Alternatively, you can run the commands below (after setting the variables).

   # The GCP Project ID if you've not already done so
   PROJECT_ID=<project-id>
   
   # The name of the Instance Store dataset.
   INSTANCE_DATASET="ASCEND_INSTANCE"

   # Enable the BigQuery API
   gcloud services enable \
     bigquery.googleapis.com \
     --project ${PROJECT_ID}
   
   # Create the Instance Store dataset
   bq mk \
     --dataset \
     --description="Ascend Instance Store" \
     ${PROJECT_ID}:${INSTANCE_DATASET}
   
   # Grant the BigQuery jobUser, readSessionUser, and dataEditor roles to the Ascend Instance service account
   gcloud projects add-iam-policy-binding \
     --member="serviceAccount:${ASCEND_INSTANCE_SERVICE_ACCOUNT}" \
     --role="roles/bigquery.jobUser" \
     --condition=None \
     ${PROJECT_ID}
   
   gcloud projects add-iam-policy-binding \
     --member="serviceAccount:${ASCEND_INSTANCE_SERVICE_ACCOUNT}" \
     --role="roles/bigquery.readSessionUser" \
     --condition=None \
     ${PROJECT_ID}
   
   gcloud projects add-iam-policy-binding \
     --member="serviceAccount:${ASCEND_INSTANCE_SERVICE_ACCOUNT}" \
     --role="roles/bigquery.dataEditor" \
     --condition="title=dataset:$INSTANCE_DATASET,expression=resource.name.startsWith('projects/$GCP_PROJECT_ID/datasets/$INSTANCE_DATASET')" \
     ${PROJECT_ID}

6. Click Check and Update Instance Store

Bigquery UI

1. In the Ascend UI, navigate to the Settings page by clicking on the instance name and your icon in the top right corner of the screen, then click on the Settings button.
2. Click on the Instance settings tab.
3. Copy the value for GCP Service Account Email listed under Instance Identity Details.
   tip

   Keep this tab open in your browser.... we'll need it later.

4. Go to your GCP IAM Settings and grant the above service account the BigQuery Admin role.
   1. Click Grant Access
   2. Paste the service account email into the New principals field
   3. Select BigQuery Admin from the Role dropdown
   4. Click Save

tip

If you don't manage IAM permissions in your GCP project, reach out to your GCP administrator to grant the above service account the BigQuery Admin role.

1. Return to the Ascend UI and update your Instance Store settings
   • GCP Project ID: The ID of your GCP project.
   • Dataset: The name of the dataset you want to use for your Instance Store. We recommend using ASCEND_INSTANCE as the name.
2. In the Ascend UI, click Check and Update Instance Store

🎉 Congratulations! Your Ascend Instance is now connected to BigQuery.

You can view the Instance Store dataset in the BigQuery UI or by running bq ls ${PROJECT_ID}:${INSTANCE_DATASET}.