Quickstart for BigQuery on Ascend

Scale up using BigQuery with Ascend.

Prerequisites

Before you begin, ensure you have the following:

• An Ascend Instance
• Google Cloud Platform (GCP) project: sufficient permissions to manage IAM roles
  • We recommend using an isolated GCP project per Ascend Environment

Grant your Ascend Environment access

You'll need to get your Ascend Environment's service account email and grant it sufficient BigQuery permissions.

BigQuery permissions

Grant BigQuery access to your Ascend Environment’s service account in IAM & Admin → IAM.

1. Navigate to IAM & Admin → IAM
2. Select your GCP project (top left)
3. Click Grant access
4. Paste the Ascend Environment's service account email into New principals
5. Select one of the role configurations below
6. Click Save

BigQuery Admin

Select BigQuery Admin to grant full access to all BigQuery resources in the project.

warning

This role grants broad, project-wide permissions. Use this configuration for initial setup or when full access is required.

Scoped roles

Use the following roles to grant the required BigQuery access while limiting project-wide permissions:

• BigQuery User
• BigQuery Read Session User

This configuration allows Ascend to run jobs and create datasets. It grants ownership and access only to datasets created by the service account.

Depending on your use case, you may also need:

• BigQuery Data Viewer to read existing datasets (project-wide or dataset-specific).
• BigQuery Data Editor to modify data in an existing dataset (configured in BigQuery → Dataset → Sharing).

Create a BigQuery Data Plane Connection

Navigate to your Workspace. Click the + button next to Connections from the Super Graph. You should only need to fill out the GCP project ID field, then you can Test Read Connection.

tip

If you don't see the Connections section in the Resources tab of the Toolbar, you are likely in a Flow Graph. Use the search bar and type "Super Graph" or click back to all resources from the Toolbar (Resources tab).

Alternatively, create the file connections/data_plane_bigquery.yaml in your Project's files:

connection:
  bigquery:
    project: MY_PROJECT
    dataset: MY_PROJECT_DEVELOPMENT
    create_dataset_if_not_exists: true

Use BigQuery by default for all Flows

Ask Otto, or find and edit your ascend_project.yaml at the root of your Project's files. You can set the defaults section to use your new BigQuery Data Plane Connection, assuming you named it data_plane_bigquery, like:

project:
  ...
  defaults:
    - kind: Flow
      name:
        regex: .*
      spec:
        data_plane:
          connection_name: data_plane_bigquery
    - kind: Flow
      name:
        regex: .*-duckdb
      spec:
        data_plane:
          connection_name: data_plane_duckdb

Any Flows that end in -duckdb will still use the default Ascend-managed DuckDB Data Plane Connection.

Next steps

Try running the demo sales Flow or use Otto to create your first data pipeline with BigQuery on Ascend.