Skip to main content

Write to BigQuery

This guide shows you how to create a BigQuery Write Component.

Prerequisites​

  • BigQuery Connection with write permissions
  • An Ascend Flow with a Component that contains data

Create a new Write Component​

From your workspace Super Graph view, follow these steps to create your Write Component:

  1. Double-click the Flow where you want to create your Component
  2. Right-click on any Component
  3. Hover over Create Downstream -> Write, and select your target Connection menu
  4. Complete the form with these details:
    • Select your Flow
    • Enter a descriptive Component Name like write_mysql form

Configure your BigQuery Write Component

Follow these steps to configure your BigQuery Write Component:

  1. Set up your Connection
    • Enter your BigQuery Connection name in the connection field
  2. Define your data source
    • Set input to the Component that contains your source data
  3. Configure the write destination
    • Set up the bigquery write connector options
    • Specify your target table name, schema, and other required properties
  4. Choose a write strategy

    Select the strategy that best fits your use case:

    StrategyDescriptionBest for
    full (default)Replaces the entire target table during each Flow RunReference tables, complete data refreshes
    partitionedUpdates only the partitions that have changedTime-series data, regional datasets, date-partitioned tables
    snapshotCreates flexible output as a single file or multiple chunksData exports, analytical datasets, flexible output formats

For detailed guidance on when to use each strategy, see the write strategies guide.

Examples​

Choose the write strategy that best fits your use case:

This example shows a BigQuery Write Component that uses a full write strategy with drop_and_recreate mode. This strategy replaces all data in the target table during each Flow run, ensuring a complete refresh.

write_bigquery_full.yaml
component:
write:
connection: write_bigquery
input:
name: my_component
flow: my_flow
bigquery:
table:
name: my_table
dataset: my_output_dataset
strategy:
full:
mode: drop_and_recreate

🎉 Congratulations! You successfully created a BigQuery Write Component in Ascend.