Write to GCS

This guide shows you how to create a Google Cloud Storage Write Component.

Prerequisites

• Google Cloud Storage Connection with write permissions
• An Ascend Flow with a Component that contains data

Create a new Write Component

From your workspace Super Graph view, follow these steps to create your Write Component:

Form

1. Double-click the Flow where you want to create your Component
2. Right-click on any Component
3. Hover over Create Downstream -> Write, and select your target Connection
4. Complete the form with these details:
   • Select your Flow
   • Enter a descriptive Component Name like write_mysql

Files panel

1. Open the Files panel in the top left corner
2. Navigate to and select your desired Flow
3. Right-click on the components directory and choose New file
4. Name your file with a descriptive name like write_mysql.yaml and press enter

Configure your Google Cloud Storage Write Component

Follow these steps to configure your Google Cloud Storage Write Component:

1. Set up your Connection
   • Enter your Google Cloud Storage Connection name in the connection field
2. Define your data source
   • Set input to the Component that contains your source data
3. Configure the write destination
   • Set up the gcs write connector options
   • Specify your target table name, schema, and other required properties
4. Choose a write strategy

   Select the strategy that best fits your use case:

   Strategy	Description	Best for
   full (default)	Replaces the entire target table during each Flow Run	Reference tables, complete data refreshes
   partitioned	Updates only the partitions that have changed	Time-series data, regional datasets, date-partitioned tables
   snapshot	Creates flexible output as a single file or multiple chunks	Data exports, analytical datasets, flexible output formats

For detailed guidance on when to use each strategy, see the write strategies guide.

Examples

Choose the write strategy that best fits your use case:

Full write strategy

This example shows a Google Cloud Storage Write Component that uses a full write strategy. Full writes now produce chunked output for better performance with large datasets.

full_gcs.yaml

component:
  write:
    connection: write_gcs
    input:
      name: my_component
      flow: my_flow
    gcs:
      path: /some-other-dir/my_data.parquet
      formatter: parquet

Output: Multiple files like part_001.parquet, part_002.parquet, etc. in the specified directory.

Snapshot (chunked)

This example shows a Google Cloud Storage Write Component using snapshot strategy with chunked output. The path ends with a trailing slash (/), producing multiple chunk files.

write_gcs_snapshot.yaml

component:
  write:
    connection: write_gcs
    input:
      name: my_component
      flow: my_flow
    strategy: snapshot
    gcs:
      path: /snapshot_data/
      formatter: parquet

Output: Multiple files like part_001.parquet, part_002.parquet, etc. in the /snapshot_data/ directory.

Snapshot (single file)

This example shows a Google Cloud Storage Write Component using snapshot strategy with single file output. The path ends with a specific filename and extension.

write_gcs_snapshot_single.yaml

component:
  write:
    connection: write_gcs
    input:
      name: my_component
      flow: my_flow
    strategy: snapshot
    gcs:
      path: /snapshot_data/my_snapshot.parquet
      formatter: parquet

Output: A single file named my_snapshot.parquet in the /snapshot_data/ directory.

🎉 Congratulations! You successfully created a Google Cloud Storage Write Component in Ascend.