Build Transforms

Transforms handle the data processing logic in your pipelines. They allow you to clean, aggregate, enrich, and shape data using SQL or Python. This section provides step-by-step guides for building different types of Transforms.

Before you begin

• Ensure you have an Ascend Project and Workspace
• Have Read Components or other data sources ready as inputs

Table of Contents (ToC)

• Build PySpark Transforms
  • Simple PySpark Transform
  • Incremental PySpark Transform
  • Smart PySpark Transform
• Build Python Transforms
  • Simple Python Transform
  • Incremental Python Transform
  • Smart Python Transform
• Build Snowpark Transforms
  • Simple Snowpark Transform
  • Incremental Snowpark Transform
  • Smart Snowpark Transform
• Build SQL Transforms
  • Simple SQL Transform
  • Incremental SQL Transform
  • Smart SQL Transform
  • Macros
• Custom BigQuery partitioning and clustering