Build Transform Components
Transform Components handle the data processing logic in your pipelines. They allow you to clean, aggregate, enrich, and shape data using SQL or Python. This section provides step-by-step guides for building different types of Transform Components.
Before you begin
- Ensure you have an Ascend Project and Workspace
- Have Read Components or other data sources ready as inputs
Transform guides by language
🗃️ SQL Transforms
Use SQL for set-based operations, aggregations, filtering, and joins. Perfect for relational data processing.
🐍 Python Transforms
Build flexible transforms with Python for complex logic, text processing, and custom calculations.
⚡ PySpark Transforms
Process large datasets with PySpark for distributed computing and big data transformations.
❄️ Snowpark Transforms
Leverage Snowpark for native Snowflake transformations with Python-like syntax.
Next steps
After building your Transform Components:
- 📤 Write transformed data to destinations
- 🧪 Add data tests to validate transformation logic
- 📋 Create Task Components for additional processing
- 🔄 Set up Automations to orchestrate your pipeline