Simple Python Transform

In this guide, we'll build a Simple Python Transform that processes data without using Incremental or Smart Partitioning strategies.

To learn more about the supported input formats for Python Transforms, check out our concept guide.

Let's keep it Simple!

Prerequisites

• Ascend Flow

Create a Transform

You can create a Transform in two ways: through the form UI or directly in the Files panel.

Form

1. Double-click the Flow where you want to add your Transform
2. Right-click on an existing component (typically a Read component or another Transform) that will provide input data
3. Select Create Downstream → Transform
4. Complete the form with these details:
   • Select your Flow
   • Enter a descriptive name for your Transform (e.g., sales_aggregation)
   • Choose the appropriate file type for your Transform logic

Files panel

1. Open the files panel in the top left corner
2. Navigate to and select your desired Flow
3. Right-click on the components directory and choose New file
4. Name your file with a descriptive name that reflects its purpose (e.g., sales_aggregation)
5. Choose the appropriate file extension based on your Transform type:
   • .py for Python Transforms
   • .sql for SQL Transforms

Create your Python Transform

Structure your Python Transform Component following this pattern:

1. Import necessary packages: Include Ascend resources (transform), context handlers (ComponentExecutionContext), and data processing libraries (like ibis in this example)
2. Define your transform function: Create a function that performs some transform action on your data (cleaning in this example)
3. Apply the @transform() decorator: Use your desired input refs. This transforms your function's output into Ascend's internal data format
4. Return structured data: Ensure your function returns a dataframe or table

The @transform() decorator handles the conversion between your Python data structures and Ascend's internal format, enabling seamless integration with downstream Components in your data pipeline.

Example

Here's a simple example that shows how to transform input data by applying cleaning operations:

simple_transform.py

"""
Example of a Simple Python Transform Component.

This file demonstrates how to create a basic Python transform that processes
data without using Incremental or Smart partitioning strategies.
"""

import ibis
from ascend.application.context import ComponentExecutionContext
from ascend.resources import ref, transform


@transform(inputs=[ref("read_weather_routes", flow="extract-load-duckdb")])
def weather_routes(read_weather_routes: ibis.Table, context: ComponentExecutionContext) -> ibis.Table:
    """

🎉 Congratulations! You've successfully created a Simple Python Transform in Ascend.