Snowpark Transform
Snowpark transforms execute Python code to transform data within the Snowflake platform.
SnowparkTransform
info
SnowparkTransform
is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the SnowparkTransform
. Each property links to the specific details section further down in this page.
Property | Default | Type | Required | Description |
---|---|---|---|---|
dependencies | array[None] | No | List of dependencies that must complete before this component runs. | |
event_time | string | No | Timestamp column in the component output used to represent event time. | |
microbatch | boolean | No | Whether to process data in microbatches. | |
batch_size | string | No | The size/time granularity of the microbatch to process. | |
lookback | 1 | integer | No | The number of time intervals prior to the current interval (and inclusive of current interval) to process in time-series processing mode. |
begin | string | No | The 'beginning of time' for this component. If provided, time intervals before this time will be skipped in a time-series run. | |
inputs | array[None] | No | List of input components to use as data sources for the transform. | |
strategy | Any of: PartitionedStrategy IncrementalStrategy string ("view", "table") | No | Transform strategy - incremental, partitioned, or view/table. | |
snowpark | No | Snowpark transform function to execute for transforming the data. |
Property Details
Component
A component is a fundamental building block of a data flow. Types of components that are supported include: read, transform, task, test, and more.
Property | Default | Type | Required | Description |
---|---|---|---|---|
component | One of: CustomPythonReadComponent ApplicationComponent AliasedTableComponent ExternalTableComponent | Yes | Configuration options for the component. |
TransformComponent
A component that executes SQL or Python code to transform data.
Property | Default | Type | Required | Description |
---|---|---|---|---|
data_plane |