Execute Components in order with inputs
Use inputs when your Component needs to process data from upstream Components. Use dependencies (a special kind of input) when your Component only needs upstream Components to complete first without accessing their data, such as for setup tasks or configuration.
To gain a deeper understanding of these concepts, see the Components guide.
Looking to reference Components from other Flows? Use cross-Flow references.
Syntax
You can add inputs or dependencies to any Component type using the appropriate syntax:
| Component type | Syntax | Location |
|---|---|---|
| YAML Components | inputs: / dependencies: field | Field in the Component configuration |
| SQL Components | inputs[] / dependencies=[] parameter | Parameter in the config block |
| Python Components | inputs[] / dependencies=[] parameter | Parameter in the Component decorator |
Read Components support a dependencies field but not an inputs field, since they don't process data from upstream Components.
YAML example
Define dependencies using the dependencies field in your YAML configuration:
component:
read:
connection: read_gcs_lake
gcs:
path: ottos-expeditions/lakev0/generated/events/sales_store.parquet/year=
include:
- glob: "*/month=*/day=*/*.parquet"
dependencies:
- name: sales_website
flow: extract-load-duckdb
- name: sales_vendors
flow: extract-load-duckdb
SQL examples
Specify inputs and dependencies in the configuration block at the top of your SQL file:
{{
config(
type="transform",
inputs=[
ref('sales_vendors', flow='transform-duckdb'),
ref('read_sales_vendors', flow='extract-load-duckdb'),
]
)
}}
SELECT * FROM {{ ref('sales_vendors', flow='transform-duckdb') }}
Python examples
Add inputs and dependencies as parameters in your Component decorator. Tables referenced via ref() must also be passed as function arguments. The implementation varies by Component type:
Transform
Incremental Python Read Component
Smart Python Read Component
Tasks
Tasks use the same syntax for dependencies and inputs as Transforms. Python Tasks follow the Python Transform syntax, while SQL Tasks follow the SQL Transform syntax.
Python Task
import ibis
from ascend.application.context import ComponentExecutionContext
from ascend.common.events import log
from ascend.resources import ref, task
@task(
dependencies=[
ref("social_media", flow="transform-duckdb"),
]
)
def task_respond_on_socials(social_media: ibis.Table, context: ComponentExecutionContext) -> None:
for i in range(1000):
log("Thank you for your comment!")
SQL Task
{{
config(
type="task",
dependencies=[ref("guides", flow="transform-duckdb")]
)
}}
SELECT 1 as id