Retry Components on failure

Implement retry strategies in your Components to handle transient failures automatically.

The following examples show how to configure retry strategies in different Component types.

YAML Components

Basic retry with attempts

Simple retry configuration for HTTP Read Components:

read-component-retry.yaml

component:
  read:
    http:
      url: "http://api.example.com/data"
      parser: "json"
  retry_strategy:
    stop_after_attempt: 3

Combined retry limits

Robust retry configuration with both attempt and time limits:

combined-retry.yaml

component:
  read:
    http:
      url: "http://api.example.com/data"
      parser: "json"
  retry_strategy:
    stop_after_attempt: 5
    stop_after_delay: 600

SQL Components

Basic retry with attempt limit

This example retries up to 2 times before failing:

simple_retry.sql

{{ config(retry_strategy=retry_strategy(stop_after_attempt=2)) }}

SELECT *
FROM {{ ref("read_local") }}

Combined retry limits

Robust retry configuration with both attempt and time limits:

combined_retry.sql

{{ config(retry_strategy=retry_strategy(stop_after_attempt=5, stop_after_delay=300)) }}

SELECT *
FROM {{ ref("your_data") }}

Python Components

Python Components configure retry strategies using parameters inside their respective decorators (e.g. @read, @transform), providing the same flexibility as YAML configurations.

Python Read Component

This example shows retry configuration for a Python Read Component with Incremental processing:

retry.py

from datetime import datetime, timedelta

from ascend.application.context import IncrementalComponentExecutionContext
from ascend.resources import read, retry_strategy
from pandas import to_datetime


@read(strategy="incremental", incremental_strategy="append", retry_strategy=retry_strategy(stop_after_attempt=5))
def market_data_incremental(context: IncrementalComponentExecutionContext):
    import yfinance as yf

    if context.is_incremental:
        current_data = context.current_data()
        date_column = context.data_plane.normalize_identifier("Date")  # This normalizes the letter casing between DuckDB and Snowflake
        start_date = to_datetime(current_data[date_column].max().execute()) + timedelta(days=1)
    else:
        current_data = None
        start_date = datetime.now() - timedelta(days=1)

    if start_date < datetime.now():
        data = yf.download("^GSPC", start=start_date).reset_index().rename(columns={"Adj Close": "Adj_Close"})
        if data.count().iloc[0] == 0:
            # if the data is empty, set the type of the date column to datetime (otherwise it will resolve to type NULL, which causes an error)
            data["Date"] = to_datetime(data["Date"])
        return data
    else:
        return None

Python Transform

Transforms can also benefit from retry strategies for handling temporary processing failures:

transform_retry.py

import ibis
from ascend.application.context import ComponentExecutionContext
from ascend.resources import ref, retry_strategy, transform


@transform(inputs=[ref("read_local")], retry_strategy=retry_strategy(stop_after_attempt=2))
def transform_with_retry(read_local: ibis.Table, context: ComponentExecutionContext) -> ibis.Table:
    # Your transform logic here
    return read_local

Pattern-based retry clauses

Handle different error types with independent retry limits. Each pattern maintains its own counter — transient 503 errors won't exhaust your timeout retry budget.

YAML Components

pattern-retry.yaml

component:
  read:
    http:
      url: "http://api.example.com/data"
      parser: "json"
  retry_strategy:
    stop_after_attempt: 15  # hard cap
    retry_clauses:
      - pattern: "503.*Service Unavailable"
        max_attempts: 10
      - pattern: "timeout|timed out"
        max_attempts: 3

SQL Components

pattern_retry.sql

{{ config(
  retry_strategy=retry_strategy(
    retry_clauses=[
      {"pattern": "deadlock", "max_attempts": 5},
      {"pattern": "connection reset", "max_attempts": 3}
    ]
  )
) }}

SELECT * FROM {{ ref("source_data") }}

Python Components

pattern_retry.py

from ascend.resources import read, retry_strategy

@read(
  retry_strategy=retry_strategy(
    stop_after_delay=300,
    retry_clauses=[
      {"pattern": "rate limit", "max_attempts": 10},
      {"pattern": "timeout", "max_attempts": 3},
    ]
  )
)
def api_data(context):
    # fetch from rate-limited API
    ...

How pattern matching works

Behavior	Description
Regex matching	Patterns are matched against exception messages (case-insensitive)
First match wins	Clauses evaluated in order; put specific patterns before general ones
Independent counters	Each pattern tracks retries separately
Global hard cap	stop_after_attempt/stop_after_delay override pattern limits
Fallback	Non-matching errors use global limits or fail immediately

Project-level defaults

You can set default retry behavior for all Components in your Project, which individual Components can override as needed:

ascend_project.yaml

project:
  description: A collection of flows showcasing a variety of capabilities
  version: 0.0.1
  connections: ["connections/"]
  flows: ["flows/"]
  profiles: ["profiles/"]
  tests: ["tests/"]
  vaults: ["vaults/"]
  defaults:
    - kind: Component
      name:
        regex: .*
      spec:
        retry_strategy:
          stop_after_attempt: 3

Individual Components can override these defaults by specifying their own retry_strategy configuration.

Next steps

• Configure Project-level defaults for consistent retry behavior
• Use Profiles to manage environment-specific retry settings