Automation

Automations in Ascend help you build intelligent, event-driven data workflows without manual intervention. They connect triggers with actions to orchestrate your data pipelines, improving efficiency and reliability throughout the data lifecycle.

Key components

Ascend's automation framework consists of three main components that work together to create powerful workflows:

1. Triggers: Sensors and Events

Sensors actively monitor for specific conditions:

• Time-based: Run on schedules using cron expressions
• Custom Python: Execute custom logic to detect complex conditions

Events listen for system activities:

• Flow completion: Trigger when Flows complete successfully or fail
• Component status: Respond to individual Component state changes

2. Actions

Actions define what happens when a trigger activates:

• Flow Runs: Start or restart specific Flows
• Custom Python: Execute specialized business logic or integrations

3. SQL Filters

SQL filters refine which events trigger actions:

• Filter by Flow name, Component name, status, and other attributes
• Create precise conditions for when automations should run

Architecture

This diagram illustrates the architecture of Ascend's Automation framework. Triggers (both Sensors and Events) are detected through the Event Bus and then processed by the DataAware Automation Engine, which orchestrates the execution of corresponding Actions. This event-driven architecture enables seamless workflow automation without requiring manual intervention between processing stages.

Automation types

Ascend offers two approaches to building automations:

Native Automations

Built-in capabilities requiring no coding, ideal for common patterns:

• Scheduled Flow runs (time-based)
• Flow dependencies (run downstream Flows automatically)
• Conditional Flow orchestration (if-then logic)

Custom Automations

Extend capabilities with Python code for specialized needs:

• Integration with external systems and APIs
• Complex conditional logic and data validation
• Custom notifications and alerts

Common use cases

Ascend's automation capabilities excel in these scenarios:

• Scheduled data processing: Run Flows on time-based schedules
• Pipeline orchestration: Automatically trigger downstream Flows
• Error handling: Implement retry logic and failure notifications
• Cross-system coordination: Integrate with external systems and events
• Dynamic optimization: Scale resources based on workload demands

Practical implementation

To implement automations effectively:

1. Identify the trigger points in your data workflows
2. Define the appropriate actions for each trigger
3. Configure the right sensors or events with proper SQL filters
4. Test and iterate on your automation logic

For step-by-step instructions, see our Automation tutorial.