Automate pipeline activity summaries

Learn how to set up automated email summaries of your pipeline activity using an Ascend Automation. Otto will analyze your Flow runs and send digestible reports at your specified frequency.

Prerequisites

• At least one Ascend Flow

Create your Automation

This example demonstrates how to configure an Automation that monitors pipeline activity over the weekend. Specifically, we'll set up a summary that:

• Sends every Monday at 9 AM PST
• Covers the previous 3 days of Flow runs
• Provides insights into pipeline performance

You can create Automations using the UI or by writing YAML files directly. This aligns with Ascend's Flex Code philosophy, allowing you to choose the method that works best for your workflow.

Configuration UI

Using the configuration UI

1. In the Build panel, scroll to the Automations section and click + to create a new Automation

   

2. The configuration interface opens in a new tab

3. Give your Automation a descriptive name like activity-summary

4. Click + Add next to Sensors to create a schedule for your email summaries

5. Give your Sensor a descriptive name like monday-am

6. Set up the email frequency using a cron expression. Change the timezone from UTC if needed for your local time

   

7. Click + Add next to Events to specify what pipeline activity to monitor

8. Select the Flow you want included in the activity summary

9. Click + Add next to Actions to define what happens when the automation triggers

10. Give your Action a descriptive name like otto-activity-summary

11. Change the Action type from Run Flow to Otto Activity Summary

12. Add email recipients in the Email Recipients field, then click Add to confirm each address

    

13. Select Custom for the summary window to control how much historical data Otto analyzes

14. Enter 3d to create a 3-day lookback window for the summary

    

15. Click Save to finalize your Automation configuration

    

16. Switch to the Code tab in the top right to view the generated YAML configuration

Files panel

Using the Files panel

1. Open the Files panel in your workspace
2. Right-click the Automations folder and select New file

1. Name your file descriptively, such as activity-summary.yaml
2. Use the following configuration as your starting point:

automations/activity-summary.yaml

automation:
  name: activity-summary
  enabled: true
  triggers:
    sensors:
      - name: monday-am
        type: timer
        config:
          schedule:
            cron:
              pattern: 0 9 * * MON
              timezone: America/Los_Angeles
    events:
      - sql_filter: json_extract_string(event, '$.data.flow') = 'transform-bigquery'
        types:
          - FlowRunSuccess
  actions:
    - type: activity_summary
      name: otto-activity-summary
      config:
        emails:
          - otto@example.com
        window: 3d

This configuration creates an Automation that:

• Monitors successful runs of the transform-bigquery Flow
• Sends weekly summaries every Monday at 9 AM PST
• Analyzes the previous 3 days of pipeline activity in each report

Customize the summary window

Summary windows can be custom-configured in units of hours (h), days (d), or weeks (w) depending on your desired summary report frequency. 23h, 3d, and 4w are all valid summary windows.

note

The summary window controls how far back Otto looks for data to analyze, not how often emails are sent. Email frequency is determined by the cron expression in your Sensor configuration.