Skip to main content
Version: 3.0.0

Intelligence Core

Ascend's Intelligence Core is a powerful foundation that combines metadata, automation, and AI in a layered architecture.

markitechture

Architecture layers​

The Intelligence Core consists of three integrated layers that work together to simplify and accelerate data engineering:

Unified metadata collection​

Ascend tracks data, code, and user actions in a unified metadata layer, creating a comprehensive knowledge graph of your data ecosystem. This metadata foundation powers:

  • End-to-end observability - Monitor every aspect of your data pipeline
  • Automated lineage tracking - Understand dependencies and impacts
  • Comprehensive auditability - Track changes through the entire data lifecycle

DataAware™ automation engine​

Ascend's automation engine leverages the rich metadata layer to orchestrate pipelines intelligently:

  • Dependency management - Automatically identifies and manages relationships between components
  • Smart task triggering - Executes only what's needed based on changes
  • Event-driven workflows - Supports custom triggers for complex pipelines
  • Incremental processing - Updates only affected data partitions

Integrated AI agents​

Ascend's embedded AI Agents take the toil out of data engineering.

With user permission, Otto has access to the metadata captured by Ascend, and even to the data within your pipelines, making it deeply context-aware. Otto helps engineers in 4 key ways:

  • Chat with your stack - Ask questions about run history, code changes, errors, and more
  • Inline code suggestions - Get autocomplete code assistance as you build directly in the code editor (coming soon)
  • Background Agents - Monitor and autonomously work on your behalf, doing things like auto-generating documentation and commit messages
  • Custom Agents - Build your own agents to help build and monitor your pipelines based on your specific use cases and business logic (coming soon!)

Benefits​

The Intelligence Core delivers significant advantages for data teams:

  • Accelerated development - Create pipelines up to 7x faster with AI assistance and automation
  • Reduced costs - Cut processing costs up to 83% through intelligent optimization
  • Enhanced collaboration - Shared metadata and documentation improve team communication
  • Improved reliability - Automated testing and validation reduce errors
  • Future-proof architecture - Extensible framework evolves with your needs

Next steps​

Explore how the Intelligence Core powers specific Ascend capabilities:

Last updated on