💭 Concepts

Ascend is the Agentic Data Engineering platform. Effective use of AI starts with a solid engineering foundation. Ascend is a fully-featured data platform for ETL/ELT workloads and orchestration with familiar data engineering language:

Data engineers work on an Ascend Project that is version-controlled in an Ascend Repository
Data engineers develop Ascend Flows (data pipelines) for a Project in an Ascend Workspace
Data engineers deploy that same Project to an Ascend Deployment for production use
The development and deployment lifecycle in Ascend follows DataOps best practices
Otto is your intelligent data engineering agent in Ascend, helping you every step of the way

Core architecture

🏢Instance

The top-level resource that houses all your Ascend Environments, Projects, and data infrastructure.

🌍Environment

Isolated compute and storage environments for development, staging, and production workloads.

📁Project

Version-controlled collections of data pipelines and configurations organized for team collaboration.

🏗️Workspace

Development environments where data engineers build and test data pipelines before deployment.

Building pipelines

🌊Flow

Data pipelines that orchestrate the movement and transformation of data through multiple Components.

🔧Component

Individual building blocks for reading, transforming, and writing data in your pipelines.

🔌Connection

Secure configurations for connecting to external data sources, destinations, and services.

🎛️Parameters

Configurable values that make your Components and Flows reusable across different environments.

Development lifecycle

🚀Deployment

Production environments where your tested data pipelines run reliably at scale.

👤Profile

Environment-specific configurations that customize how your pipelines run in different contexts.

📦Repository

Git-based version control integration for managing your data pipeline code and configurations.

🔐Vault

Secure storage and management of secrets, credentials, and sensitive configuration data.

Platform features

🤖Otto

Your intelligent data engineering agent that helps with development, troubleshooting, and optimization.

⚡️DataOps

Best practices and methodologies for reliable, scalable data engineering workflows.

🔄Automation

Scheduling and orchestration capabilities for running your data pipelines automatically.

🚀Application

Data Applications that can be deployed and managed alongside your data pipelines.

🔬Advanced features

Specialized capabilities for complex data engineering scenarios and enterprise requirements.

Next steps

Start with the Instance page to understand the top-level resource and go from there.

If you have access to an Ascend Instance, you can use Otto to help you navigate the documentation and explain concepts!

Core architecture​