💭 Concepts

curriculum

If you're working on this page as part of the Ascend Curriculum, click here to navigate back or here to move onto the next section.

Ascend is the Agentic Data Engineering platform. Effective use of AI starts with a solid engineering foundation. Ascend is a fully-featured data platform for ETL/ELT workloads and orchestration with familiar data engineering language:

• Data engineers work on an Ascend Project that is version-controlled in an Ascend Repository
• Data engineers develop Ascend Flows (data pipelines) for a Project in an Ascend Workspace
• Data engineers deploy that same Project to an Ascend Deployment for production use
• The development and deployment lifecycle in Ascend follows DataOps best practices
• Otto is your intelligent data engineering agent in Ascend, helping you every step of the way

Core architecture

🏢Instance

The top-level resource that houses all your Ascend Environments, Projects, and data infrastructure.

Dive deeper

🌍Environment

Isolated compute and storage environments for development, staging, and production workloads.

Dive deeper

📁Project

Version-controlled collections of data pipelines and configurations organized for team collaboration.

Dive deeper

🏗️Workspace

Development environments where data engineers build and test data pipelines before deployment.

Dive deeper

Building pipelines

🌊Flow

Data pipelines that orchestrate the movement and transformation of data through multiple Components.

Dive deeper

🔧Component

Individual building blocks for reading, transforming, and writing data in your pipelines.

Dive deeper

🔌Connection

Secure configurations for connecting to external data sources, destinations, and services.

Dive deeper

🎛️Parameters

Configurable values that make your Components and Flows reusable across different environments.

Dive deeper

Development lifecycle

🚀Deployment

Production environments where your tested data pipelines run reliably at scale.

Dive deeper

👤Profile

Environment-specific configurations that customize how your pipelines run in different contexts.

Dive deeper

📦Repository

Git-based version control integration for managing your data pipeline code and configurations.

Dive deeper

🔐Vault

Secure storage and management of secrets, credentials, and sensitive configuration data.

Dive deeper

Platform features

🤖Otto

Your intelligent data engineering agent that helps with development, troubleshooting, and optimization.

Dive deeper

⚡️DataOps

Best practices and methodologies for reliable, scalable data engineering workflows.

Dive deeper

🔄Automation

Scheduling and orchestration capabilities for running your data pipelines automatically.

Dive deeper

🚀Application

Data Applications that can be deployed and managed alongside your data pipelines.

Dive deeper

🔬Advanced features

Specialized capabilities for complex data engineering scenarios and enterprise requirements.

Dive deeper

Next steps

Start with the Instance page to understand the top-level resource and go from there.

If you have access to an Ascend Instance, you can use Otto to help you navigate the documentation and explain concepts!