💭 Concepts
Ascend is the Agentic Data Engineering platform. Effective use of AI starts with a solid engineering foundation. Ascend is a fully-featured data platform for ETL/ELT workloads and orchestration with familiar data engineering language:
- Data engineers work on an Ascend Project that is version-controlled in an Ascend Repository
- Data engineers develop Ascend Flows (data pipelines) for a Project in an Ascend Workspace
- Data engineers deploy that same Project to an Ascend Deployment for production use
- The development and deployment lifecycle in Ascend follows DataOps best practices
- Otto is your intelligent data engineering Agent in Ascend, helping you every step of the way
Core architecture
🏢 Instance
The top-level resource that houses all your Ascend Environments, Projects, and data infrastructure.
🌍 Environment
Isolated compute and storage environments for development, staging, and production workloads.
📁 Project
Version-controlled collections of data pipelines and configurations organized for team collaboration.
🏗️ Workspace
Development environments where data engineers build and test data pipelines before deployment.
Building pipelines
🌊 Flow
Data pipelines that orchestrate the movement and transformation of data through multiple Components.
🔧 Component
Individual building blocks for reading, transforming, and writing data in your pipelines.
🔌 Connection
Secure configurations for connecting to external data sources, destinations, and services.
🎛️ Parameters
Configurable values that make your Components and Flows reusable across different environments.
Development lifecycle
🚀 Deployment
Production environments where your tested data pipelines run reliably at scale.
👤 Profile
Environment-specific configurations that customize how your pipelines run in different contexts.
📦 Repository
Git-based version control integration for managing your data pipeline code and configurations.
🔐 Vault
Secure storage and management of secrets, credentials, and sensitive configuration data.
Platform features
🤖 Otto
Your intelligent data engineering Agent that helps with development, troubleshooting, and optimization.
🏭 DataOps
Best practices and methodologies for reliable, scalable data engineering workflows.
🔄 Automation
Scheduling and orchestration capabilities for running your data pipelines automatically.
🚀 Application
Data Applications that can be deployed and managed alongside your data pipelines.
🔬 Advanced features
Specialized capabilities for complex data engineering scenarios and enterprise requirements.
Next steps
Start with the Instance page to understand the top-level resource and go from there.
If you have access to an Ascend Instance, you can use Otto to help you navigate the documentation and explain concepts!