Connections

Connections in Ascend links Data Flows and their Components with external data sources and destinations. They provide the details needed to read and write data, facilitating data movement within Ascend's ecosystem. Connections support a wide range of external systems, including databases, lakes, warehouses, and APIs.

Key features

1. Standardized Integrations: Connect to various external data sources and destinations, from relational databases to cloud storage platforms, enabling seamless data ingestion and output.
2. No Code: Connections are defined in Ascend, without the need to write code, making it easy to connect to new systems.
3. Connectivity Testing: Ascend can test the connectivity to the external system to ensure that the connection is valid, and warn you if there are any issues.
4. Data Browser: Ascend includes a data browser to allow you to browse the data in the external system, preview the data, and more.
5. Configurable Settings: Customize configurations to meet the requirements of different systems, ensuring efficient and secure data access.
6. Parameterization: Connections can be parameterized to allow for different configurations of the same system. For example, you may have a single connection to a Snowflake instance, but parameterized with different schemas, tables, and/or query parameters for each Flow.

Where Connections are used

Flows & Data Planes

Every Flow is configured to run on a specific Data Plane, which must be connected to an external data source or destination. This ensures seamless data movement between Ascend and the underlying data platform.

Read & Write Components

Many Components in Ascend need to connect to external data sources and destinations. For example, a Read Component may need to connect to a lake or warehouse, and a Write Component may need to connect to a database.

Connections types

info

For complete details on the connections available and their configuration options, see the Connection Reference.

Ascend supports various connectors for flexible data integration and processing:

• Cloud Storage: Connect to services like Amazon S3, Google Cloud Storage, and Azure Blob Storage to handle large volumes of unstructured data.
• File Sources: Integrate with FTP/SFTP servers and local file systems for ingesting data from legacy systems or exporting processed data to on-premises environments.
• Warehouses: Read from and write to data warehouses like Snowflake, Google BigQuery, and Amazon Redshift for analytical data processing.
• Databases: Access structured and semi-structured data from relational and NoSQL databases, including PostgreSQL, MySQL, MongoDB, and SQL Server.
• APIs and Web Services: Integrate with RESTful APIs, SOAP services, and other web data sources to fetch or push data.
• Data Planes: Configure Data Planes within Ascend to control where data processing occurs, optimizing for performance and cost.

Best practices for Connections

Connections influence Data Plane configuration and workflow optimization. Understanding their role in data workflows is critical for efficient data pipeline design.

• Select the Right Connection: Choose Connections that align with your technical needs and cloud ecosystem to optimize data ingestion and workflow performance.
• Optimize Data Processing: Properly configured Connections enhance Data Plane settings, optimizing for data locality and cost.
• Secure Data Access: Configure security parameters to manage data access and ensure compliance with security standards.

Conclusion

Connections are a core component of Ascend, enabling seamless data flow and Data Plane configuration for optimized data processing. Their ability to integrate diverse data sources and destinations is key to building efficient, secure, and scalable data pipelines. Understanding Connections' strategic role helps you maximize Ascend's capabilities, ensuring your data workflows are efficient, scalable, and secure.