Skip to main content

Manage custom package dependencies

Add Python packages to your Ascend Project by creating a pyproject.toml file in your Project's root directory.

This is the recommended approach for:

  • Private packages from internal PyPI registries (AWS CodeArtifact, GCP Artifact Registry)
  • Pinning specific package versions
  • Following Python packaging standards

Prerequisites​

Authentication​

Ascend authenticates to private package repositories using the cloud identity of your Environment. To enable access to your private packages:

Grant read permissions to your Ascend Environment's cloud identity for your private package registry:

  • Google Artifact Registry: Add the Ascend Environment's service account to your Artifact Registry with Artifact Registry Reader permissions
  • AWS CodeArtifact: Grant the Ascend Environment's IAM role read access to your CodeArtifact repository

Ascend-managed vs. self-hosted Instances​

  • Ascend-managed Instances (recommended): You'll need to explicitly grant cross-cloud access since Ascend's managed service runs in a separate cloud environment
  • Self-hosted Instances: Authentication is typically pre-configured since your Kubernetes clusters usually run in the same cloud environment as your package repositories

For detailed instructions on obtaining your Environment's cloud identity details for granting access, see Environment identity information.

Once permissions are configured, package installation will work automatically without requiring additional credentials in your pyproject.toml file.

pyproject.toml vs. ascend_project.yaml​

pyproject.toml is the standard for Python package management, ensuring seamless maintenance of private packages and specific package versions.

Meanwhile, ascend_project.yaml remains the appropriate place for Ascend-specific configurations, particularly Data Plane information and default Flow specifications. These two files work together to provide complete Project configuration.

To learn more about overall Project setup, refer to this how-to guide.

Example pyproject.toml​

[project]
name = "my_project"
version = "0.1.0"
description = "Additional dependencies for Project Components"
requires-python = ">=3.12"

dependencies = [
  # Otto's Expeditions – Ascend sample project
  'ottos-expeditions>=0.0.5',
  # Local secrets
  'python-dotenv',
  # Project config
  'PyYAML',
  # CLI
  'rich',
  'typer',
  # GUI
  'streamlit',
  'shiny',
  'shinyswatch',
  'shinywidgets',
  # Data
  'ibis-framework[sqlite,duckdb,datafusion,polars,bigquery,snowflake,deltalake]',
  'snowflake-snowpark-python',
  'gcsfs',
  'fsspec',
  # Synthetic data
  'Faker',
  # Viz
  'plotly',
]
Last updated on