Skip to main content
Version: 3.0.0

Profile

This guide will walk you through setting up a Profile.

A Profile is a configuration tool in Ascend that serves as a collection of parameters and Flow defaults. It defines key/value pairs and Data Plane connections for a specific Project + Workspace combination.

  • Parameters are a collection of key/value pairs used to configure Components in a Project or Flow.
  • Defaults specify default Data Plane configurations for each Flow, by matching on exactly a Flow name or using a regex pattern.

Prerequisites​

  • Existing Ascend Project
  • Existing Environment with access to a Data Plane
  • Connection to your Data Plane (either BigQuery or Snowflake)

Creating a Profile​

  • Navigate to your Workspace on the Homepage
  • Create a new yaml file in the profiles directory. Be sure to include the .yaml suffix when you name the file.
  • Click the Save button.

To use a Profile, you will need to configure your Workspace to use it.

Defaults​

Defaults are a list of specifications Flows should use which Data Plane connection. They follow this syntax:

profile:
  defaults:
    - kind: <Flow | Component>
      name: <exact_flow_name>
      spec:
        data_plane:
          connection_name: <connection_name>

Or, alternatively you can use a regex pattern to match multiple Flows:

profile:
  defaults:
    - kind: <Flow | Component>
      name:
        regex: <regex_pattern_for_flow_name>
      spec:
        data_plane:
          connection_name: <connection_name>

Parameters​

Profiles can also flexibly set parameters which can be referenced elsewhere in the Project, including in Connection and Component definitions.

Parameters are created in Profiles using this syntax:

profile:
  parameters:
    <key_name>: <key_value>
    <top_level_key>:
      <nested_key>: <nested_value>

and are referenced using this syntax:

${parameter.path.to.key}

Examples​

For the following example project:

  your_project_name/
  ├── ascend_project.yaml
  ├── connections/
  │   ├── foo_bigquery_connection.yaml
  │   ├── foo_snowflake_connection.yaml
  │   └── bar_snowflake_connection.yaml
  ├── flows/
  ├── profiles/
  │   └── (+ New File) foo-profile.yaml
  └── vaults/

Below is an example of a Profile with the minimum requirements for configuration. It uses a single connection for all Flows and does not set any parameters.

your_project_name/profiles/foo-profile.yaml
profile:
  defaults:
    - kind: Flow
      name:
        regex: .*
      spec:
        data_plane:
          connection_name: foo_bigquery_connection

Below is an example of a Profile configuration that sets some parameters and uses regex to separate Flows into different Data Plane connections.

<your-project-root>/profiles/foo-profile.yaml
profile:
  parameters:
    dev_host: foo_value
    nested_field:
      dev_database: bar_value

  defaults:
    - kind: Flow
      name: bar_flow
      spec:
        data_plane:
          connection_name: bar_snowflake_connection

    - kind: Flow
      name:
        regex: foo.*
      spec:
        data_plane:
          connection_name: foo_snowflake_connection

These values can then be substituted into a MySQL connection and allow for using different external resources based on the Profile.

<your-project-root>/connections/foo-profile.yaml
connection:
  mysql:
    host: ${parameter.dev_host}
    port: 3306
    database: ${parameter.nested_field.dev_database}
    user: ${secret.my_vault.mysql.username}
    password: ${secret.my_vault.mysql.password}
Last updated on