RuntimeSize
Enumeration of available runtime sizes for deployments, workspaces, and flow runners.
Each size corresponds to specific resource allocations (CPU, memory, disk).
RuntimeSize
info
RuntimeSize is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the RuntimeSize. Each property links to the specific details section further down in this page.
No properties defined.
Property Details
BackfillRun
Defines the parameters for a backfill run.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| backfill_run | Yes | Backfill run options. |
BackfillRunOptions
Options for a backfill run.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| description | string | No | Brief description of what the model does. | |
| metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
| name | string | Yes | The name of the model | |
| flow_name | string | Yes | Name of the Flow that is to be backfilled. | |
| start_time | string | Yes | Start time of the time range to be backfilled. | |
| end_time | string | Yes | End time of the time range to be backfilled. | |
| granularity | string ("day", "week", "month") | Yes | Time granularity to use for backfill. Must be one of: 'day', 'week', 'month'. The backfill runner divides the date range into Flow runs of this granularity and launches these Flow runs. | |
| max_concurrent_flow_runs | 1 | integer | No | Maximum number of concurrent Flow runs used for backfill. This is used to limit the number of Flow runners (and hence cluster resources) that are launched simultaneously. |
| backfill_order | string ("forward_chronological", "reverse_chronological") | No | Order to use for backfilling - either forward or reverse chronological order. | |
| flow_run_options | No | Additional options for each Flow run launched during the backfill. | ||
| run_final_sync | boolean | No | Boolean flag indicating whether to run a final sync after concurrent backfill Flow runs. This final sync is a single Flow run that is executed without any time parameters, and is meant to sync the data to the latest state and capture any missing time intervals. |
Flow
A Flow is the primary unit of execution in Ascend and contains a collection of Components assembled into a directed acyclic graph (DAG).
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| flow | Yes |
FlowOptions
Defines the options for a Flow
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| parameters | object with property values of type None | No | Dictionary of parameters to use for resource. | |
| defaults | array[None] | No | List of default configs with filters that can be applied to a resource config. | |
| description | string | No | Brief description of what the model does. | |
| metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
| name | string | Yes | The name of the model | |
| data_plane | No | Data plane to use for the flow. | ||
| version | string | No | Flow version. | |
| bootstrap | string | No | Bootstrap command to run within the Docker container. | |
| runner | RunnerConfig | No | Runner configuration. | |
| component_concurrency | integer | No | Maximum number of concurrent Components to run within this Flow. |
FlowRun
Defines the run-specific parameters for a Flow, one flow can have multiple Flow runs
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| flow_run | Yes |
FlowRunBaseOptions
Base options for a Flow Run
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| parameters | object with property values of type None | No | Dictionary of parameters to use for resource. | |
| defaults | array[None] | No | List of default configs with filters that can be applied to a resource config. | |
| description | string | No | Brief description of what the model does. | |
| metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
| run_tests | True | boolean | No | Boolean flag indicating whether to run tests after processing data. |
| store_test_results | boolean | No | Boolean flag indicating whether to store test results. | |
| components | array[string] | No | List of Component names to run. | |
| component_categories | array[string] | No | List of Component categories to run. | |
| halt_flow_on_error | boolean | No | Boolean flag indicating whether to halt the Flow on error. | |
| disable_optimizers | boolean | No | Boolean flag indicating whether to disable optimizers. | |
| disable_incremental_metadata_collection | boolean | No | Boolean flag indicating whether to disable collection of Incremental Read and Transform Component metadata. | |
| full_refresh | False | boolean | No | Boolean flag indicating whether to perform a full refresh of each Component. ⚠ If true, will drop all internal data and metadata tables/views and re-compute them from scratch. |
| update_materialization_type | False | boolean | No | Boolean flag indicating whether to update Component materialization types (e.g., changing types between 'simple', 'view', 'incremental', and 'smart'). ⚠ If materialization type changes are detected, existing data and metadata tables/views will be dropped and re-computed from scratch. Otherwise, existing data and metadata tables/views will be preserved and type changes will result in an error. |
| runner_overrides | RunnerConfig | No | Override runner configuration for this specific flow run. If not specified, inherits from the flow's runner configuration, or the deployment/workspace defaults. |
FlowRunOptions
Options for a Flow Run
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| parameters | object with property values of type None | No | Dictionary of parameters to use for resource. | |
| defaults | array[None] | No | List of default configs with filters that can be applied to a resource config. | |
| description | string | No | Brief description of what the model does. | |
| metadata | No | Meta information of a Flow run. In most cases, it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
| run_tests | True | boolean | No | Boolean flag indicating whether to run tests after processing data. |
| store_test_results | boolean | No | Boolean flag indicating whether to store test results. | |
| components | array[string] | No | List of Component names to run. | |
| component_categories | array[string] | No | List of Component categories to run. | |
| halt_flow_on_error | boolean | No | Boolean flag indicating whether to halt the Flow on error. | |
| disable_optimizers | boolean | No | Boolean flag indicating whether to disable optimizers. | |
| disable_incremental_metadata_collection | boolean | No | Boolean flag indicating whether to disable collection of Incremental Read and Transform Component metadata. | |
| full_refresh | False | boolean | No | Boolean flag indicating whether to perform a full refresh of each Component. ⚠ If true, will drop all internal data and metadata tables/views and re-compute them from scratch. |
| update_materialization_type | False | boolean | No | Boolean flag indicating whether to update Component materialization types (e.g., changing types between 'simple', 'view', 'incremental', and 'smart'). ⚠ If materialization type changes are detected, existing data and metadata tables/views will be dropped and re-computed from scratch. Otherwise, existing data and metadata tables/views will be preserved and type changes will result in an error. |
| runner_overrides | RunnerConfig | No | Override runner configuration for this specific flow run. If not specified, inherits from the flow's runner configuration, or the deployment/workspace defaults. | |
| name | string | No | Flow run name. | |
| flow_name | string | Yes | Name of the Flow to run. | |
| event_start_time | string | No | Event start time to be used for time-series processing. | |
| event_end_time | string | No | Event end time to be used for time-series processing. |
RunnerConfig
Configuration for the flow runner
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| size | RuntimeSize | No | Override the size of the flow runner. If not specified, the flow runner inherits the size from the deployment or workspace. |