In Range Test
Test to check if a value is within a certain range.
InRangeTest
InRangeTest
is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the InRangeTest
. Each property links to the specific details section further down in this page.
Property | Default | Type | Required | Description |
---|---|---|---|---|
severity | error | string ("error", "warn") | No | The severity level for issues raised by the test. Default is 'error'. Use 'error' for critical issues that should interrupt flow processing. Use 'warn' for warnings/minor issues that should not interrupt flow processing. |
in_range | Yes |
Property Details
Component
A Component is a fundamental building block of a data Flow. Supported Component types include: Read, Transform, Task, Test, and more.
Property | Default | Type | Required | Description |
---|---|---|---|---|
component | One of: CustomPythonReadComponent ApplicationComponent AliasedTableComponent ExternalTableComponent FivetranComponent | Yes | Component configuration options. |
AliasedTableComponent
Component that makes data in a pre-existing table available in the Ascend Flow.
Property | Default | Type | Required | Description |
---|---|---|---|---|
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
alias | Yes | Configuration options for the aliased table Component. |
CustomPythonReadComponent
Component that reads data using user-defined custom Python code.
Property | Default | Type | Required | Description |
---|---|---|---|---|
data_plane | One of: SnowflakeDataPlane BigQueryDataPlane DuckdbDataPlane DatabricksDataPlane | No | Data Plane-specific configuration options for Components. | |
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
custom_python_read | Yes |
ExternalTableComponent
Component that constructs and updates an External Table. Currently supported for Snowflake only.
Property | Default | Type | Required | Description |
---|---|---|---|---|
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
external_table | Any of: | Yes | Configuration options for the External Table Component. |
FivetranComponent
Component that integrates with Fivetran to extract and sync data from various sources. This Component connects to a Fivetran Landing Zone application and synchronizes objects (tables, schemas, etc.) from a Fivetran connection to the configured destination. It can function as either: - A derived Component of a Fivetran Application Component (for multi-object connectors) - A top-level Component (for single-object connectors)
Property | Default | Type | Required | Description |
---|---|---|---|---|
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
fivetran | Yes | Configuration options for the Fivetran data synchronization component. |
ReadComponent
Component that reads data from a system.
Property | Default | Type | Required | Description |
---|---|---|---|---|
data_plane | One of: SnowflakeDataPlane BigQueryDataPlane DuckdbDataPlane DatabricksDataPlane | No | Data Plane-specific configuration options for Components. | |
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
read | One of: GenericFileReadComponent LocalFileReadComponent SFTPReadComponent S3ReadComponent GcsReadComponent AbfsReadComponent HttpReadComponent MSSQLReadComponent MySQLReadComponent OracleReadComponent PostgresReadComponent SnowflakeReadComponent BigQueryReadComponent DatabricksReadComponent | Yes | Read component that reads data from a system. |
TaskComponent
Property | Default | Type | Required | Description |
---|---|---|---|---|
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
task | One of: TaskSqlComponent TaskPythonComponent | Yes |
TransformComponent
Component that executes SQL or Python code to transform data.
Property | Default | Type | Required | Description |
---|---|---|---|---|
data_plane | One of: SnowflakeDataPlane BigQueryDataPlane DuckdbDataPlane DatabricksDataPlane | No | Data Plane-specific configuration options for Components. | |
skip | boolean | No | Boolean flag indicating whether to skip processing for the Component or not. | |
retry_strategy | No | Retry strategy configuration options for the Component if any exceptions are encountered. | ||
description | string | No | Brief description of what the model does. | |
metadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | ||
name | string | Yes | The name of the model | |
flow_name | string | No | Name of the Flow that the Component belongs to. | |
data_maintenance | No | The data maintenance configuration options for the Component. | ||
tests | No | Defines tests to run on this Component's data. | ||
transform | One of: SqlTransform PythonTransform SnowparkTransform PySparkTransform | Yes | Transform that executes SQL or Python code for data transformation. |
ComponentTestOptions
Options for component tests, including data quality tests and schema checks.
InRangeTestOptions
Configuration options for the in_range test.
Property | Default | Type | Required | Description |
---|---|---|---|---|
min | Any of: integer number string | Yes | Minimum value to expect. | |
max | Any of: integer number string | Yes | Maximum value to expect. |