In Set Test

Test to check if a value is in a set of values.

InSetTest

info

InSetTest is defined beneath the following ancestor nodes in the YAML structure:

• Component
• AliasedTableComponent
• CustomPythonReadComponent
• ExternalTableComponent
• FivetranComponent
• ReadComponent
• TaskComponent
• TransformComponent
• ComponentTestOptions

Below are the properties for the InSetTest. Each property links to the specific details section further down in this page.

Property	Default	Type	Required	Description
severity	error	string ("error", "warn")	No	The severity level for issues raised by the test. Default is 'error'. Use 'error' for critical issues that should interrupt flow processing. Use 'warn' for warnings/minor issues that should not interrupt flow processing.
in_set			Yes

Property Details

Component

A Component is a fundamental building block of a data Flow. Supported Component types include: Read, Transform, Task, Test, and more.

Property	Default	Type	Required	Description
component		One of:   CustomPythonReadComponent   ApplicationComponent   AliasedTableComponent   ExternalTableComponent   FivetranComponent	Yes	Component configuration options.

AliasedTableComponent

Component that makes data in a pre-existing table available in the Ascend Flow.

Property	Default	Type	Required	Description
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
alias			Yes	Configuration options for the aliased table Component.

CustomPythonReadComponent

Component that reads data using user-defined custom Python code.

Property	Default	Type	Required	Description
data_plane		One of:     SnowflakeDataPlane     BigQueryDataPlane     DuckdbDataPlane     DatabricksDataPlane	No	Data Plane-specific configuration options for Components.
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
custom_python_read			Yes

ExternalTableComponent

Component that constructs and updates an External Table. Currently supported for Snowflake only.

Property	Default	Type	Required	Description
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
external_table		Any of:	Yes	Configuration options for the External Table Component.

FivetranComponent

Component that integrates with Fivetran to extract and sync data from various sources. This Component connects to a Fivetran Landing Zone application and synchronizes objects (tables, schemas, etc.) from a Fivetran connection to the configured destination. It can function as either: - A derived Component of a Fivetran Application Component (for multi-object connectors) - A top-level Component (for single-object connectors)

Property	Default	Type	Required	Description
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
fivetran			Yes	Configuration options for the Fivetran data synchronization component.

ReadComponent

Component that reads data from a system.

Property	Default	Type	Required	Description
data_plane		One of:     SnowflakeDataPlane     BigQueryDataPlane     DuckdbDataPlane     DatabricksDataPlane	No	Data Plane-specific configuration options for Components.
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
read		One of:   GenericFileReadComponent   LocalFileReadComponent   SFTPReadComponent   S3ReadComponent   GcsReadComponent   AbfsReadComponent   HttpReadComponent   MSSQLReadComponent   MySQLReadComponent   OracleReadComponent   PostgresReadComponent   SnowflakeReadComponent   BigQueryReadComponent   DatabricksReadComponent	Yes	Read component that reads data from a system.

TaskComponent

Property	Default	Type	Required	Description
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
task		One of:   TaskSqlComponent   TaskPythonComponent	Yes

TransformComponent

Component that executes SQL or Python code to transform data.

Property	Default	Type	Required	Description
data_plane		One of:     SnowflakeDataPlane     BigQueryDataPlane     DuckdbDataPlane     DatabricksDataPlane	No	Data Plane-specific configuration options for Components.
skip		boolean	No	Boolean flag indicating whether to skip processing for the Component or not.
retry_strategy			No	Retry strategy configuration options for the Component if any exceptions are encountered.
description		string	No	Brief description of what the model does.
metadata			No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	Name of the Flow that the Component belongs to.
data_maintenance			No	The data maintenance configuration options for the Component.
tests			No	Defines tests to run on this Component's data.
transform		One of:   SqlTransform   PythonTransform   SnowparkTransform   PySparkTransform	Yes	Transform that executes SQL or Python code for data transformation.

ComponentTestOptions

Options for component tests, including data quality tests and schema checks.

Property	Default	Type	Required	Description
columns		object with property values of type array[One of: (Any of: (not_null, NotNullTest), Any of: (not_empty, NotEmptyTest), Any of: (unique, UniqueTest), CombinationUniqueTest, InRangeTest, DateInRangeTest, InSetTest, SubstringMatchTest, CountDistinctEqualTest, CountGreaterThanOrEqualTest, CountGreaterThanTest, CountLessThanOrEqualTest, CountLessThanTest, CountEqualTest, GreaterThanTest, LessThanTest, GreaterThanOrEqualTest, LessThanOrEqualTest, MeanInRangeTest, StddevInRangeTest, ColumnTestSql, ColumnTestPython)]	No	List of column-level data quality tests for a Component.
component		array[One of: (Any of: (not_null, NotNullTest), Any of: (not_empty, NotEmptyTest), Any of: (unique, UniqueTest), CombinationUniqueTest, InRangeTest, DateInRangeTest, InSetTest, SubstringMatchTest, CountDistinctEqualTest, CountGreaterThanOrEqualTest, CountGreaterThanTest, CountLessThanOrEqualTest, CountLessThanTest, CountEqualTest, GreaterThanTest, LessThanTest, GreaterThanOrEqualTest, LessThanOrEqualTest, MeanInRangeTest, StddevInRangeTest, ColumnTestSql, ColumnTestPython)]	No	List of Component-level tests.
schema			No	List of schema checks for a Component.

InSetTestOptions

Configuration options for the in_set test.

Property	Default	Type	Required	Description
values		array[Any of: (integer, number, string)]	Yes	Set of values to expect.