Skip to main content

Tests

Ascend offers two approaches to validation: tests applied within Components (like Read Components and Transforms) and standalone Test Components dedicated solely to validation. Both ensure your data meets quality and integrity criteria, from simple column checks to complex cross-dataset validations.

Test types​

Test typeDescription
Column-level testPre-defined tests that validate individual columns in your data. These tests check properties like non-null values, uniqueness, and value ranges. Add them directly to Component decorators or as Jinja blocks in SQL.
Component-level testPre-defined tests that validate the entire output of a Component rather than specific columns. These tests check table counts, schema validation, and other Component-level properties. Add them directly to Component decorators or as Jinja blocks in SQL.
Test ComponentDedicated Components whose sole purpose is testing. These Components contain custom logic in Python or SQL and perform complex validations that aren't possible with native tests.
Generic testA kind of Test Component decorated with @generic_test that run custom validation logic and return a TestResult object. Use these for reusable, parameterized test functions.
Singular testA kind of Test Component decorated with @singular_test that run custom validation logic and return a TestResult object. Use these for specialized test cases that require custom processing.

Test Components (the last two rows in the table) are standalone entities dedicated specifically to validation tasks, while column-level and Component-level tests are added to existing data processing Components.

info

See the Test Reference for more information on the syntax and capabilities of all test types.

Native tests​

Native tests are reusable, pre-built validators for your data. Apply them to both Python and SQL Components to verify data quality at two levels:

  • Column level: Validate specific columns (e.g., check for null values or unique entries)
  • Component level: Validate entire datasets (e.g., verify row counts or schema structure)
tip

You'll know it's a Component-level test when no column arguments are passed!

Column-level tests​

Column-level tests validate individual columns in your dataset. Common use cases include:

  • Data presence: Ensure columns contain values (not_null, not_empty)
  • Uniqueness: Verify columns have unique values (unique, count_distinct_equal)
  • Value ranges: Check that numeric values fall within expected bounds (greater_than, less_than, between)
  • Data formats: Validate strings match expected patterns (matches_regex, length_equal)
  • Reference integrity: Ensure values exist in lookup tables (accepted_values, relationships)

Component-level tests​

Component-level tests validate the entire output of a Component. Common use cases include:

  • Record counts: Verify the expected number of rows (count_equal, count_between)
  • Schema validation: Ensure the output schema matches expectations (schema_test)
  • Data freshness: Check that data is recent enough for downstream processing
  • Cross-table consistency: Validate relationships between multiple datasets

Native tests vs. Test Components​

Use native tests when:

  • Performing common validations (null checks, uniqueness, ranges)
  • Testing individual columns or basic dataset properties
  • You need simple, reusable validation logic
  • The validation can be expressed with pre-built test functions

Use Test Components when:

  • Implementing complex business logic validations
  • Testing relationships across multiple datasets
  • Performing statistical analysis or data profiling
  • Creating custom validation rules specific to your domain
  • You need full control over the test logic and error handling

Next steps​

Apply what you've learned by following our implementation guides:

Last updated on