Compound Component
Specification for a compound component. A compound component is composed of one or more sub-components. One way to think of compound components is as a group of logically related components. Compound components are typically generated by an Application by passing in some configuration. However, this isn't a strict requirement, and a compound component can be declared directly in a DAG as well.
CompoundComponent
CompoundComponent is defined beneath the following ancestor nodes in the YAML structure:
Below are the properties for the CompoundComponent. Each property links to the specific details section further down in this page.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| name | string | No | The name of the model | |
| description | string | No | A brief description of what the model does. | |
| metadata | ResourceMetadata | No | Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources. | |
| compound | CompoundComponentOptions | Yes | Configuration options for Compound Component. | |
| flow_name | string | No | The name of the flow that the component belongs to. | |
| skip | boolean | No | A boolean flag indicating whether to skip processing for the component or not. |
Property Details
Component
A component is a fundamental building block of a data flow. Types of components that are supported include: read, transform, task, test, and more.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| component | One of: ReadComponent TransformComponent TaskComponent SingularTestComponent CustomPythonReadComponent WriteComponent CompoundComponent AliasedTableComponent ExternalTableComponent | Yes | Configuration options for the component. |
CompoundComponentOptions
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| application | string | No | Reference to the Application that generates the sub-components of this compound component. | |
| application_config | object | Yes | Configuration passed to the Application to generate the sub-components of this compound component. The structure/shape of this configuration is defined by the Application. | |
| components | array[string] | Yes | List of components that make up the compound component. |
ResourceMetadata
Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| source | ResourceLocation | No | The origin or source information for the resource. | |
| source_event_uuid | string | No | UUID of the event that is associated with creation of this resource. |
ResourceLocation
The origin or source information for the resource.
| Property | Default | Type | Required | Description |
|---|---|---|---|---|
| path | string | Yes | Path within repository files where the resource is defined. | |
| first_line_number | integer | No | First line number within path file where the resource is defined. |