Oracle Write Component

A component that writes data to a Oracle table

Examples

oracle_write_component_no_normalize_uppercase.yaml

component:
  write:
    connection: my-oracle-connection
    input:
      flow: my-flow
      name: my-input-component
    oracle:
      table:
        name: my_table
        schema: my_schema

oracle_write_configuration.yaml

component:
  write:
    connection: my-oracle-connection
    input:
      flow: my-flow
      name: my-input-component
    oracle:
      table:
        name: my_table
        schema: my_schema

oracle_write_incremental.yaml

component:
  write:
    connection: my-oracle-connection
    input:
      flow: my-flow
      name: my-input-component
    oracle:
      table:
        name: my_table
        schema: my_schema
    strategy:
      incremental: append  # Specifies a simple append write strategy.
      incremental_column: updated_at  # Name of the column to use for tracking incremental updates to the data.
      on_schema_change: append_new_columns

OracleWriteComponent

info

OracleWriteComponent is defined beneath the following ancestor nodes in the YAML structure:

• Component
• WriteComponent

Below are the properties for the OracleWriteComponent. Each property links to the specific details section further down in this page.

Property	Default	Type	Required	Description
connection		string	Yes	The name of the connection to use for writing data.
input		InputComponent	Yes	Input component name.
normalize		boolean	No	A boolean flag indicating if the output column names should be normalized to a standard naming convention when writing.
preserve_case		boolean	No	A boolean flag indicating if the case of the column names should be preserved when writing.
uppercase		boolean	No	A boolean flag indicating if the column names should be transformed to uppercase when writing.
strategy	full:   mode: drop_and_recreate	Any of:   string   FullWriteStrategy   IncrementalWriteStrategyWithSchemaChange   PartitionedWriteStrategyWithSchemaChange	No	Resource for write strategy.
pre_sql		Any of:   string   array[string]	No	SQL statements to execute before the main write operation. Can be a single SQL statement as a string or multiple statements as a list of strings.
post_sql		Any of:   string   array[string]	No	SQL statements to execute after the main write operation. Can be a single SQL statement as a string or multiple statements as a list of strings.
oracle		SingleTableWithSchema	Yes

Property Details

Component

A component is a fundamental building block of a data flow. Types of components that are supported include: read, transform, task, test, and more.

Property	Default	Type	Required	Description
component		One of:   ReadComponent   TransformComponent   TaskComponent   SingularTestComponent   CustomPythonReadComponent   WriteComponent   CompoundComponent   AliasedTableComponent   ExternalTableComponent	Yes	Configuration options for the component.

WriteComponent

Property	Default	Type	Required	Description
description		string	No	A brief description of what the model does.
metadata		ResourceMetadata	No	Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
name		string	Yes	The name of the model
flow_name		string	No	The name of the flow that the component belongs to.
skip		boolean	No	A boolean flag indicating whether to skip processing for the component or not.
write		One of:   BigQueryWriteComponent   SnowflakeWriteComponent   S3WriteComponent   SFTPWriteComponent   GcsWriteComponent   AbfsWriteComponent   MySQLWriteComponent   OracleWriteComponent	Yes

FullWriteStrategy

Container for specifying the full write strategy used in write components.

Property	Default	Type	Required	Description
full		FullWriteStrategyOptions	Yes	Options to use when fully writing data to a Write component.

FullWriteStrategyOptions

Resource options for full writes, including mode selection.

Property	Default	Type	Required	Description
mode		FullWriteModeEnum	Yes	Specifies the mode to use when fully writing data: 'drop_and_recreate' to drop the output table and recreate it.

FullWriteModeEnum

No properties defined.

IncrementalWriteStrategyWithSchemaChange

Incremental write strategy that defines how new data is merged with existing data, along with policy for handling schema changes.

Property	Default	Type	Required	Description
incremental		Any of:   string   MergeStrategy	Yes	Options to use when incrementally writing data to a Write component.
incremental_column		string	Yes	Name of the column to use for tracking incremental updates to the data.
on_schema_change		string ("ignore", "fail", "append_new_columns", "sync_all_columns")	No	Policy to apply when schema changes are detected. Defaults to 'fail' if not provided.

InputComponent

Specification for input components, including how partitioning behaviors should be handled. This additional metadata is required when a component is used as an input to other components in a flow.

Property	Default	Type	Required	Description
flow		string	Yes	Name of the parent flow that the input component belongs to.
name		string	Yes	The input component name.
alias		string	No	The alias to use for the input component.
partition_spec		Any of:   string ("full_reduction", "map")   RepartitionSpec	No	The type of partitioning to apply to the component's input data before processing the component's logic. Input partitioning is applied before the component's logic is executed.
where		string	No	An optional filter condition to apply to the input component's data.
partition_binding		Any of:   string   PartitionBinding	No	An optional partition binding specification to apply to the component on a per-output-partition basis against other inputs' partitions.

MergeStrategy

A strategy that involves merging new data with existing data by updating existing records that match the unique key.

Property	Default	Type	Required	Description
merge		KeyOptions	No	Options for merge strategy.

KeyOptions

Column options needed for merge and SCD Type 2 strategies, such as unique key and deletion column name.

Property	Default	Type	Required	Description
unique_key		string	Yes	Column or comma-separated set of columns used as a unique identifier for records, aiding in the merge process.
deletion_column		string	No	Column name used in the upstream source for soft-deleting records. Used when replicating data from a source that supports soft-deletion. If provided, the merge strategy will be able to detect deletions and mark them as deleted in the destination. If not provided, the merge strategy will not be able to detect deletions.
merge_update_columns		Any of:   string   array[string]	No	List of columns to include when updating values in merge. These columns are mutually exclusive with respect to the columns in merge_exclude_columns.
merge_exclude_columns		Any of:   string   array[string]	No	List of columns to exclude when updating values in merge. These columns are mutually exclusive with respect to the columns in merge_update_columns.
incremental_predicates		Any of:   string   array[string]	No	List of conditions to filter incremental data.

PartitionBinding

Property	Default	Type	Required	Description
logical_operator	logical_operator	string ("AND", "OR")	No	The logical operator to use to combine the partition binding predicates provided
predicates	predicates	array[string]	No	The list of partition binding predicates to apply to the input component's data

PartitionedWriteStrategyWithSchemaChange

Container for specifying the partitioned write strategy that supports different behaviors when schema changes.

Property	Default	Type	Required	Description
partitioned		PartitionedWriteStrategyOptions	Yes	Options to use when writing data in partitions to a Write component.
on_schema_change		string ("ignore", "fail", "append_new_columns", "sync_all_columns")	No	Policy to apply when schema changes are detected. Defaults to 'fail' if not provided.

PartitionedWriteStrategyOptions

Resource options for incremental writes, including mode selection and criteria for detecting deletions and unique records.

Property	Default	Type	Required	Description
mode		PartitionedWriteModeEnum	Yes	Specifies the mode to use when writing data in partitions: 'append' to append new or modified partitions, 'insert_overwrite' to insert new partitions and replace/overwrite modified partitions, and 'sync' to encompass both 'insert_overwrite' functionality and to delete partitions when deleted at the source.
partition_col		string	No	Column name used for partitioning, uses the internal Ascend partition identifier by default.

PartitionedWriteModeEnum

No properties defined.

RepartitionSpec

Specification for repartitioning operations on input component's data

Property	Default	Type	Required	Description
repartition		RepartitionOptions	No	Options for repartitioning the input component's data.

RepartitionOptions

Options for repartitioning the input component's data.

Property	Default	Type	Required	Description
partition_by		string	Yes	The column to partition by.
granularity		string	Yes	The granularity to use for the partitioning.

SingleTableWithSchema

Options for reading from a single table in a specific schema.

Property	Default	Type	Required	Description
table		TableWithSchemaOptions	Yes	Table (in specified schema) to read data from.

TableWithSchemaOptions

Options for reading from a specific table in a schema.

Property	Default	Type	Required	Description
name		string	Yes	Name of the table to be read.
schema		string	No	Schema of the table, if applicable.