Skip to main content
Version: 3.0.0

S3 Connection

Connection to a bucket in Amazon S3.

Examples​

connection:
s3:
root: "s3://my-bucket/"

S3Connection​

info

S3Connection is defined beneath the following ancestor nodes in the YAML structure:

Below are the properties for the S3Connection. Each property links to the specific details section further down in this page.

PropertyDefaultTypeRequiredDescription
descriptionstring
NoA brief description of what the model does.
metadataNoMeta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.
namestringYesThe name of the model
s3YesThe S3 connection options.

Property Details​

Connection​

A connection to a data source/sink.

PropertyDefaultTypeRequiredDescription
connectionOne of:
  S3Connection
  GcsConnection
  AbfsConnection
  LocalFileConnection
  SnowflakeConnection
  BigQueryConnection
  MSSQLConnection
  MySQLConnection
  OracleConnection
  PostgresConnection
  HttpConnection
  DuckDBConnection
  SFTPConnection
  DatabricksConnection
YesThe connection to the data system.

S3ConnectionOptions​

Amazon S3 connection options.

PropertyDefaultTypeRequiredDescription
regionstring
NoThe AWS region to connect to.
rootstringYesthe s3 URL for the root prefix.
aws_access_key_idstring
NoThe access key ID to use for the S3 connection.
aws_secret_access_keystring
NoThe secret access key to use for the S3 connection.

ResourceMetadata​

Meta information of a resource. In most cases it doesn't affect the system behavior but may be helpful to analyze project resources.

PropertyDefaultTypeRequiredDescription
sourceNoThe origin or source information for the resource.
source_event_uuidstring
NoUUID of the event that is associated with creation of this resource.

ResourceLocation​

The origin or source information for the resource.

PropertyDefaultTypeRequiredDescription
pathstringYesPath within repository files where the resource is defined.
first_line_numberinteger
NoFirst line number within path file where the resource is defined.