Key Terminology
Ascend enhances data engineering by employing familiar terms and workflows, making it intuitive for data professionals. At its core, Ascend uses terms and workflows familiar to data engineers, aligning closely with the tasks you perform every day. Here are some key concepts:
Language of the Data Engineer
What are the basic concepts/words you already use every day when you work with data pipelines?
When working with data, do you:
- Create projects for your data pipelines?
- Connect to systems to read and write data?
- Transform data using SQL or Python?
- Run tasks by executing code in sequence?
- Organize tasks into a flow of events that must occur in order?
If any of the above sound similar to what you do every day, you'll be right at home with Ascend. Our language is derived from the basic concepts you already know and use.
Data Pipeline and Data Flow
A data pipeline puts together all the pieces to ensure that data flows in systems that connect people, processes, software, and machines.
As you know, data pipelines often include much more than just data, such as business process/workflow-like logic, complex rules for orchestration/automation, scheduling, event-driven behaviors, and more.
You will often see the phrase "data pipeline" and "data flow" used interchangeably in our industry. At Ascend, we choose to distinguish between "data pipelines" and "data flows" to simplify and clarify the value and most important concept - flow.
A data pipeline describes the thing you assemble from all the machinery you put together. It is correct to say you build data pipelines with Ascend. It is even more powerful to say that you enable data and process flow in (and between) your organizations with Ascend.
Flow is the most important and ultimate function of data pipelines.
Declarative Programming
Ascend is a declarative programming platform that empowers users to blend simple, low-code configurations in YAML with Python, SQL, or other languages, facilitating the creation of sophisticated data processing flows.
A fundamental aspect of Declarative Programming deals with "simple building blocks" that we often refer to as primitives in Ascend. As you read on, you will see that these primitives form a simple language that you can use to define your data flows and automations quickly and easily. With Ascend, you entrust the complex orchestration, automation, and execution tasks to the platform, and focus your effort on the tasks of highest value - your business logic, process flow, and transformation code.
You will learn that this is the ultimate goal of declarative programming and Ascend in general - describe what you want the system to do in easy terms, write only the code you need to write, and let advanced automation and the platform handle the rest.
Flex Code
Ascend introduces Flex Code, a concept that encourages writing "just the right amount" of SQL or Python code to achieve optimal functionality in your flows and automations.
Ascend is not a visual drag-and-drop builder tool, by design. Ascend is designed for analytics, data, and AI/ML/LLM engineers, data scientists, and other technical users who know how to perform basic coding tasks in SQL and Python.