Components

Components are the elements of Data Flow that enable you to query, process, store data, and apply custom logic. Arrange them according to your preference to build your custom ETL pipeline.

Source Components

Source components get data from the data source so you can use this data later in your data flow. There two types of source components: Source and CSV Source. Choose one depending on where you are getting data from.

Component Description
Source Source Gets data from a database or cloud application and adds it to the data flow scope.
CSV Source CSV Source Gets data from the CSV file and adds it to the data flow scope. You can use a local CSV file or the one from the file storage service.

Target Components

Target components are used to modify data in a data source and create logs if needed. There three types of target components: Target, CSV Target, and JSON Target. To modify data in data source target components use actions. There are custom actions that are specific to certain connections and four actions available for each connection by default: Insert, Update, Delete, and Execute Command.

Component Description
Target Target Uses connection to choose a data source and modifies its data based on selected actions.
CSV Target CSV Target Inserts data to the CSV file that can be stored locally or on the storage service.
JSON Target JSON Target Inserts data to the JSON file that can be stored locally or on the storage service.

Transformation Components

Transformation components are in charge of data transformation and custom data flow logic. With their help you can create custom data pipelines and apply data transformations before uploading to data target destination.

Component Description
Split Split Divides data flow into multiple branches and allow actions to be executed in parallel.
Conditional Split Conditional Split Enables you to route records based on conditions.
Bufferizer Bufferizer Sets a specific size of the batch.
Row Count Row Count Counts the number of input rows and writes the result to a variable.
Lookup Lookup Matches input records with records from another data source and adds columns of the matched records to the scope.
Extend Extend Receives data from the output of the previous component and modifies it. You can use it to add calculated fields, change the field's type, modify nested objects, and more.
Unwind Unwind Deconstructs nested type properties.