Parameters and Variables

Parameters

Parameters allow you to store values between data flow executions. You can create any number of parameters of any type in a Data Flow. Besides, for any parameter, a variable with the same name is automatically created in a Data Flow. When the data flow execution starts, Skyvia assigns values of data flow parameters to these variables, and when the run finished, Skyvia stores values of these variables in data flow parameters.

You can use parameters whenever you need to pass values from previous data flow executions to next data flow executions. For example, you may query records once, store max value of record creation time in a parameter, and query only records with the created time more than this value during the next data flow execution.

Data flow parameters described here are not the same as action parameters, which exist only within an action of a component and are used for passing input column values and variable values into the action.

Creating Parameters

If you want to add a parameter to the data flow package, click Parameters in the toolbar on the left. When the parameter editor window opens, click to add and adjust its settings. Give a name to the parameter, select its type from the drop-down list, enter a value and click Save. The same way you add another parameter if needed.

Using Parameters

You can use parameters whenever you need to pass values from previous data flow executions to next data flow executions. For example, you can use parameters to load only the new data each time the data flow runs.

Whether the record is new, can be determined in different ways — by storing record creation or last modification time like most data in a cloud sources, or using an autoincrement primary keys or version fields as it is often done in databases. In this way you can store the max value of such a field in a parameter using a Value component, and query only records with this field greater than the value of this parameter in a Source component next time.

Variables

Variables are used to store values during the data flow execution. Components, like Count or Value, can assign values to them, and you can use their values in other components.

Creating Variables

To add a variable to the data flow package, click Variables in the toolbar on the left. This opens the Variable sidebar, which displays all your variables, including variables created automatically for parameters. Click to add and adjust its settings. In the opened window, name a variable, select its type from the drop-down list and click Save.

Using Variables

You can use variables, for example, to count records from certain outputs, using Count component, and then define the number of success and failed records based on these variables. These numbers are then written to package send the number to the data flow log as the number of successful and failed records.