Overview

dbt (data build tool) is an open-source tool for data analysts that allows transforming data in a database or data warehouse by writing SQL SELECT statements and converting them to tables and views.

Skyvia hosts dbt Core - an open-source CLI from dbt and allows you to run dbt transformations. You can link your dbt models from a git repository with target databases and run the transformations manually or on schedule.

Supported dbt Core Versions

Skyvia supports the following dbt Core versions:

• 1.7.0
• 1.10.11

Target

dbt Transformations are performed against a target database or a data warehouse. Skyvia supports the following target databases and data warehouses:

• Google BigQuery - see dbt provider documentation
• SQL Server - see dbt provider documentation
• PostgreSQL - see dbt provider documentation
• Amazon Redshift - see dbt provider documentation
• Snowflake - see dbt provider documentation
• MySQL - see dbt provider documentation

MySQL is not supported in dbt Core 1.10.x versions. You need to use dbt Core version 1.7.0 if you use MySQL as a target.

Git

Skyvia links your dbt projects from Git repository. You can create and edit dbt projects, for example, Visual Studio Code with dbt Fusion extension or any other tool, upload them to a Git repository, and then use them in dbt Transformations on Skyvia.

Skyvia ignores the target database, specified in the dbt project configuration. It runs dbt Transformations against a database connections, selected as their targets.

Skyvia allows you to link projects from GitHub using a GitHub connection or connect to any Git repository available via HTTPS. Skyvia requires at least Read-only access to the Git repository with your dbt project.

Using dbt Transformations

dbt Transformations are often used to prepare data for data analysis, improve data quality, etc. You can use them on Skyvia separately or together with other integration tools in order to implement database data transformation before or after data loading process. For example, you can implement ELT process in full on Skyvia, by extracting data from a source and loading it to a data warehouse with a replication and then transforming it and preparing for the data analysis with a dbt Transformation.

You can orchestrate your dbt Transformations either by scheduling them to run before or after other integrations or you can use Control Flow and connect different integrations with dbt transformations into a sequence of actions.

When Skyvia runs a dbt Transformation, it runs the dbt build command.