dbt-trino

Introduction

dbt is a data transformation workflow tool that lets teams quickly and collaboratively deploy analytics code, following software engineering best practices like modularity, CI/CD, testing, and documentation. It enables anyone who knows SQL to build production-grade data pipelines.

One frequently asked question in the context of using dbt tool is:

Can I connect my dbt project to two databases?

(see the answered question on the dbt website).

TL;DR dbt stands for transformation as in T within ELT pipelines, it doesn't move data from source to a warehouse.

dbt-trino adapter uses Trino as a underlying query engine to perform query federation across disperse data sources. Trino connects to multiple and diverse data sources (available connectors) via one dbt connection and process SQL queries at scale. Transformations defined in dbt are passed to Trino which handles these SQL transformation queries and translates them to queries specific to the systems it connects to create tables or views and manipulate data.

This repository represents a fork of the dbt-presto with adaptations to make it work with Trino.

Compatibility

This dbt plugin has been tested against Trino version 414, Starburst Enterprise version 413-e and Starburst Galaxy.

Setup & Configuration

For information on installing and configuring your profile to authenticate to Trino or Starburst, please refer to Starburst and Trino Setup in the dbt docs.

Trino- and Starburst-specific configuration

For Trino- and Starburst-specific configuration, you can refer to Starburst (Trino) configurations on the dbt docs site.

Contributing

• Want to report a bug or request a feature? Let us know on Slack in the #db-presto-trino channel, or open an issue
• Want to help us build dbt-trino? Check out the Contributing Guide

Release process

Before doing a release, it is required to bump the dbt-trino version by triggering release workflow version-bump.yml. The major and minor part of the dbt version are used to associate dbt-trino's version with the dbt version.

Next step is to merge the bump PR and making sure that test suite pass.

Finally, to release dbt-trino to PyPi and GitHub trigger release workflow release.yml.

Backport process

Sometimes it is necessary to backport some changes to some older versions. In that case, create branch from x.x.latest branch. There is a x.x.latest for each minor version, e.g. 1.3.latest. Make a fix and open PR back to x.x.latest. Create changelog by changie new as ususal, as separate changlog for each minor version is kept on every x.x.latest branch. After merging, to make a release of that version, just follow instructions from Release process section, but run every workflow on x.x.latest branch.

Code of Conduct

Everyone interacting in the dbt project's codebases, issue trackers, chat rooms, and mailing lists is expected to follow the PyPA Code of Conduct.