At this year’s Coalesce 2023 conference, the dbt team unveiled an array of exciting features designed to empower analytics engineers. These innovations promise to streamline and elevate the entire data workflow. In this article, we’ll dive into these exciting updates, exploring their potential impact on your work with dbt.
The dbt Cloud IDE now includes several new key improvements. Dependencies will now be internalized, deleting the need for running ‘dbt deps’ when installing packages. Additionally, formatting and streamlining code is now made effortless with automatic integration features like SqlFluff and sqlfmt.
Plus, CI/CD processes become easily available as you can simply select ‘Continuous Integration’ as a job type. Other exciting features are the automatic cancellation of stale CI jobs and the option to rerun a failed test from the point of failure, saving time and computing power. Great stuff!
As the data mesh concept continues to be a hot topic, dbt introduces its own architecture to split large projects into domain-specific ones. These projects can cross-reference each other, allowing for the expansion of existing work but within new projects.
This will be incredibly exciting for many of our customers with large projects, making developing much more manageable. It is also great news for organizations with strict data governance, as different projects will be controlled by data contracts and access controls.
Embracing your preferred tools, dbt Cloud CLI brings dbt to IDEs like VS Code and Vim. This addition eliminates the hassle of manual integration while including cloud functionalities like authentication.
Developers will have easier access to dbt in their favorite terminal, making dbt development more enjoyable.
The semantic layer of dbt also receives a makeover powered by MetricFlow, an advanced query generator. This enables complex metric types, while SQL becomes more legible.
Excitingly, dbt also announced that Cloud is set to support Microsoft Fabric and Azure Synapse adapters and that Tableau and Google Sheets join the growing list of available integrations.
dbt Explorer, designed to seamlessly integrate with dbt Mesh, offers a visual tour of your projects and their dependencies, with an improved search function and access to metadata. If a company has many domain-specific projects powered by dbt Mesh, a comprehensive overview of the whole dbt landscape can now be visualized and shared across the organization.
It will be especially interesting to work with this feature to power efficient onboarding and handovers.
“In summary, dbt’s latest innovations are not only exciting but also transformative. They promise a smoother experience as an analytics engineer, with several automations and the promising introduction of dbt Mesh and Explore.
We are all excited to dive into the new features! ”