Again this year, I had the pleasure of joining the Databricks DATA+AI World tour in Stockholm with two of my colleagues. The conference follows up on the DATA+AI Summit earlier this year and focuses on talks from Nordic companies and smaller breakout sessions.

In general, Databricks is really expanding in the world of AI, and there are so many interesting features to explore, but as it is not possible to join all sessions, you will notice that my three highlights below focus on my personal interests in engineering and data governance.

Unity Catalog

This session by Gustav Byberg Skyle was my favorite of the day! So much is happening in Unity Catalog, which makes it easier to govern your data, and with the added governance, it makes it much easier to support nice features and cool use cases. Some of the new features that I am most excited about are:

• Row and column level security has gone GA
• Attribute-based access control (ABAC) is on the way!
• Automated PII detection
• Table health indicator in the explorer to show table expectations passed
• Customizable request for access of visible tables in the Unity Catalog UI
• Metrics will come in the future!
• Large increases to lineage incl., federated tables, volumes, models, and serving endpoints

Databricks SQL 

The Databricks SQL session hosted by Joshua Green also deep-dived into lots of new features. A few things to highlight are more GenAI under the hood for advanced search and to automate dashboard creation, a new ERD relationship between tables with the possibility to publish the relation to Power BI and an exciting SQL library that includes multiple AI functions.

Cyber Security with Databricks

Cyber Security is often an overlooked aspect when setting up a Data Platform, therefore I found the session by Derek King very interesting. He presented how the large amount of data in the modern world brings new challenges to Cyber Security and how Databricks are working to help solve those challenges. Functionalities that Databricks provides are, e.g., IOC Matching and Cybersecurity Analytics with Splunk, and they have also made a new Databricks AI Security Framework.

I hope that some of these features also excite you!

Helle also wrote about the highlights from the Data + AI Summit 2024 this Summer. You can read about those highlights here.