(Ociacia/Shutterstock)
In terms of utilizing AI to speed up BI, practitioners are discovering out that semantics actually issues. In reality, while you’re utilizing AI to generate SQL to question a database desk, understanding precisely what information a specific desk holds is completely a requirement. So-called semantic layers perform as a type of map that enables the pure language question to work. It’s not shocking, then, {that a} battle is breaking out over management of this semantic layer.
Earlier than the age of AI, few folks anxious about semantic layers apart from these in command of implementing BI instruments and modeling information in information warehouses. In case your group was adopting Tableau or Looker atop an analytic database from Oracle or Teradata, you’d in all probability use the semantic instruments related to these BI merchandise to outline precisely how the “working income” database view was developed, for instance.
The semantic layer defines and enforces guidelines about information, resembling what information sources can be utilized, what transformations are required, what checks should be carried out. Your CFO want to know that the “working income” entry on her dashboard was calculated accurately–not only one time however each single time–and the semantic layer is a key piece of the analytics plumbing that ensured that occurred.
Constructing a semantic layer historically has been an information modeling train (Tee11/Shutterstock)
Since ChatGPT ignited the AI revolution again in 2022, the semantic layer has skilled a renaissance. That’s because of the downside alluded to above: Letting language fashions lose on a database with no map of what the info means is a path to poor information, frustration, and enterprise failure. With out a semantic layer to information the language mannequin’s entry to information, it’s nearly assured to make errors that can upset your CFO. That’s not one thing that you simply wish to get within the behavior of doing if job safety is essential to you.
The emergence of the common-or-garden semantic layer as a keystone within the AI-BI panorama has been occurring for the previous couple of years. Distributors that developed semantic layers that exist unbiased of BI instruments and permit organizations to standardize how their analysts work together with underlying databases, resembling AtScale and Dice, have stepped up their growth and advertising and marketing efforts to capitalize on the brand new demand. The parents at dbt Labs, developer of what’s arguably the world’s hottest information transformation software, additionally launched their very own semantic layer again in 2023.
Now the massive canines are beginning to eat on the semantic buffet. Snowflake launched its semantic views function throughout its latest Snowflake Summit 2025. To not be outdone, Databricks launched its personal tackle semantic views, Databricks Unity Catalog metric views, at its Databricks AI & Knowledge Summit 2025.
Snowflake Semantic Views
In its weblog asserting semantic views, Snowflake engineers describe semantic views as “a brand new schema-level object that shops all semantic mannequin data natively within the database.” It replaces the present Cortex Analyst YAML file, and offers an agreed-upon metadata commonplace for a spread of Snowflake experiences, together with AI-powered analytics, BI shoppers, Streamlit purposes, Workspaces, Notebooks, and customized purposes.
Snowflake’s semantic views perform is in beta
Snowflake offers analysts with a number of methods to create semantic views, together with through the use of the Cortex Analyst UI, the database object explorer in Snowsight, or by writing a customized DDL assertion. Snowflake says it has adopted a mannequin definition language that lets customers outline the core attributes of a semantic view, which incorporates bodily mannequin objects (tables or views; SQL queries to be added sooner or later); relationships between these bodily objects; dimensions (or business-friendly attributes that group and filter analytical queries); and metrics (or business-friendly calculations and aggregations that characterize KPIs). Semantic views are presently in beta.
“We developed semantic views in response to a standard buyer want,” the Snowflake engineers wrote. “Clients need AI-powered conversational analytics however fear about ungoverned information entry and inconsistent or inaccurate solutions.”
Databricks Unity Catalog Metric Views
Databricks is doing one thing much like its rival with its new Unity Catalog metric views. Unity Catalog, in fact, is the centralized information catalog and information governance providing that helps clients handle entry to information.
“Not like metrics outlined solely within the BI layer, which restrict reuse and integration, defining metrics on the information layer makes enterprise semantics reusable throughout all workloads, like dashboards, AI fashions, and information engineering jobs,” Databricks writes in its June 12 weblog publish. “Unity Catalog Metrics are additionally totally addressable by way of SQL to make sure that everybody within the group can have the identical view of metrics, no matter what software they select.”
Based on Databricks’ metric view documentation, metric views specify a set of definitions, together with dimensions (resembling product names or buyer varieties) and measures (the suitable values that summarize a enterprise exercise). Metric views are outlined in YAML and registered in Unity Catalog; customers can create them utilizing SQL or the Catalog Explorer UI.
Databricks Unity Catalog metrics view is anticipated to go GA this summer time
Metric views are ruled and auditable by default, Databricks says. “Licensed metrics include auditing and lineage out of the field, enabling trusted, compliant insights throughout groups,” the corporate says. Customers can create metrics as soon as in Unity Catalog, and use them throughout a number of instruments and experiences in Databricks, together with AI/BI Dashboards, Genie, Notebooks, SQL, and Lakeflow jobs, the corporate says. Sooner or later, Databricks say it can lengthen metric views to help BI instruments like Tableau, Hex, Sigma, ThoughtSpot, and Omni, in addition to observability instruments like Anomalo and Monte Carlo, the corporate writes in its weblog. Metric views are presently in beta, and are anticipated to be obtainable later this summer time.
Built-in or Impartial?
There may be clearly demand for a semantic layer, as Snowflake’s and Databricks’ strikes into the product class show. With out this vital metadata layer, pure language question of enterprise databases will stay an AI pipedream.
The massive query appears to be whether or not there may be sufficient momentum and demand for semantic layers to attain escape velocity and grow to be its personal product class, unbiased of the BI merchandise they have been initially tied to, resembling Looker and Tableau, or information platforms just like the Snowflake and Databricks. The strikes by Snowflake and Databricks recommend in any other case.
That’s to not say it couldn’t occur. Greater up within the stack, that demand and momentum appeared for an unbiased desk format to emerge: Apache Iceberg. Snowflake and Databricks laid down arms and determined to standardize on Iceberg, which was a win for purchasers and information independence and a loss for information lock-in and vendor greed.
Additional down the stack, the semantic layer has equally emerged as vital element for attaining information interoperability and bringing repeatability and reliability to AI-powered BI. Will the massive canines agree on a common layer that advantages everybody, or search to make semantic layers a aggressive benefit? Time will inform.
Associated Gadgets:
Rethinking AI-Prepared Knowledge with Semantic Layers
Is the Common Semantic Layer the Subsequent Large Knowledge Battleground?
Semantic Layer Belongs in Middleware, and dbt Desires to Ship It
