Saying OIDC Token Federation for Enhanced Delta Sharing Safety

We’re excited to introduce the Public Preview of OIDC Token Federation for Enhanced Delta Sharing Safety a serious safety and usefulness enhancement for when sharing with non-Databricks recipients. With this launch, knowledge suppliers can securely share knowledge with non-Databricks customers on any computing platform preferring to authenticate utilizing a customized Identification Supplier (IdP), comparable to Azure Entra ID or Okta. This eliminates the necessity for static credentials, enhances safety, and allows fine-grained entry management—making certain that solely the precise customers or machines have entry to shared knowledge.

5 Advantages of Safe OpenID Join Token Federation

Delta Sharing is the trade’s first open-source strategy to knowledge sharing throughout knowledge, analytics and AI. This implies you’re not locked into any particular vendor or platform. Delta Sharing from Databricks permits for 2 forms of sharing: Sharing with Non-Databricks Recipients and Sharing with Databricks Recipients (also referred to as D2D Sharing). Examine out the weblog “How Delta Sharing Permits Safe Finish-to-Finish Collaboration” for extra particulars on each some of these sharing. If you happen to’re sharing knowledge with one other Databricks buyer into their Databricks account, you’ll be able to already use D2D Sharing, which offers a seamless and built-in expertise inside the Databricks ecosystem.

Alternatively, when securely sharing with exterior customers who are usually not on Databricks, Delta Sharing has at all times offered a easy and quick strategy to share knowledge utilizing bearer tokens. Nonetheless, for non-Databricks customers who prioritize enhanced safety, OIDC Token Federation for Enhanced Safety gives a extra sturdy and versatile authentication mechanism. This strategy minimizes publicity dangers and ensures safe collaboration.

Key Advantages of utilizing OIDC Token Federation when sharing with non-Databricks customers on any computing platform:

1. IdP-Managed Identities: Clients can use their present Identification Suppliers (like Entra ID or Okta) for authentication, avoiding the necessity to arrange new methods or processes.
2. Nice-Grained Person Entry Management: Clients acquire exact management over who can entry knowledge, making certain solely the precise folks or methods have permissions.
3. Multi-Issue Authentication (MFA) Help: If their Identification Supplier helps Multi-Issue Authentication (MFA), it provides an additional layer of safety, making certain solely approved customers entry shared knowledge.
4. Decreased Safety Dangers: Brief-lived tokens routinely expire, minimizing the prospect of unauthorized entry with out requiring guide intervention.
5. No Shared Secrets and techniques: Eliminates the necessity to distribute static credentials between Databricks, suppliers, and recipients.

How does the OIDC Token Federation work when sharing with non-Databricks recipients?

With OIDC Token Federation when sharing with non-Databricks recipients, every Delta Sharing recipient is configured with federation insurance policies, making certain that solely approved customers or machines can entry shared knowledge. Here is the way it works:

1. Organising Entry for Non-Databricks Recipient

• The info supplier (Databricks person) configures an OIDC Token Federation coverage for the recipient, specifying their exterior IdP (e.g., Entra ID, Okta).
• The coverage defines which customers, functions, or methods from the recipient’s identification system are allowed to entry the shared knowledge.

2. Safe Authentication with Brief-Lived Tokens

1. When the recipient makes an attempt to entry shared knowledge, Delta Sharing Connector will authenticate on their behalf in opposition to their configure IdP (e.g., Entra ID or Okta).
2. Upon a profitable authentication, identification system creates a brief digital cross, referred to as a JSON Net Token (JWT), which incorporates details about who they’re. That is shared with Delta Sharing Connector.
3. The Delta Sharing Connector will mix the JWT token issued by the IdP alongside the Delta Sharing request and ship it to Databricks Delta Sharing Server.
4. Databricks Delta Sharing will validates the JWT in opposition to the recipient’s coverage, and matches the foundations set by the information supplier, comparable to verifying who issued it, who it’s for, and who’s requesting entry.
5. Upon a profitable authentication, Databricks Delta Sharing server shares the requested knowledge.
6. Delta Sharing Consumer in flip returns it to recipient workflow.

This strategy eliminates the necessity for shared secrets and techniques. As an alternative, it makes use of short-term authentication tokens that expire rapidly and it permits exact management over who or what (a selected person or machine) can entry the information.

Three Authentication Situations Supported

OIDC Token Federation strategy helps each Person-to-Machine (U2M) and Machine-to-Machine (M2M) authentication flows, enabling a broad vary of use instances.

1. Person-to-Machine (U2M) Authentication

• A human person from the recipient group authenticates by way of their IdP.
• If Multi-Issue Authentication (MFA) is enabled within the recipient’s or supplier’s IdP, will probably be enforced.
• The person can then use instruments like Energy BI or Tableau to entry and analyze the shared knowledge simply.
• The info supplier can set guidelines to permit entry solely to particular folks or teams, making certain tight management over who will get entry.

This demo exhibits learn how to securely share knowledge from Databricks to Energy BI with EntraID authentication

2. Machine-to-Machine (M2M) Authentication

Delta Sharing now helps two safe methods for non-Databricks recipient machines to authenticate routinely:

Situation 1: OAuth Consumer Credentials Grant Circulation

• The recipient or supplier group registers a Service Principal of their IdP. A service principal is sort of a “person identification” for functions or automated methods, permitting them to securely entry assets while not having a human to log in.
• No credentials are shared externally between Databricks, Supplier or Recipient, and secret administration is native—the whole lot stays safe inside every group.
• Help for Python Delta Sharing Consumer and Spark Delta Sharing Consumer ensures that recipients can entry shared knowledge via scripts/automation.

This demo exhibits learn how to securely share knowledge from Databricks to Python Delta Sharing Consumer utilizing OAuth Consumer Credentials Grant

Situation 2: Managed Identification Authentication (Coming Quickly)

• For workloads working in cloud environments (e.g., Azure VMs), authentication happens routinely utilizing managed identities.
• No secrets and techniques or guide credential administration is required.
• Preliminary help will deal with Azure Compute, with potential growth to different cloud suppliers.

This demo exhibits learn how to securely share knowledge from Databricks to Python Delta Sharing Consumer utilizing Cloud supplier Managed Identification

Selecting the Identification Supplier:

Clients can select to authenticate utilizing an Exterior Identification Supplier (recipient-managed) or an Inside Identification Supplier (provider-managed).

• Exterior Identification Supplier (Recipient-Managed): The recipient’s identification system (like Entra ID or Okta) is used. The supplier units it up within the sharing coverage, so the recipient controls who from their group can entry the information.
• Inside Identification Supplier (Supplier-Managed): The supplier’s identification system is used. The supplier manages authentication by including exterior recipients as company in their very own identification system. This enables the supplier’s system to deal with entry on behalf of the recipient.

What’s Subsequent?

We’ll make it simpler to arrange safe knowledge sharing with pre-built templates for widespread OIDC Federation Insurance policies tailor-made for widespread identification suppliers like Entra ID and Okta. Moreover, upcoming help for managed identification authentication will allow cloud-based workloads (e.g., Azure VMs) to authenticate while not having passwords or secrets and techniques, making certain a seamless and safe connection to Databricks Delta Sharing endpoints.

Get Began

OIDC Token Federation for Enhanced Safety when sharing with non-Databricks recipients is on the market in Public Preview immediately to AWS, GCP and Azure clients. Find out how Delta Sharing makes it straightforward for organizations to securely share knowledge with non-Databricks customers on any computing platform—with out compromising on safety.