Azure Databricks News - May 2026 | Marczak.IO

Highlight

Welcome to another news post, this time there is a lot of ground to cover. Grab and coffee and let’s go.

This roundup focuses on what changes day-to-day work for administrators, architects, data engineers, and users so you can prioritize what to adopt first.

Big Changes

These are the updates that can materially change platform strategy

Persona	Impact Score	News	Why it’s important	News Link
Architects	⭐⭐⭐⭐⭐	Unity Catalog Iceberg capabilities are now GA	Managed Iceberg tables, foreign Iceberg tables, and Iceberg v3 features moving to GA is a big interoperability milestone and affects long-term architecture choices across engines	Unity Catalog Iceberg GA
Administrators	⭐⭐⭐⭐⭐	Automatic identity management with Microsoft Entra ID is now GA	This removes a lot of SCIM operational friction and improves identity consistency, including groups and service principals, which matters in large enterprises	Automatic identity management GA
Data Engineers	⭐⭐⭐⭐	Lakeflow Connect query-based connectors are now GA	You can ingest directly with cursor-based queries without full CDC and gateways, which simplifies many source onboarding patterns	Query-based connectors GA
Data Engineers	⭐⭐⭐⭐	Real-time mode in Lakeflow Spark Declarative Pipelines is now Public Preview	With low-latency update semantics and update_flow, this can shift how teams design streaming and decision pipelines	Real-time mode for Lakeflow Spark Declarative Pipelines
Developers	⭐⭐⭐⭐	Databricks CLI is now GA	A GA CLI with secure OS-native token storage is a major quality jump for automation and DevEx in enterprise CI/CD	Databricks CLI GA
Developers	⭐⭐⭐⭐	Databricks Apps horizontal scaling is in Beta	Better app resilience and zero-downtime deployment options are meaningful for teams exposing internal or customer-facing apps	Databricks Apps horizontal scaling
Architects	⭐⭐⭐⭐	Vector Search storage-optimized endpoints are now GA	Higher scale capacity and faster indexing can significantly reduce operational pain for large vector workloads	Vector Search storage-optimized endpoints GA
Data Engineers	⭐⭐⭐⭐	Catalog commits are now GA	Multi-table and multi-statement consistency capabilities are a big deal for robust data product design and cross-engine coordination	Catalog commits GA
Data Engineers	⭐⭐⭐⭐	Lakeflow Spark Declarative Pipelines sinks are now GA	Direct sinks for Delta, Kafka, Event Hubs, and Python data sources simplify production pipeline outputs	Lakeflow Spark Declarative Pipelines sinks GA

Medium Changes

Important updates, but usually scoped to specific workloads or teams

Persona	Impact Score	News	Why it’s good to know	News Link
Administrators	⭐⭐⭐	Instance events and instance pools system tables are in Public Preview	Better observability over compute lifecycle and pool history supports stronger governance and troubleshooting workflows	Compute system tables preview
Users	⭐⭐⭐	Databricks SQL alerts are now GA	This strengthens proactive KPI monitoring and can reduce the lag between data drift and business action	Databricks SQL alerts GA
Data Engineers	⭐⭐⭐	Databricks Runtime 18.2 is now GA	Important for upgrade planning, but rollout pace depends on your compatibility and testing windows	Databricks Runtime 18.2 GA
Developers	⭐⭐⭐	Faster package installs with %uv pip in serverless notebooks	Better notebook iteration speed, but only directly impacts teams heavily using notebook package installation loops	Faster installs with %uv pip
Developers	⭐⭐⭐	Edits to resources in the UI now automatically update YAML	Nice quality-of-life improvement for bundle workflows, though it is incremental rather than transformational	UI edits auto-update YAML

Small but Still Relevant

These are meaningful, but they will not change every team’s daily workflow immediately

Persona	Impact Score	News	Why it’s not top priority for most teams	News Link
Users	⭐⭐	Workspace sidebar navigation updates	Useful UX cleanup, but mostly a navigation and terminology change instead of a platform capability change	Workspace sidebar navigation updates
Administrators	⭐⭐	Additional recipient for workspace operational emails	Helpful for operational communication hygiene, especially for shared operations teams and distro lists	Additional workspace email recipient

Practical Takeaway

What does this mean for us?

If you run a mature Databricks platform, start with three tracks:

Governance track: adopt GA automatic identity management and evaluate ABAC plus Iceberg interoperability
Delivery track: standardize on GA CLI and align release automation around updated Runtime 18 patterns
Data product track: prioritize Lakeflow connector and pipeline upgrades where they remove custom ingestion or orchestration glue

Trust me, this month has enough high-impact changes that a structured adoption plan is worth it instead of ad hoc upgrades.

Happy building