Press "Enter" to skip to content

Category: Microsoft Fabric

Deploying Fabric Resources via ADO

Published 2026-08-11 by Kevin Feasel

Jon Lunn continues a series on source control and branching with Microsoft Fabric:

There are a few things you need to check first:

  1. Does the account your going to use to authenticate to (In this case a Service Principal) been added to the workspace with contributor level access?
  2. Has the DevOps Library been updated with the values for the Service Principal and workspace id you are deploying to?
  3. Has any DevOps Environment that is production been gated to to allow deployments only after approval?
  4. Got the list of items to deploy?

You have done all those, then alright! Lets push that button! Flick that switch! Saddle that horse!

Be right back; acquiring horse.

Leave a Comment

Tips on When to Use Microsoft Fabric Shortcuts

Published 2026-08-10 by Kevin Feasel

James Serra offers up some guidance:

Imagine separate Sales, Finance, Shared Data, and Executive Analytics workspaces. Sales owns sales transactions, Finance owns budgets, and Shared Data owns common tables such as Customer, Product, and Date. Executive Analytics needs selected data from all three, but it does not want to copy everything into another lakehouse and maintain another set of pipelines. Instead, it creates shortcuts to the authoritative tables and presents them together in its own lakehouse.

This is where the beauty of Fabric OneLake shortcuts becomes obvious. To a report developer or analyst, the Executive Analytics lakehouse can look like one complete collection of tables. That person might not even know which tables are physically stored there and which are shortcuts—and usually should not need to know. Fabric resolves those paths behind the scenes, which is one reason I call Fabric “the great data unifier”: it can present one logical data estate without forcing all the data into one physical location.

Click through to learn more about what shortcuts are, how they work, and when you should (or should not) use them.

Leave a Comment

Handling Backpressure in Fabric Real-Time Intelligence

Published 2026-08-10 by Kevin Feasel

Greg Low talks about backpressure:

In any real-time data system, there’s a point where the incoming event rate can exceed what the system can process. This condition is known as backpressure.

Backpressure can occur for a few reasons — a sudden spike in data volume, slow or overloaded consumers, or limited throughput in one part of the pipeline. If it’s not handled properly, it can cascade through the system, eventually causing delays or even a complete stall in event processing.

There are several strategies to handle backpressure effectively.

Click through for those mechanisms. The pedant in me who hates how “real-time” has replaced “online” in terms of systems terminology—not Greg’s fault in the least—would point out that if you truly have a real-time system, you can’t afford to have backpressure because any sort of delay would be inimical to it being real-time.

Leave a Comment

Datatypes and Constraints in Microsoft Fabric Data Warehouses

Published 2026-08-07 by Kevin Feasel

Louis Davidson starts digging in:

There are three major “engines” in Fabric that I have seen that use T-SQL. There is the SQL Azure Fabric engine, the Lakehouse, and the Data Warehouse. Having this capability to manipulate data is wonderful, but there are some things you need to understand before you start writing code (unless you want to learn them the hard way like I have. I also will not profess to have found all of these differences. Changing my mindset when using these engines was the subject of this editorial

Read on for some inconsistency.

Leave a Comment

Checking Table Sizes in Microsoft Fabric Warehouses

Published 2026-08-04 by Kevin Feasel

Nikola Ilic wants to know how big that warehouse is:

If you’ve tried to check how much storage your Fabric warehouse tables consume, you’ve probably discovered that the usual SQL Server approaches don’t work. This post explains why, what else doesn’t work, and the one approach that does (and that I can use as a reference going forward, instead of wasting time and tokens asking LLMs).

Click through for the problem description and the answer. I completely agree with Nikola’s conclusion that it’s harder than it should be.

Leave a Comment

The Medallion Architecture’s Silver Layer: Physical or Virtual?

Published 2026-08-03 by Kevin Feasel

Nikola Ilic noodles a concept:

If you’ve been working with data platforms in recent years, chances are that you’ve implemented the medallion design pattern (or architecture:)) at least once. Bronze, silver, gold – raw, cleansed, curated – this design pattern has become so widely adopted that you’ll find it in every reference architecture, every certification exam, and every conference talk (including some of mine, I have to admit). And, in the vast majority of implementations I’ve seen (and built myself), all three layers are physically materialized as tables.

In this article, I’d like to challenge that habit. Not the medallion pattern itself – the logical separation of layers is, in my opinion, still of paramount importance. What I want to question is something much more specific: why is the silver layer a set of physical tables? And, with the recent announcement of GPU acceleration for the Fabric Data Warehouse, I believe this question deserves a serious answer, rather than “because that’s how the diagram looks”

Click through for Nikola’s thoughts, including when it might work for the silver layer to be virtual and when it doesn’t make sense.

Leave a Comment

Source Control and Branching in Microsoft Fabric

Published 2026-08-03 by Kevin Feasel

Jon Lunn has a series with the first two parts up. First up is part one:

Developing in Microsoft Fabric within a multi-team environment has not always been straightforward. Fabric was initially geared more towards individual users and citizen developers working independently within a workspace, which created challenges when applying traditional source control and DevOps practices.

Part two covers integration with Azure DevOps:

Now we’re going to look at deploying Fabric items through DevOps using the Microsoft-supported Fabric CI/CD Python library.

  • Set up DevOps components
    • Environments
    • Libraries
  • Creating an DevOps YAML deployment pipeline
  • Creating a Fabric CI/CD script to deploy the items
  • Using AI Tools to help speed up development of DevOps

Stay tuned for part three.

Leave a Comment

Microsoft Fabric Purview Protection Policies

Published 2026-07-30 by Kevin Feasel

Gilbert Quevauvilliers has a video:

How to use Microsoft Purview Protection Policies so a Fabric workspace Admin can still fully administer the workspace… but is completely blocked from seeing any of the sensitive data inside it.

You keep the person as Admin (so they can manage items, permissions, capacity, etc.) while the protection policy + sensitivity label combination simply removes their ability to open or view the protected Lakehouses, Notebooks, etc.

Click through for the video.

Leave a Comment

Incremental Liquid Clustering in Apache Spark

Published 2026-07-28 by Kevin Feasel

Miles Cole lays out a recommendation:

Liquid Clustering was already a better abstraction than static partitioning due to its flexible nature. Fabric Spark Runtime 2.0 fixes the part that had me actively cautioning customers to reconsider blindly adopting it: the cost of maintaining the layout.

In Runtime 1.3 (Delta 3.2) a small append followed by OPTIMIZE would rewrite every file in a partial Z-Cube. In Runtime 2.0, the incremental strategy only touches files that are unclustered, small, or carrying a high density of deletion vectors. That changes Liquid Clustering from an occasional, potentially expensive maintenance operation into something that works beautifully with workloads of any shape and with adjacent layout optimizations. Batch or streaming writes. Auto Compaction and/or Fast Optimize. With the new incremental strategy, Liquid Clustering is highly compatible and highly efficient, and should now take its rightful place as the de facto new data layout strategy.

Read on to learn more about what Liquid Clustering is, how it works, and why you should think about it if you’re using Fabric Spark Runtime 2.0.

Leave a Comment

Inbound Network Protection Feature Gaps in Microsoft Fabric

Published 2026-07-27 by Kevin Feasel

Meagan Longoria tells a cautionary tale:

Fabric’s inbound network protection gives you two tenant-level controls: Private Link, which routes traffic through your virtual network instead of the public internet, and Block Public Internet Access, which closes the public internet off entirely once Private Link is in place. Together they look like the obvious move if you’re trying to get a data platform off the public internet. Flip two settings, close off the internet, ship a more secure platform. That’s the pitch.

But read on to learn the downside cost to this: you’re giving up quite a bit of capability.

Leave a Comment