Press "Enter" to skip to content

Category: Microsoft Fabric

Checking Table Sizes in Microsoft Fabric Warehouses

Published 2026-08-04 by Kevin Feasel

Nikola Ilic wants to know how big that warehouse is:

If you’ve tried to check how much storage your Fabric warehouse tables consume, you’ve probably discovered that the usual SQL Server approaches don’t work. This post explains why, what else doesn’t work, and the one approach that does (and that I can use as a reference going forward, instead of wasting time and tokens asking LLMs).

Click through for the problem description and the answer. I completely agree with Nikola’s conclusion that it’s harder than it should be.

Comments closed

The Medallion Architecture’s Silver Layer: Physical or Virtual?

Published 2026-08-03 by Kevin Feasel

Nikola Ilic noodles a concept:

If you’ve been working with data platforms in recent years, chances are that you’ve implemented the medallion design pattern (or architecture:)) at least once. Bronze, silver, gold – raw, cleansed, curated – this design pattern has become so widely adopted that you’ll find it in every reference architecture, every certification exam, and every conference talk (including some of mine, I have to admit). And, in the vast majority of implementations I’ve seen (and built myself), all three layers are physically materialized as tables.

In this article, I’d like to challenge that habit. Not the medallion pattern itself – the logical separation of layers is, in my opinion, still of paramount importance. What I want to question is something much more specific: why is the silver layer a set of physical tables? And, with the recent announcement of GPU acceleration for the Fabric Data Warehouse, I believe this question deserves a serious answer, rather than “because that’s how the diagram looks”

Click through for Nikola’s thoughts, including when it might work for the silver layer to be virtual and when it doesn’t make sense.

Comments closed

Source Control and Branching in Microsoft Fabric

Published 2026-08-03 by Kevin Feasel

Jon Lunn has a series with the first two parts up. First up is part one:

Developing in Microsoft Fabric within a multi-team environment has not always been straightforward. Fabric was initially geared more towards individual users and citizen developers working independently within a workspace, which created challenges when applying traditional source control and DevOps practices.

Part two covers integration with Azure DevOps:

Now we’re going to look at deploying Fabric items through DevOps using the Microsoft-supported Fabric CI/CD Python library.

  • Set up DevOps components
    • Environments
    • Libraries
  • Creating an DevOps YAML deployment pipeline
  • Creating a Fabric CI/CD script to deploy the items
  • Using AI Tools to help speed up development of DevOps

Stay tuned for part three.

Comments closed

Microsoft Fabric Purview Protection Policies

Published 2026-07-30 by Kevin Feasel

Gilbert Quevauvilliers has a video:

How to use Microsoft Purview Protection Policies so a Fabric workspace Admin can still fully administer the workspace… but is completely blocked from seeing any of the sensitive data inside it.

You keep the person as Admin (so they can manage items, permissions, capacity, etc.) while the protection policy + sensitivity label combination simply removes their ability to open or view the protected Lakehouses, Notebooks, etc.

Click through for the video.

Comments closed

Incremental Liquid Clustering in Apache Spark

Published 2026-07-28 by Kevin Feasel

Miles Cole lays out a recommendation:

Liquid Clustering was already a better abstraction than static partitioning due to its flexible nature. Fabric Spark Runtime 2.0 fixes the part that had me actively cautioning customers to reconsider blindly adopting it: the cost of maintaining the layout.

In Runtime 1.3 (Delta 3.2) a small append followed by OPTIMIZE would rewrite every file in a partial Z-Cube. In Runtime 2.0, the incremental strategy only touches files that are unclustered, small, or carrying a high density of deletion vectors. That changes Liquid Clustering from an occasional, potentially expensive maintenance operation into something that works beautifully with workloads of any shape and with adjacent layout optimizations. Batch or streaming writes. Auto Compaction and/or Fast Optimize. With the new incremental strategy, Liquid Clustering is highly compatible and highly efficient, and should now take its rightful place as the de facto new data layout strategy.

Read on to learn more about what Liquid Clustering is, how it works, and why you should think about it if you’re using Fabric Spark Runtime 2.0.

Comments closed

Inbound Network Protection Feature Gaps in Microsoft Fabric

Published 2026-07-27 by Kevin Feasel

Meagan Longoria tells a cautionary tale:

Fabric’s inbound network protection gives you two tenant-level controls: Private Link, which routes traffic through your virtual network instead of the public internet, and Block Public Internet Access, which closes the public internet off entirely once Private Link is in place. Together they look like the obvious move if you’re trying to get a data platform off the public internet. Flip two settings, close off the internet, ship a more secure platform. That’s the pitch.

But read on to learn the downside cost to this: you’re giving up quite a bit of capability.

Comments closed

Thoughts on Fabric User Data Functions

Published 2026-07-22 by Kevin Feasel

Jon Lunn shares some thoughts:

Fabric User Data Functions are a bit of a strange hybrid. They can be used like Azure Functions, to call some code outside and act like a serverless compute function, but also can extend notebooks as a source of reusable data logic. I’ve been using them since preview, and now they support Azure Key Vault (AKV) and Fabric Variable Libraries, I think they are ready (mostly) for productionable use.

I think the part that annoys me the most about them is that they’re called “User Data Functions” instead of “User-Defined Functions” and yet the acronym is the same.

Comments closed

Workspace Monitoring in Power BI

Published 2026-07-21 by Kevin Feasel

Chris Webb monitors Power BI:

This week, in the announcement about support for Fabric Pipelines in Workspace Monitoring, I noticed that it came with an Operations Agent that actively monitors and analyses Pipeline activity. And that got me thinking, since Workspace Monitoring also contains Power BI activity data, why not create an Operations Agent to actively monitor Power BI too?

Read on for a fairly simple demonstration of how it works.

Comments closed

Performance Tuning with Microsoft Fabric Warehouse Query Insights

Published 2026-07-21 by Kevin Feasel

Koen Verbeeck troubleshoots query performance:

We are in the process of building a data warehouse in Microsoft Fabric. Since we have good knowledge of T-SQL, we are using Fabric Warehouse as the database engine. We are hitting some performance issues in our ETL and we would like to investigate further, but traditional SQL Server DMVs don’t give the results we are hoping for. How can we troubleshoot performance in the Fabric Warehouse?

Click through to see how.

Comments closed

Replacing Item IDs and Connection Strings with Variable Libraries

Published 2026-07-16 by Kevin Feasel

Gilbert Quevauvilliers makes use of variable libraries in Microsoft Fabric:

In this blog post I’m going to show you how you can use variable libraries with connection strings as well as items.

This approach allows you to manage environment-specific configuration when deploying Fabric items across development, test, and production workspaces

If you’re looking to move your fabric capacities across workspaces or regions, you can do this quickly and easily without having to re-Plumb a lot of the work.

Click through for a working example.

Comments closed