Press "Enter" to skip to content

Category: Microsoft Fabric

Microsoft Fabric Purview Protection Policies

Published 2026-07-30 by Kevin Feasel

Gilbert Quevauvilliers has a video:

How to use Microsoft Purview Protection Policies so a Fabric workspace Admin can still fully administer the workspace… but is completely blocked from seeing any of the sensitive data inside it.

You keep the person as Admin (so they can manage items, permissions, capacity, etc.) while the protection policy + sensitivity label combination simply removes their ability to open or view the protected Lakehouses, Notebooks, etc.

Click through for the video.

Comments closed

Incremental Liquid Clustering in Apache Spark

Published 2026-07-28 by Kevin Feasel

Miles Cole lays out a recommendation:

Liquid Clustering was already a better abstraction than static partitioning due to its flexible nature. Fabric Spark Runtime 2.0 fixes the part that had me actively cautioning customers to reconsider blindly adopting it: the cost of maintaining the layout.

In Runtime 1.3 (Delta 3.2) a small append followed by OPTIMIZE would rewrite every file in a partial Z-Cube. In Runtime 2.0, the incremental strategy only touches files that are unclustered, small, or carrying a high density of deletion vectors. That changes Liquid Clustering from an occasional, potentially expensive maintenance operation into something that works beautifully with workloads of any shape and with adjacent layout optimizations. Batch or streaming writes. Auto Compaction and/or Fast Optimize. With the new incremental strategy, Liquid Clustering is highly compatible and highly efficient, and should now take its rightful place as the de facto new data layout strategy.

Read on to learn more about what Liquid Clustering is, how it works, and why you should think about it if you’re using Fabric Spark Runtime 2.0.

Comments closed

Inbound Network Protection Feature Gaps in Microsoft Fabric

Published 2026-07-27 by Kevin Feasel

Meagan Longoria tells a cautionary tale:

Fabric’s inbound network protection gives you two tenant-level controls: Private Link, which routes traffic through your virtual network instead of the public internet, and Block Public Internet Access, which closes the public internet off entirely once Private Link is in place. Together they look like the obvious move if you’re trying to get a data platform off the public internet. Flip two settings, close off the internet, ship a more secure platform. That’s the pitch.

But read on to learn the downside cost to this: you’re giving up quite a bit of capability.

Comments closed

Thoughts on Fabric User Data Functions

Published 2026-07-22 by Kevin Feasel

Jon Lunn shares some thoughts:

Fabric User Data Functions are a bit of a strange hybrid. They can be used like Azure Functions, to call some code outside and act like a serverless compute function, but also can extend notebooks as a source of reusable data logic. I’ve been using them since preview, and now they support Azure Key Vault (AKV) and Fabric Variable Libraries, I think they are ready (mostly) for productionable use.

I think the part that annoys me the most about them is that they’re called “User Data Functions” instead of “User-Defined Functions” and yet the acronym is the same.

Comments closed

Workspace Monitoring in Power BI

Published 2026-07-21 by Kevin Feasel

Chris Webb monitors Power BI:

This week, in the announcement about support for Fabric Pipelines in Workspace Monitoring, I noticed that it came with an Operations Agent that actively monitors and analyses Pipeline activity. And that got me thinking, since Workspace Monitoring also contains Power BI activity data, why not create an Operations Agent to actively monitor Power BI too?

Read on for a fairly simple demonstration of how it works.

Comments closed

Performance Tuning with Microsoft Fabric Warehouse Query Insights

Published 2026-07-21 by Kevin Feasel

Koen Verbeeck troubleshoots query performance:

We are in the process of building a data warehouse in Microsoft Fabric. Since we have good knowledge of T-SQL, we are using Fabric Warehouse as the database engine. We are hitting some performance issues in our ETL and we would like to investigate further, but traditional SQL Server DMVs don’t give the results we are hoping for. How can we troubleshoot performance in the Fabric Warehouse?

Click through to see how.

Comments closed

Replacing Item IDs and Connection Strings with Variable Libraries

Published 2026-07-16 by Kevin Feasel

Gilbert Quevauvilliers makes use of variable libraries in Microsoft Fabric:

In this blog post I’m going to show you how you can use variable libraries with connection strings as well as items.

This approach allows you to manage environment-specific configuration when deploying Fabric items across development, test, and production workspaces

If you’re looking to move your fabric capacities across workspaces or regions, you can do this quickly and easily without having to re-Plumb a lot of the work.

Click through for a working example.

Comments closed

Direct Lake Mode Benefits in Power BI

Published 2026-07-13 by Kevin Feasel

Chris Webb lays out the pros:

This is a blog post I’ve been meaning to write for a long time. Since Fabric launched there has been a lot of focus on Direct Lake mode in Power BI and a lot of people used it because it was the cool new thing. Arguably, we at Microsoft have been guilty of telling people to use it because it was the cool new thing without properly explaining what the benefits are of using it. Direct Lake doesn’t completely replace other storage modes: in a recent post I talked about when Import/DirectQuery composite models are the best choice; Marco wrote a good article on Direct Lake vs Import mode which makes the case for the continuing relevance of Import mode for many scenarios. So what are the main benefits of using Direct Lake mode? 

Click through for Chris’s answer.

Comments closed

Performing a Lakehouse Table Health Check

Published 2026-07-13 by Kevin Feasel

Jon Lunn looks at a stored procedure:

Microsoft have rolled out a new ‘sys’ stored procedure called ‘sys.sp_get_table_health_metrics’ for checking the health of your tables in your Lakehouse. And it outputs a lot of metrics looking at the state of your table and anomaly detection in them.

What does it check for?

Click through for that answer, as well as some of the things you should watch for in the procedure results, and what you can do if it does catch something.

Comments closed

Merging Data into a Fabric Lakehouse via Python Notebook

Published 2026-07-10 by Kevin Feasel

GIlbert Quevauvilliers uses a pure Python notebook:

In this blog post I am going to show you how to use a Fabric Python runtime notebook (This is the notebook which only uses Pure Python functions and consumes significantly lower Capacity Units (CUs)).

The pattern is how to get new data and merge it into an existing Lakehouse table. This ensures that if the notebook is run again data will not be duplicated.

Why I am sharing this is I have found that there is not a lot of useful information about how to use a Python notebook to write to a lakehouse table easily. And then also how to use a Merge statement making it easier to insert or update your lakehouse tables. This simplifies the ingestion process, runs faster and consumes the least amount of CUs

Gilbert doesn’t mention it in the blog post but the notebook does use DuckDB to query the data using SQL.

Comments closed