Press "Enter" to skip to content

Category: Microsoft Fabric

Custom Libraries in Microsoft Fabric Data Engineering

Published 2025-06-13 by Kevin Feasel

Gerhard Brueckl isn’t content with the defaults:

When working with Spark or data engineering in general in Microsoft Fabric, you will sooner or later come to the point where you need to reuse some of the code that you have already written in another notebook. Best practice is to put these code pieces into a central place from where it can be referenced and reused. This way you can make sure all notebooks always use the very same code and it is also easy to develop, update and test the common functions.

As Gerhard mentions, having common notebooks with utilities is fine for when you’re getting started with development, but being able to centralize functions in proper libraries can make that code a lot more useful, not just in the context of the single notebook.

I believe that this does allow for arbitrary code execution, so someone with sufficient permissions to create a notebook and import code from arbitrary locations would be able to execute that code. I think there are ways of limiting this risk (such as not allowing your Fabric hosts to connect to any remote servers other than ones you explicitly allow), but it’s something I’d have to puzzle through.

Comments closed

Fronting Fabric APIs with Azure API Management

Published 2025-06-13 by Kevin Feasel

Ed Lima combines expensive with expensive:

Integrating Azure API Management (APIM) with Microsoft Fabric’s API for GraphQL can significantly enhance your API’s capabilities by providing robust scalability and security features such as identity management, rate limiting, and caching. This post will guide you through the process of setting up and configuring these features.

API Management is a really neat service, though it’s rather costly. That’s my biggest complaint about it, though it is a doozy.

Comments closed

Refreshing SQL Analytics Endpoint Metadata in Fabric

Published 2025-06-12 by Kevin Feasel

Ancy Philip makes an announcement:

We’re excited to announce that the long-awaited refresh SQL analytics endpoint metadata REST API is now available in preview. You can now programmatically trigger a refresh of your SQL analytics endpoint to keep tables in sync with any changes made in the parent artifact, ensuring that you can keep your data up to date as needed.

Click through to see how it works.

Comments closed

Microsoft Fabric Mirroring and Live Monitoring

Published 2025-06-11 by Kevin Feasel

Teo Lachev is waiting for a message:

A current project called for mirroring a Google BigQuery dataset to Fabric. This feature is currently in private preview so don’t try to find it. However, the tips I share here should be applicable to other available mirroring scenarios, such as mirroring from Azure SQL Database.

One of the GBQ tables was a transaction fact table with some 130 million rows. The issue was that the mirroring window would show this table as normally replicating table with Running green status, but we waited and waited and nothing was happening…

Read on to learn more and how Teo was able to get a better idea of how the initial sync progressed.

Comments closed

Debugging Fabric UDFs in Visual Studio Code

Published 2025-06-10 by Kevin Feasel

Sunitha Muthukrishna takes us through a debugging exercise:

Debugging your code is important to identify issues and mitigate them when you’re working with user data functions in Microsoft Fabric. You want to make sure everything works as it should and that’s where local debugging lets you catch problems in your code without messing with the live environment. In this blog post, I will walk you through the steps to make local debugging easier and faster.

Click through to see what you’ll need, as well as the process to debug a function locally.

Comments closed

Listing Items in a Fabric Workspace using Powershell

Published 2025-06-06 by Kevin Feasel

Rob Sewell continues a series on working with Microsoft Fabric assets via Powershell:

Having created a workspace, a lakehouse, a warehouse, a Fabric SQL database previously, it’s time to move onto something a little more interesting. I was getting bored writing the same post over and over again, so I thought I would show you how to list the items in a Fabric Workspace using PowerShell. This command was super useful for me today to answer the question “What do they have in this workspace?” and I hope it will be useful for you too.

Click through to see how you can accomplish this.

Comments closed

Purview Data Loss Prevention in Microsoft Fabric

Published 2025-06-06 by Kevin Feasel

Yael Biss doesn’t want people walking off with the data:

As data volume and complexity soar, protecting sensitive information has become non-negotiable. With the latest enhancements to Purview Data Loss Prevention (DLP) Policies in Microsoft Fabric, organizations now have the power to proactively secure their data in Onelake.

Whether you’re just getting started or looking to take your data governance to the next level, following proven best practices will maximize your security, compliance, and productivity.

Click through for several tips on how to use Microsoft Purview DLP in Fabric. One of those tips ought to be “Get a side hustle so you can afford both Purview and Fabric.”

Comments closed

Securing Mirrored Databricks Data in Fabric

Published 2025-06-05 by Kevin Feasel

Aaron Merrill has a catalog:

With this update, Azure Mirrored Databricks Catalog items can now be enabled with OneLake security. Security at the table, column, or row level can be defined directly in each item, allowing access to be controlled at a granular level. This allows security to be defined directly over the data mirrored into OneLake so it can be securely used by downstream sources such as lakehouses, notebooks, or semantic models.  

Read on to see what to do in order to make use of this. It is, of course, currently in preview.

Comments closed

Creating Fabric Databases via Powershell

Published 2025-06-05 by Kevin Feasel

Rob Sewell has a trio of posts. First up, Rob creates a warehouse:

We are going to use [FabricTools, a PowerShell module that simplifies the management of Microsoft Fabric resources. You can find it on GitHub at[FabricTools provides functions to create, update, and delete Fabric warehouses and other resources. It has been developed and is maintained by members of the community.

Next up is a lakehouse:

Having created a Microsoft Fabric workspace using PowerShell, assigned users to it using PowerShell, and created a Microsoft Fabric Warehouse using PowerShell, we now turn our attention to creating a Microsoft Fabric Lakehouse using PowerShell. These posts all look remarkably similar 🙂 and thats because the functionality they are showing is exactly same..

Just like dbatools and dbachecks creating functions that follow the same pattern makes it easier to learn. This is why FabricTools has chosen to use the same pattern for creating Fabric resources using PowerShell.

And from there is a Fabric SQL Database:

A Fabric SQL Database is a SQL Database that is hosted in Microsoft Fabric. It provides a fully managed SQL database service that allows you to store and query your data using SQL using the same SLQ Engine as Azure SQL Database.

Click through for examples of all three.

Comments closed

Populating Microsoft Fabric Data Agents with Semantic Model Synonyms

Published 2025-06-04 by Kevin Feasel

Marc Lelijveld explains some terms:

It was only yesterday, that I wrote a blog post on Semantic Models as a source for Fabric Data Agents. Not much time has passed, since I learned that Fabric Data Agents does not (always) respect the Synonyms that have been added to a Semantic Model. As a result, the Data Agent may start creating implicit measures, not respecting the definitions and logic in the explicit measures that are part of the Semantic Model.

Long story short, I think we should be able to do better! Therefore, I created a Notebook that helps you to setup Data Agents, collect additional information from your Semantic Model and populate that information automatically as AI notes to the Data Agent.

Read on for the notebook and some additional explanation.

Comments closed