Press "Enter" to skip to content

Category: Microsoft Fabric

Variable Libraries in Microsoft Fabric

Nikola Ilic digs into a feature:

I hear you, I hear you: Nikola, that’s what deployment rules in Fabric Deployment Pipelines are for, isn’t it? Well, partly. But there’s a Fabric item built specifically to put an end to this whole genre of pain, and it’s the variable library. This article is the long version: what it is, how it’s wired together under the hood, who can actually consume it, when you should reach for it, when you absolutely shouldn’t, how it compares to the other parameterization features in Fabric, and a real telco demo to make it all concrete.

Click through for a deep dive into how it works.

Comments closed

Moving Fabric Notebooks between Workspaces

Gilbert Quevauvilliers takes advantage of source control:

With the new Lakehouse Auto-Binding capability in Notebook Git integration, Fabric can now intelligently preserve and resolve the binding between your notebooks and their attached Lakehouses as you move them across workspaces. This makes true multi-environment development and CI/CD workflows in Fabric significantly smoother and more reliable.

I am going to show you how to do this in the blog post below.

That is pretty nice, and Gilbert has a demo of the process, showing that it’s not particularly onerous

Comments closed

The Benfit of Disabling V-Order in Fabric Dataflows Gen2

Chris Webb covers a specific use case:

Quite a few new Dataflows Gen2 features were released recently without much fanfare, but that doesn’t mean they aren’t important. I will take a look at them all in my next few posts; in this first post I’ll look at the ability to disable V-Order on staged data.

As the (very detailed) documentation for this new feature describes, V-Order is a write-time optimisation for the parquet files that underpin the Delta tables that OneLake uses to store data. It slows down writing data to the tables but means that reading data from them, for example in Power BI Direct Lake mode, is much faster. 

Click through to see how disabling V-Order can make certain staging loads faster.

Comments closed

Identity Columns in Fabric Runtime 2.0

Jon Lunn is happy:

Fabric Runtime 2.0 is in public preview, and there is one small change for this that makes my life as a data engineer a little bit easier. Identity columns. I miss the good old T-SQL Identity column “Id INT IDENTITY(1,1)”, now we have something like it in Spark/Delta.

So with the the old spark/delta table runtime, you couldn’t have an default, automatically increasing column. But now we have these updates.

Click through to see how it works.

Comments closed

The Limits of Parallelism with Fabric Dataflows Gen2

Chris Webb notes that parallelism does not mean “free performance improvements”:

To finish off my series of posts on concurrent evaluation in Fabric Dataflows Gen2 (see part 1 and part 2) I decided to do some more realistic tests to see how much parallelism I could get. To do this I uploaded 244 identical Excel files containing almost 542000 rows of data each to a SharePoint document library. Excel files are probably the worst-performing file format for dataflows (see here for some tests that show this), while SharePoint is probably the worst-performing place to store data for a dataflow and also has a reputation for throttling applications that make too many requests.

Click through for Chris’s test results.

Comments closed

Accessing Data between Private Link and Public Fabric Workspaces

Gilbert Quevauvilliers uses the private endpoint:

In this blog post I show how it is possible to access data between a Private Link Workspace, where I want to read the data from my Public Workspace.

An example of this is where I wanted to use a DirectLake Semantic Model sitting in my Public Workspace where the data is from data in my Private Link Workspace.

Click through to see how it works.

Comments closed

Digging into Fabric Apps

Kurt Buhler explains a new capability:

For years, the Power BI community has been clamoring to have native support for visuals-as-code; the ability to create visuals, pages, or even entire dashboards with libraries like Vega, and D3.js. This is now possible in Microsoft Fabric with Fabric Apps, specifically, using a data app.

Fabric App is a new item type that lets you create and distribute any interactive experiences in Fabric, rather than pre-defined reports, dashboards, and data agents. These are web applications (or webapps). A webapp is any program that runs in a browser (like YouTube, Facebook, or Microsoft Word’s online editor) instead of one that you install on your computer (like Power BI Desktop or Microsoft Word).

Read on to see how you can build Fabric Apps, as well as what they are and aren’t.

Comments closed

GraphQL Deployment Error due to dm_exec_describe_first_result_set()

Koen Verbeeck troubleshoots an issue:

A while ago we suddenly had an error while trying to deploy one Fabric workspace to another using fabric-cicd. The issue was with a GraphQL object and the following error was returned:

Failed to publish GraphQLApi ‘my_graphql’: Operation failed. Error Code: DatasourceInvalidStoredProcedure. Error Message: Only those stored procedures whose metadata for the first result set described by sys.dm_exec_describe_first_result_set are supported.

Read on for Koen’s diagnosis and resolution for this issue.

Comments closed

Data Agent Prompt Handling and Semantic Models

Marc Lelijveld digs into Microsoft Fabric Data Agents:

What makes Fabric Data Agents particularly interesting is the wide range of supported data sources. Today, Data Agents can connect to nearly everything that lives inside Microsoft Fabric, or data that is linked into Fabric through shortcuts. Whether your data sits in a Lakehouse, Warehouse, KQL Database, Power BI semantic model, or even external storage connected through OneLake shortcuts.

However, the way Data Agents handle sources can differ significantly from one source type to another. Semantic Models in particular behave quite differently compared to other Fabric data sources. In this blog, I’ll dive deeper into how prompt handling works for Semantic Models, what happens behind the scenes, and the common gotchas you’re likely to encounter along the way.

Read on to see how semantic model behavior differs in particular from SQL or DAX queries.

Comments closed