Press "Enter" to skip to content

Category: Microsoft Fabric

Using a Microsoft Fabric Variable Library in a Dataflow

Laura Graham-Brown shows another way to use variable libraries:

One of the popular low-code tools within Microsoft Fabric is the Gen2 Dataflow. Power BI report builders already know some Power Query. So armed with this knowledge is a popular starting point to load data into Microsoft Fabric. Adding values from the Variable Library in a Dataflow is an obvious plan to make it more future proof and to work better with Deployment pipelines.

I will confess the first time I tried these I could not get them to work till I read the instructions correctly. So they do work just understand the limitations!

To be fair, following instructions is one of the most challenging things to do, it seems.

Comments closed

Near-Real-TIme Reporting on SQL Server Data with Microsoft Fabric

Rebecca Lewis continues a series on Microsoft Fabric:

You already know the options. Run heavy reporting queries against production. eewgh. Or stand up a reporting replica, build ETL to keep it current, maintain a refresh schedule, and hope nothing breaks on a holiday weekend. It works, but it’s expensive and has an awful lot of moving pieces.

Fabric gives you a third path: continuously replicate your SQL Server data into OneLake using Fabric Mirroring, and let Power BI read it using Direct Lake mode. Your SQL Server stays focused on OLTP and your reporting runs against a near real-time copy in Fabric. No pipelines. No refresh schedules. Nice.

Read on for the options available with Microsoft Fabric, as well as an endearing note that “real-time” isn’t.

Comments closed

Interoperability and OneLake Security

Aaron Merrill introduces a new whitepaper:

In our whitepaper, The future of data security is interoperability, we make the case for a different data foundation: interoperable security that’s defined once and enforced everywhere your data is used. Using OneLake security as the lens, it walks through the core concepts and architectural choices behind centralized policy definition with distributed, engine-level enforcement, and explores how fine-grained access controls and enterprise governance fit into a multi-engine world.   

Click through for Aaron’s summary and check out the link for the whitepaper itself, in PDF format.

Comments closed

Mirroring to OneLake without Public Internet Access

Paul Hernandez builds a (virtual) network:

Mirroring has been a transformative technology for data integrations tasks since the early Microsoft Fabric days. Moreover, this feature has been called “pain killer as a service” in community posts. In many projects, data sources to be mirrored are behind private networking and for security reasons they are not accessible using public internet. If you want to mirror, for example, an Azure SQL database, you’ll need a data gateway. According to the official docs: “If your Azure SQL Database is not publicly accessible and doesn’t allow Azure services to connect to it, you can set up virtual network data gateway or on-premises data gateway to mirror the data”.

In this post I’ll show you step-by-step how to set up connectivity to be able to use mirroring when Azure SQL allows only private access.

There are several steps involved, but the end result is worth it compared to not having the data at all or needing to make it accessible over the Internet.

Comments closed

Estimating Overall Fabric Capacity Utilization

Gilbert Quevauvilliers backs into a number:

I was recently working with a customer and one of the questions they had is we are going to be running an ingestion process. We want to know how much Fabric Capacity this will be consuming.

The challenge with this question is that in Fabric a background capacity gets smoothed over 24 hours.

For example, when looking at the Capacity Metrics App I can see my overall usage, but HOW MUCH CAPACITY IS IT CONSUMING?

Read on for the answer.

Comments closed

Combining Fabric Real-Time Intelligence, Notebooks, and Spark Structured Streaming

Arindam Chatterjee and QiXiao Wang show off some preview functionality:

Building event-driven, real-time applications using Fabric Eventstreams and Spark Notebooks just got a whole lot easier. With the Preview of Spark Notebooks and Real-Time Intelligence integration — a new capability that brings together the open-source community supported richness of Spark Structured Streaming with the real-time stream processing power of Fabric Eventstreams — developers can now build low-latency, end-to-end real-time analytics and AI pipelines all within Microsoft Fabric.

You can now seamlessly access streaming data from Eventstreams directly inside Spark notebooks, enabling real-time insights and decision-making without the complexity & tediousness of manual coding and configuration.

Click through to learn more.

Comments closed

The Troubles of Documentation: Microsoft Fabric API Edition

Rob Sewell walks through a recent experience:

Firstly, an apology to my friends (especially Randolph) in the documentation team at Microsoft. I know how hard you work to produce accurate and useful documentation, and I appreciate your efforts. This is not a criticism of your work, but rather an observation about the challenges I faced.

This is a story about a recent experience and the lessons learned along the way.

Read on for the issue and what Rob had to do. This is a case study in how hard it is to write good documentation, especially around the edges of what is possible.

Comments closed

Data Series Colors in Microsoft Fabric

Michal Bar shows off a new capability:

A frequent request we receive from dashboard authors is the ability to have greater control over color settings.

Until now, color assignments in real-time dashboards were largely automatic. While this worked for basic scenarios, it often fell short in operational and reporting use cases where color isn’t decoration—it’s meaning. Data Series Colors is a new capability that gives authors direct control over how colors are applied to their visuals.

Read on to see how it works for real-time dashboards.

Comments closed

Testing the Performance of Direct Lake vs Import Mode for Semantic Models

Gilbert Quevauvilliers performs some testing:

In this blog post I am going to show you how I completed the automated testing and then the results where I am going to compare Direct Lake, Import and DirectQuery and which one appears to be the best.

As always, your testing may very or be different to my tests below.

I would highly recommend that you use the method I have used and apply this testing technique to your data to understand and validate which semantic model would be best for your capacity.

Click through for details on the tests, query durations, and how the three major modes of data loading into Microsoft Fabric semantic models (Import, Direct Lake, Direct Query) fare.

Comments closed

Setting up Azure SQL Mirroring to Microsoft Fabric

Olivier Van Steenlandt troubleshoots an issue:

When setting up database mirroring from Azure SQL to Microsoft Fabric for one of my demo databases, I ran into an issue while trying to connect to my Azure SQL database.

As you can see in the screenshot above, it seems that a setting on my logical SQL Server in Azure is misconfigured. Let’s resolve that in a couple of steps.

Click through for the screenshot, the specific error, and how Olivier was able to get things working.

Comments closed