Press "Enter" to skip to content

Category: Microsoft Fabric

Microsoft Fabric and Dataverse

Published 2023-10-13 by Kevin Feasel

Jose Mendes let us know what’s going on with Dataverse:

If like me, you’ve been keeping taps on what Microsoft has been up to on the Power Platform world, you would have noticed that there are two concepts that are regularly referenced in their architectures and generally associated to each other, Azure Data Lake Storage (ADLS) Gen 2 and Common Data Model (CDM).

As Francesco referred in his blog, Microsoft ultimate vision is for the CDM to be the de facto standard data model, however, although there is a fair amount of resources talking about the capabilities and features, it can be a bit confusing to understand how you can actually store your data in the CDM format in ADLS and use it to run data analytics such as data warehousing, Power BI reporting and Machine Learning.

Read on for more of what’s happening on that front. I will admit that Dataverse tends to be way down on my list of priorities, but that’s because I’m a relational database snob.

Comments closed

Microsoft Fabric Roadmap

Published 2023-10-13 by Kevin Feasel

James Serra shares some thoughts on the Microsoft Fabric roadmap:

Just released was the Microsoft Fabric roadmap that you can check out at https://aka.ms/FabricRoadmap. It’s great to see Microsoft be transparent on what features they are working on and when they will be available.

Here are my top 18 features on the roadmap that I am most excited about (in the order found in the roadmap):

Seems like about half of what James is looking forward to releases in Q4 and the other half releases in mid-2024.

Comments closed

Augmenting the Gold Layer in Microsoft Fabric with Semantic Link

Published 2023-10-12 by Kevin Feasel

Nikola Ilic shows off one use case for Semantic Link:

I won’t spend time explaining what Semantic Link is – you can check a wonderful article written by my friend Sandeep Pawar, or refer to the official blog post. Sandeep’s blog post does a great job explaining not just what Semantic Link is, but also what are the possible use cases of this new feature.

Therefore, I will focus on explaining how you can leverage Semantic Link for a specific use case: I call it “Augmenting Gold Layer” (copyrights reserved). And, we will perform this “operation” by using SQL! Yes, you heard me well – we will leverage SparkSQL language to go above and beyond and “transform” the data currently sitting in Power BI datasets.

I will say that, for obvious reasons, this blog supports the Raw/Refined/Curated naming convention rather than Bronze/Silver/Gold, so I’d posit that this should be called the Augmented Curated Layer.

I can also recommend reading the blog post from Sandeep Pawar. It did a really good job of explaining why Semantic Link is worth getting excited about.

Comments closed

Refreshing a Direct Lake Power BI Dataset in Microsoft Fabric

Published 2023-10-12 by Kevin Feasel

Chris Webb refreshes our memories:

If you’ve heard about the new Direct Lake mode for Power BI datasets in Fabric you’ll know that it gives you the query performance of Import mode (well, almost) without the need to actually import any data. Direct Lake datasets can be refreshed though – in fact, they refresh automatically by default – and if you look at the dataset’s Refresh History you’ll see there’s a Direct Lake section which sometimes shows errors:

Chris goes on to ask and answer the question, what does it mean to refresh a Direct Lake dataset if you’re not actually importing the data into Power BI?

Comments closed

Cache Management and Semantic Link in Fabric Notebooks

Published 2023-10-12 by Kevin Feasel

Marc Lelijveld warms up the cache:

In the previous blog, I wrote about data temperature as part of Fabric when you’re using Direct Lake storage mode. In that blog, I explained how you can get insights in the temperature of a column, what that temperature means and what the impact of the temperature is on columns that are queried more often.

In this blog, I will continue this story by elaborating on a process called framing and how you can influence data eviction to drop data from memory. Finally, this blog goes into more details on how you could use Semantic Link in Fabric Notebooks to warm up the data for most optimal end-user performance.

The SQL Server analog here is having some automated queries which keep specific pages in the buffer pool, like a warm-up script for an instance with plenty of memory but slow disks.

Comments closed

A Path to Avoid Getting Overwhelmed with Microsoft Fabric

Published 2023-10-10 by Kevin Feasel

Kurt Buhler tries to limit information overload:

It’s just too much; I don’t have time for all this stuff.

I think this is a big problem. It’s a problem not just because people shouldn’t feel overwhelmed, but also because it says something about how effectively these new features, tools, and resources are being communicated, understood, and used. But what is the problem, exactly? And if you’re in the minority of people not feeling overwhelmed, why should you care?

Perhaps most importantly, how can we approach these new features, tools, and resources to ensure we understand them and can find value without feeling overwhelmed?

Read on for several tips on how to tackle learning about a product with a large surface area. And I’d also note that anybody who is comfortable working in SQL Server had to go through the same process.

Comments closed

Workaround for Primary Keys in Fabric Data Warehouses

Published 2023-10-09 by Kevin Feasel

Gilbert Quevauvilliers needs a key:

When I started looking into using the data warehouses feature in Fabric, I did see that there were limitations on Primary Key columns.

Below is my blog post on how I still use keys in my data warehouse, instead of using GUID’s which to me are long and hard to use.

In my example I am going to create a simple data warehouse which is going to consist of two-dimension tables (Date and Country) and a fact table with the Sales amounts.

This seems sub-optimal, though at least Gilbert shows us a workaround.

Comments closed

Data Temperature in Microsoft Fabric

Published 2023-09-29 by Kevin Feasel

Marc Lelijveld breaks out the thermometer:

As part of Microsoft Fabric, a new storage mode to connect from Power BI to data in OneLake has been introduced. Direct Lake it makes to possible to use your data from OneLake in Power BI without taking an additional copy of the data. Where Direct Lake promises to deliver the performance of Import-mode with the real-time capabilities of Direct query, it is time to have a closer look how data gets loaded into memory and delving into the concept of data dictionary temperature.

In this blog I will explain when data gets loaded into memory, elaborate on how you can measure the dictionary temperature of your data and the effect of queries on the temperature.

Click through to see what affects this measure and how.

Comments closed