Press "Enter" to skip to content

Category: Microsoft Fabric

Constraints in Microsoft Fabric Data Warehouses

Published 2023-11-29 by Kevin Feasel

Brian Bønk slips out of the constraints:

When working with data and building data models, I personally seldom use the constraints feature on a database. Call me lazy – but I think constraints are adding unnessesary complexity when building data models for reporting. Especially if you are working with the some of new platforms – like Microsoft Fabric, where you are using staleless compute, aka. data storage is seperated from the compute layer.

I understand the need for contraints on other database systems like OLTP systems.

In reporting models it can be somewhat usefull to have constraints between tables, as they help/force you to some level of governance in your datamodel.

But how can we use this in Microsoft Fabric and are they easy to work with?

Read on for those answers. I will note that I’m a stickler about constraints in transactional systems, though I agree that constraints in warehouses are not critical—assuming, at least, that you’re following the Kimball approach and have one and only one mechanism to write data, and that you have other mechanisms for vetting data quality.

Comments closed

Scraping the Microsoft Fabric Road Map with Microsoft Fabric

Published 2023-11-28 by Kevin Feasel

Prathy Kamasani wants a report, not a webpage:

Like many I am also playing with Fabric, many of my clients are also excited about Fabric and want to know more about it. Being a solution architect in the consulting world one of the most common questions I get asked is: “When certain features will be available, Where are they in the roadmap?”. That’s what sparked the idea of scraping the Microsoft Fabric Roadmap and creating this Power BI report. It is based on a Direct Lake connection, so it has been a bit temperamental.

So, what did I do it? If you are not interested in the whole story. Here is Python code you can run to get a road map. If you are interested in my process carry on reading 

Click through for the process and explanation.

Comments closed

Controlling Fallback Behavior in Direct Lake

Published 2023-11-28 by Kevin Feasel

Sandeep Pawar talks about fallback options:

When you create a Direct Lake semantic model, by default it is in Direct Lake mode, i.e. you will directly query the delta table from the lakehouse/warehouse. This is what we want because the query performance will be very much comparable to the import mode. However, under certain circumstances, the DAX query can fallback to DirectQuery if Direct Lake limitations are hit.

Read on to learn more about circumstances in which this could happen and ways to change the default behavior.

Comments closed

Exposing KQL Data in OneLake

Published 2023-11-28 by Kevin Feasel

Brian Bønk gets in on the Microsoft Fabric fun:

Microsoft has released the final piece of the current puzzle around the OneLake as a one-stop-shopping service for dat in Fabric. Until now we had only access to the KQL data in the KQL database.

With this addition, we can now finally say that OneLake is the one place for your data in Fabric.

Read on to see how you can make data in an existing KQL database usable in OneLake.

Comments closed

Scheduling Fabric Capacity Pause/Resume with Azure Logic Apps

Published 2023-11-27 by Kevin Feasel

Soheil Bakhshi doesn’t want to forget to turn off the power at night:

In the previous blog post, I explained Microsoft Fabric capacities, shedding light on diverse capacity options and how they influence data projects. We delved into Capacity Units (CUs), pricing nuances, and practical cost control methods, including manually scaling and pausing Fabric capacity. Now, we’re taking the next step in our Microsoft Fabric journey by exploring the possibility of automating the pause and resume process. In this blog post, we’ll unlock the secrets to seamlessly managing your Fabric Capacity with automation that helps us save time and resources while optimising the usage of data and analytics workloads.

Right off the bat, this is a rather long blog, so I added a bonus section at the end for those who are reading from the beginning to the end. With that, let’s dive in!

To spoil the bonus a little bit, Soheil shows us not only how to turn things on and off on a schedule, but also how to ignore certain days of the week. Read the whole thing to get that.

Comments closed

Starting a Free Trial of Microsoft Fabric

Published 2023-11-22 by Kevin Feasel

Andy Leonard kicks off a trial:

Are you interested in learning more about Microsoft Fabric?

One way to begin tinkering with the new platform is to start a free trial. At the time of this post, a free trial is available here:

Read on for instructions on how to try Fabric out. Now that Fabric is in GA, you’ll have to pay once the trial is over, but this does at least give you some time to check it out before then.

Comments closed

Visualizing JSON Files in Fabric Notebooks

Published 2023-11-22 by Kevin Feasel

Sandeep Pawar wants readability:

JSON is ubiquitous, particularly when working with APIs and logs. Its unstructured nature makes it highly flexible for handling anything from a simple array to a complex nested structure. However, this can also make it challenging for data analysis. When parsing JSON, it’s crucial to understand its structure so you can flatten it and convert it into a tabular format for analysis. Once the structure is identified, you can use pandas or PySpark to explode or normalize it into the desired shape. In this article, I will explain the method I use. While this approach is applicable to any notebook, there is a specific trick to make it work in a Fabric notebook.

Read on for that trick.

Comments closed

Getting Started with Microsoft Fabric

Published 2023-11-21 by Kevin Feasel

Eugene Meidinger is ready to be at the intersection of Dunning-Kreuger and Imposter Syndrome:

I’ve written before about struggling to learn Azure Synapse, and I’ve struggled as well with getting excited about Microsoft Fabric. I think the pitch and the potential of Microsoft Fabric is real. The issue is that it solves problems I don’t have. In my work, I don’t deal with data so big that Power BI can’t handle it. I don’t deal with data so unstructured that Power Query can’t handle it.

But I know I need to learn Fabric. Power BI is a part of Fabric, the integrations are only going to continue to improve. If nothing else, I need to be able to tell customers if they should look into using Fabric or not. So what do you do when there is a technology you aren’t excited about, but have to learn?

Read on for Eugene’s scenario, which is certainly more interesting than Adventure Works.

Comments closed

The State of Microsoft Fabric

Published 2023-11-21 by Kevin Feasel

Reitse Eskens shares some advice:

Last week the big announcement came at Microsoft Ignite, Fabric is GA.

Very cool, a lot of noise again for this shiny toolbox, but do we need to abandon everything and focus solely on the new toys?

Before I’ll answer that question, let’s look at a few moving parts of Fabric.

I think it’s still 1-2 years out from being fully baked. My hope is that there are (or will be) enough pieces in place to make it useful for enough scenarios that people don’t notice the gaps too much. There’s a lot of potential here and I don’t want Fabric to end up with a reputation of “too much stuff is missing to use it” because that reputation is hard to shake.

Comments closed

Lakehouse Management in Fabric via mssparkutils

Published 2023-11-21 by Kevin Feasel

Sandeep Pawar scripts out some lakehouse work:

At MS Ignite, Microsoft unveiled a variety of new APIs designed for working with Fabric items, such as workspaces, Spark jobs, lakehouses, warehouses, ML items, and more. You can find detailed information about these APIs here. These APIs will be critical in the automation and CI/CD of Fabric workloads.

With the release of these APIs, a new method has been added to the mssparkutils library to simplify working with lakehouses. In this blog, I will explore the available options and provide examples. Please note that at the time of writing this blog, the information has not been published on the official documentation page, so keep an eye on the documentation for changes.

This looks to be quite useful for CI/CD work.

Comments closed