A Fine Slice Of SQL Server
We are using Microsoft Fabric for our analytical data platform. The data is hosted in a warehouse and modeled in a star schema. Using the GraphQL API for Fabric, we are exposing it to end users, and for some reports, we want to join data together. Is this possible in a GraphQL query?
Read on for Koen’s solution to this issue.
Comments closedGilbert Quevauvilliers builds a report:
One of the things I have found when working with my customers in Microsoft Fabric is that there is currently no way to easily view the total storage for the entire tenant.
Not only that, but it would also be time consuming and quite a challenge to then find out what is consuming the storage. Could it be large files or tables or warehouse tables?
In this blog post I will show you how using a Notebook you can get details of the storage across your Microsoft Fabric Tenant.
Click through for an image of the Power BI report and how you can get there.
Comments closedKristina Mishra takes us down an alley of pain:
If you’ve bought a P1 reserved capacity, you may have been told “No worries – it’s the same as an F64!” (Or really, this is probably the case for any P to F sku conversion.) Just as you suspected – that’s not entirely accurate. And if you are trying to create Fabric shortcuts on a storage account that uses a virtual network or IP filtering – it’s not going to work.
The problem seems to lie in the fact that P1 is not really an Azure resource in the same way an F sku is. So when you go to create your shortcut following all the recommend settings (more on that in a minute), you’ll wind up with some random authentication message like the one below “Unable to load. Error 403 – This request is not authorized to perform this operation”:
On the “oof” scale, this rates as “big oof.” Kristina shows some of the differences between P SKUs and F SKUs and why it matters, as well as two unpalatable solutions if you happen to be using a P SKU.
Comments closedAndy Leonard continues a series on Fabric Data Factory design patterns:
In an earlier post titled Fabric Data Factory Design Pattern – Basic Parent-Child, I demonstrated one way to build a basic parent-child design pattern in Fabric Data Factory by calling one pipeline (child) from another pipeline (parent). In a later earlier post titled Fabric Data Factory Design Pattern – Parent-Child with Parameters, I modified the parent and child pipelines to demonstrate passing a parameter value from a parent pipeline when calling a child pipeline that contains a parameter.
In this post, I modify a parent pipeline to explore parameterizing which child pipeline will be called by the parent pipeline. In this post, we will:
- Copy the child pipeline id
- Clone a parent pipeline
- Add and configure a pipeline variable for the child pipeline id
- Test the dynamic pipeline id
Read on to see how.
Comments closedPaul Turley shares some thoughts:
Hello, friends. I’ve spent the past few months working with several new Fabric customers who were seeking guidance and recommendations for Fabric architecture decisions. What have we learned about using Fabric in enterprise data settings in the past 11 months? This post covers some of the important decisions points and Fabric solution design patterns.
Much of the industry’s experience with Microsoft Fabric over the past several months has been at a high-level as organizations were dipping their toe in the pool to test the water. So far, our Data & AI team have assisted around 50 clients with Fabric projects of various sizes. We have also implemented a handful of production scale projects with enterprise workloads, comparing notes with community leaders and the product teams who develop the product. What lessons have we learned?
Click through for several bits of high-level architectural guidance intended to make that adoption easier.
Comments closedGilbert Quevauvilliers grants some permissions:
This blog post explains how to configure access for my Service Principal to interact with the Azure Storage API to use the API to get details for Microsoft Fabric Storage.
This is part of a blog post series where I am going to show you how to “View Total Storage consumed in Microsoft Fabric”
When I started this blog post I realized that I first need to explain how to configure the Service Principal authentication to interact with the Azure Storage API permissions. This is because in my notebook these steps are required for the notebook to run successfully.
Read on to find out how.
Comments closedPrathy Kamasani does a bit of movement:
Regardless of which architecture we follow, during stages of data integration and transformation there’s always a step to move data from one location to another. And, we work with multiple tables, schemas, and even lake houses.Same goes with Fabric Notebooks. I often find myself in scenarios where I don’t want to attach Lakehouse to my notebook, but I do want to read or write data from various bakehouses.
I recently blogged about a way to achieve this as part of documenting your workspaces. In that post, I described how to write data to a workspace that was not attached to the notebook. I used
MsSparkUtil(renamed to NotebookUtils) to mount and then write data in the Lakehouse as Delta tables.
Read on for the answer.
Comments closedNikola Ilic forges a new identity:
If you’ve ever worked with traditional relational database management systems (RDBMS) and/or data warehouses, and you’re now trying to be a “modern data platform professional” and apply your skills in Microsoft Fabric, you may find yourself in uncharted territory. Not only because of the SaaS-ification of the environment, but also due to many puzzling “solutions”, or maybe it’s better to say – lack of the features that we were taking for granted in the “previous” (pre-Fabric) life.
The goal of this article is to introduce you with different approaches for overcoming the limitation of non-existency of the identity columns in Microsoft Fabric. Please keep in mind that all of these approaches are considered workarounds and it may happen that Microsoft in the future provide the out-of-the-box solution
Missing the identity column attribute can be a bit annoying when building out dimensions, so Nikola provides a few tips on how to emulate this functionality.
Comments closedAndy Leonard takes us through a design pattern:
In an earlier post, I demonstrated one way to build a basic parent-child design pattern in Fabric Data Factory by calling one pipeline (child) from another (parent). In this post, I modify the parent and child pipelines to demonstrate calling a child pipeline that contains a parameter. In this post, we will:
- Clone and edit the child pipeline
- Clone and edit the parent pipeline
- Test
Read on to see how it works.
Comments closedStepan Resl talks orchestration:
Let’s start by introducing what orchestration is and why it’s important to talk about shared resources. Orchestration is a discipline focused on managing and coordinating individual items or control elements to collectively manage the flow of our data operations. In the context of Fabric, this involves managing notebooks, dataflows, pipelines, stored procedures, semantic model updates, and many other items, activities, and services that may even be outside of Fabric.
Read on for some of the options, how they work in Microsoft Fabric, and tips for success.
Comments closed