A Fine Slice Of SQL Server
Chris Webb digs into the dark underbelly of a semantic model:
Following on from my recent post about refreshing semantic models with Fabric Data Pipelines and the semantic model refresh activity, a few people asked me how to refresh hidden tables because they are not displayed in the Pipeline configuration UI. I got the answer from my colleague Alex Powers (aka reddit celebrity u/itsnotaboutthecell) who kindly allowed me to blog about it.
Click through for the demonstration.
Comments closedJens Vestergaard extends a metaphor:
We have been working getting an enterprise grade event driven orchestration of our ETL system to operate like an airport control tower, managing a fleet of flights (data processes) as they progress through various stages of take-off, transit, and landing. All of this, because Microsoft Fabric has a core-based limit to the number of Notebook executions that a capacity can execute and have queued up in line for execution when invoking them using the REST API. Read the details here: limits (you know, it’s funny that there is no stated limits for Azure Service Bus Queues on number of messages in queue, but there is for Microsoft Fabric, which uses a Service Bus queue underneath…)
That limitation is a bit annoying, but read on to see how Jens uses this metaphor to explain the various parts of an ETL orchestration engine.
Comments closedKoen Verbeeck deploys a database:
When a SQL Database is in Microsoft Fabric, you can develop it locally in a database project. As part of the development process, you want to deploy this project to the online Fabric SQL Database. The database project also contains pre- and/or post-deployment scripts that need to be executed as part of the deployment process. How can this goal be achieved?
Click through for the answer.
Comments closedDennes Torres has a public service announcement:
I wrote about lakehouse maintenance before, about multiple lakehouse maintenances, published videos about this subject and provided sample code about it.
However, there is one problem: All the maintenance execution should be avoided over shortcuts.
The tables require maintenance in their original place. According to our solution advances, we start using shortcuts, lots of them. Our maintenance code should always skip shortcuts and make the maintenance only on the tables.
Click through to see how you can differentiate shortcuts from actual tables and write code to avoid shortcuts.
Comments closedChristopher Schmidt lays out a use case:
Operational reporting and historical reporting serve distinct purposes in organizations. Historically, data teams have heavily leaned on providing historical reporting, as being able to report on the operational business processes has proved elusive.
As a result, organizations have created reports directly against the operational database for operational needs or spend significant effort trying to get analytical tools to refresh faster using ‘micro-batching’ and/or keeping a tool like Power BI in directQuery mode. These efforts come with the goal of ‘moving data through the system as fast as possible’.
Click through for an architecture diagram and an example scenario.
Comments closedNicky van Vroenhoven notices a change:
In case you have access to the M365 Admin Center, or more specific the M365 Message Center, you might have seen this message. I reckon not many people did.. That’s why I’m blogging about it here
I’m specifically talking about this message in the Message Center, being a major update and with admin impact:
Read on for more information about what’s changing.
Comments closedArshad Ali announces a connector:
We are pleased to announce the availability of the Fabric Spark connector for Fabric Data Warehouse (DW) in the Fabric Spark runtime. This connector enables Spark developers and data scientists to access and work with data from Fabric DW and the SQL analytics endpoint of the lakehouse, either within the same workspace or across different workspaces, using a simplified Spark API. The connector will be included as a default library within the Fabric Runtime, eliminating the need for separate installation.
Click through to check out its capabilities. This is a tiny step toward where I think Microsoft Fabric should go: any tool accessing the same data, eliminating separate lakehouses vs warehouses and having to remember that you can’t use this syntax in this scenario unless you connect to it this way and sacrifice one live chicken.
Comments closedKevin Chant tries a Python package:
In this post I want to cover my initial tests of fabric-cicd. In order to provide some tips for those looking to work with this new offering.
Just so that everybody is aware, fabric-cicd is a Python library that allows you to perform CI/CD of various Microsoft Fabric items into Microsoft Fabric workspaces. At this moment in time there is a limited number of supported item types. However, that list is increasing.
Read on for the test. It currently supports a limit amount of functionality, but it looks promising.
Comments closedPatrick LeBlanc tells us what’s new:
There are a lot of exciting features for you this month! Here are some highlights: In Power BI, Explore from Copilot visual answers which lets you do easy ad-hoc exploration. In Data Warehouse, Browse files with OPENROWSET (Preview) and Copilot for Data Warehouse Chat (Preview). For Data Science, AI Skill is now conversational.
These are just some of the great features this month, keep reading to learn about all of what’s happened in Fabric this month.
Click through for the full report.
Comments closedMihir Wagle puts the kibosh on things:
On February 24, 2025, we launched Microsoft Fabric Quotas, a new feature designed to control resource governance for the acquisition of your Microsoft Fabric capacities. Fabric quotas aimed at helping customers ensure that Fabric resources are used efficiently and help manage the overall performance and reliability of the Azure platform while preventing misuse.
Note that these are not quotas you set on your users, but rather quotas that Microsoft sets on you.
Comments closed