A Fine Slice Of SQL Server
Reitse Eskens suffers the consequences in this Highlander world:
Last week, we ran into an issue where deployment pipelines are owned by a person who no longer works for our organisation. This meant we could not access the deployment pipelines owned by this account.
Click through to see how to do this, including the code necessary.
Leave a CommentChris Webb finds a workaround to something that used to work:
Back in September 2024 I wrote a blog post on how to create multiple copies of the same dimension in a Direct Lake semantic model without creating copies of the underlying Delta table. Not long after that I started getting comments that people who tried following my instructions were getting errors, and while some bugs were fixed others remained. After asking around I have a workaround (thank you Kevin Moore) that will avoid all those errors, so while we’re waiting for the remaining fixes here are the details of the workaround.
I look at the set of steps needed to do this and say there has to be a better way.
Leave a CommentJens Vestergaard rethinks a metaphor:
In February 2025 I wrote about building an event-driven ETL system in Microsoft Fabric. The metaphor was air traffic control: notebooks as flights, Azure Service Bus as the control tower, the Bronze/Silver/Gold medallion layers as the runway sequence. The whole system existed because Fabric has core-based execution limits that throttle how many Spark jobs run simultaneously on a given capacity SKU.
The post was about working around a constraint. You could not just fire all your notebooks at once. You needed something to manage the queue.
More than a year on, it is worth being honest about what held up and what has changed.
Read on to see what has changed in this past year and how Jens thinks of it today.
Leave a CommentMark Kromer makes an announcement:
The world of data integration is rapidly evolving, and staying up to date with the latest technologies is crucial for organizations seeking to make the most of their data assets. Available now are the newest innovations in Fabric Data Factory pipelines and Apache Airflow job orchestration, designed to empower data engineers, architects, and analytics professionals with greater efficiency, flexibility, and scalability.
Read on to see what’s newly available, including some preview functionality.
Leave a CommentPankaj Arora has a new ‘give us money’ lever:
Capacity overage, is a new opt‑in capability in Microsoft Fabric designed to help organizations keep their workloads running—even during unexpected compute spikes. Now available in preview, this feature allows for automatic billing for excess capacity usage, based on limits you set, instead of throttling operations, ensuring smoother experiences when workloads exceed the limits of your purchased capacity.
I will say that I think it’s reasonable to have the two options of throttling (you went over by 30%, so for a stretch of time you’ll be capped until you get back under the limit) or simply paying. The controversy around this was mostly in the fact that, if you shut off and restart your Fabric capacity, you’d automatically be charged for the overages you created. To that end, providing more options on how to work off that overage debt is useful.
Leave a CommentSo many cool Fabric features get announced at Fabcon that it’s easy to miss some of them. The fact that you can now not only generate Excel files from Fabric Dataflows Gen2, but that you have so much control over the format that you can use this feature to build simple reports rather than plain old data dumps, is a great example: it was only mentioned halfway through this blog post on new stuff in Dataflows Gen2 Nonethless it was the Fabcon feature announcement that got me most excited. This is because it shows how Fabric Dataflows Gen2 have gone beyond being just a way to bring data into Fabric and are now a proper self-service ETL tool where you can extract data from a lot of different sources, transform it using Power Query, and load it to a variety of destinations both inside Fabric and outside it (such as CSV files, Snowflake and yes, Excel).
Click through for an example.
Leave a CommentJoey D’Antoni applies Azure principles to Microsoft Fabric:
Let’s take a step back and talk about why I built this session. Like it or not, Microsoft’s intention with Fabric (and Power BI before it) is to make it easier for less-technical business users to build and consume data-driven reports. While I understand this mission, and it has been wildly successful in spreading love for Power BI, despite Fabric’s software-as-a-service branding, it’s actually a fully fledged data engine that needs to be well-managed to ensure data governance, security, and adherence to general best practices. In building my demos, I created a sample workspace with a couple of objects.
Click through for more notes on Joey’s talk, as well as a link to the code.
Comments closedNikola Ilic answers a question and then the follow-up question:
If you’ve spent any time in the data engineering world, you’ve likely encountered this debate at least once. Maybe twice. Ok, probably a dozen times “Should we process our data in batches or in real-time?” And if you’re anything like me, you’ve noticed that the answer usually starts with: “Well, it depends…”
Which is true. It does depend. But “it depends” is only useful if you actually know what it depends on. And that’s the gap I want to fill with this article. Not another theoretical comparison of batch vs. stream processing (I hope you already know the basics). Instead, I want to give you a practical framework for deciding which approach makes sense for your specific scenario, and then show you how both paths look when implemented in Microsoft Fabric.
Read on to learn why both are viable patterns and how you can work with both in Microsoft Fabric.
Comments closedMeagan Longoria wants to make sure things are working as expected:
I previously wrote about how the underlying technology for Fabric mirroring changed with SQL Server 2025. The latest version of mirroring that uses the SQL Server Change Feed is reading from the database transaction logs and pushing the data to a landing zone in OneLake. The data is then merged into the Delta tables for the Fabric mirrored database.
In this blog post, we will look at how to monitor this process, both in SQL Server and in Fabric.
Click through for information on the right DMVs to query and what you can find within Microsoft Fabric itself.
Comments closedJens Vestergaard takes a peek at Ontologies in Microsoft Fabric:
I have been spending a little time with the Microsoft Fabric data agent documentation lately, and one pattern keeps showing up, and it is not just in the official guidance but in community posts from people who have actually tried to deploy these things: the demo runs beautifully. The AI answers questions in plain English, leadership gets excited, the pilot gets approved. Then it hits production. Real users send real questions. The answers start drifting. Numbers that should match do not. The same question returns different results on different days. Trust evaporates faster than it was built.
And almost every time, the root cause is the same thing: the semantic foundation was not solid enough before anyone pointed an agent at it.
That is exactly the problem the Fabric Ontology is designed to address. It is the piece I think most teams will underestimate right up until the moment they need it.
Click through for an explanation. As I continue learning more about the concept of ontologies (not just in Fabric but in general), I’m slowly coming around to the idea. Though it still reminds me a lot of object-oriented programming with a no-code interface.
Comments closed