A Fine Slice Of SQL Server
Jovan Popovic notes a feature going GA:
Fabric Data Warehouse now supports the ANY_VALUE() aggregate, making it easier to write readable, efficient T-SQL when you want to group by a key but still return descriptive columns that are functionally the same for every row in the group.
Right now, this is only available in the Fabric Data Warehouse, so no Azure SQL DB, Managed Instance, or box product support at this time.
Comments closedReitse Eskens suffers the consequences in this Highlander world:
Last week, we ran into an issue where deployment pipelines are owned by a person who no longer works for our organisation. This meant we could not access the deployment pipelines owned by this account.
Click through to see how to do this, including the code necessary.
Comments closedChris Webb finds a workaround to something that used to work:
Back in September 2024 I wrote a blog post on how to create multiple copies of the same dimension in a Direct Lake semantic model without creating copies of the underlying Delta table. Not long after that I started getting comments that people who tried following my instructions were getting errors, and while some bugs were fixed others remained. After asking around I have a workaround (thank you Kevin Moore) that will avoid all those errors, so while we’re waiting for the remaining fixes here are the details of the workaround.
I look at the set of steps needed to do this and say there has to be a better way.
Comments closedJens Vestergaard rethinks a metaphor:
In February 2025 I wrote about building an event-driven ETL system in Microsoft Fabric. The metaphor was air traffic control: notebooks as flights, Azure Service Bus as the control tower, the Bronze/Silver/Gold medallion layers as the runway sequence. The whole system existed because Fabric has core-based execution limits that throttle how many Spark jobs run simultaneously on a given capacity SKU.
The post was about working around a constraint. You could not just fire all your notebooks at once. You needed something to manage the queue.
More than a year on, it is worth being honest about what held up and what has changed.
Read on to see what has changed in this past year and how Jens thinks of it today.
Comments closedMark Kromer makes an announcement:
The world of data integration is rapidly evolving, and staying up to date with the latest technologies is crucial for organizations seeking to make the most of their data assets. Available now are the newest innovations in Fabric Data Factory pipelines and Apache Airflow job orchestration, designed to empower data engineers, architects, and analytics professionals with greater efficiency, flexibility, and scalability.
Read on to see what’s newly available, including some preview functionality.
Comments closedPankaj Arora has a new ‘give us money’ lever:
Capacity overage, is a new opt‑in capability in Microsoft Fabric designed to help organizations keep their workloads running—even during unexpected compute spikes. Now available in preview, this feature allows for automatic billing for excess capacity usage, based on limits you set, instead of throttling operations, ensuring smoother experiences when workloads exceed the limits of your purchased capacity.
I will say that I think it’s reasonable to have the two options of throttling (you went over by 30%, so for a stretch of time you’ll be capped until you get back under the limit) or simply paying. The controversy around this was mostly in the fact that, if you shut off and restart your Fabric capacity, you’d automatically be charged for the overages you created. To that end, providing more options on how to work off that overage debt is useful.
Comments closedSo many cool Fabric features get announced at Fabcon that it’s easy to miss some of them. The fact that you can now not only generate Excel files from Fabric Dataflows Gen2, but that you have so much control over the format that you can use this feature to build simple reports rather than plain old data dumps, is a great example: it was only mentioned halfway through this blog post on new stuff in Dataflows Gen2 Nonethless it was the Fabcon feature announcement that got me most excited. This is because it shows how Fabric Dataflows Gen2 have gone beyond being just a way to bring data into Fabric and are now a proper self-service ETL tool where you can extract data from a lot of different sources, transform it using Power Query, and load it to a variety of destinations both inside Fabric and outside it (such as CSV files, Snowflake and yes, Excel).
Click through for an example.
Comments closedJoey D’Antoni applies Azure principles to Microsoft Fabric:
Let’s take a step back and talk about why I built this session. Like it or not, Microsoft’s intention with Fabric (and Power BI before it) is to make it easier for less-technical business users to build and consume data-driven reports. While I understand this mission, and it has been wildly successful in spreading love for Power BI, despite Fabric’s software-as-a-service branding, it’s actually a fully fledged data engine that needs to be well-managed to ensure data governance, security, and adherence to general best practices. In building my demos, I created a sample workspace with a couple of objects.
Click through for more notes on Joey’s talk, as well as a link to the code.
Comments closedNikola Ilic answers a question and then the follow-up question:
If you’ve spent any time in the data engineering world, you’ve likely encountered this debate at least once. Maybe twice. Ok, probably a dozen times “Should we process our data in batches or in real-time?” And if you’re anything like me, you’ve noticed that the answer usually starts with: “Well, it depends…”
Which is true. It does depend. But “it depends” is only useful if you actually know what it depends on. And that’s the gap I want to fill with this article. Not another theoretical comparison of batch vs. stream processing (I hope you already know the basics). Instead, I want to give you a practical framework for deciding which approach makes sense for your specific scenario, and then show you how both paths look when implemented in Microsoft Fabric.
Read on to learn why both are viable patterns and how you can work with both in Microsoft Fabric.
Comments closedMeagan Longoria wants to make sure things are working as expected:
I previously wrote about how the underlying technology for Fabric mirroring changed with SQL Server 2025. The latest version of mirroring that uses the SQL Server Change Feed is reading from the database transaction logs and pushing the data to a landing zone in OneLake. The data is then merged into the Delta tables for the Fabric mirrored database.
In this blog post, we will look at how to monitor this process, both in SQL Server and in Fabric.
Click through for information on the right DMVs to query and what you can find within Microsoft Fabric itself.
Comments closed