Press "Enter" to skip to content

Author: Kevin Feasel

Microsoft Fabric ETL and the Air Traffic Controller

Jens Vestergaard rethinks a metaphor:

In February 2025 I wrote about building an event-driven ETL system in Microsoft Fabric. The metaphor was air traffic control: notebooks as flights, Azure Service Bus as the control tower, the Bronze/Silver/Gold medallion layers as the runway sequence. The whole system existed because Fabric has core-based execution limits that throttle how many Spark jobs run simultaneously on a given capacity SKU.

The post was about working around a constraint. You could not just fire all your notebooks at once. You needed something to manage the queue.

More than a year on, it is worth being honest about what held up and what has changed.

Read on to see what has changed in this past year and how Jens thinks of it today.

Leave a Comment

Updates to Straight Path Solutions sp_Check Procedures

Jeff Iannucci has some updates:

This month though there are mostly a few small updates for the tools, as next month’s updates should also include many of the issues noted in GitHub (thanks for those!) Most of this month’s updates were to sp_CheckSecurity, although we did add “Initial File Size” to the output of sp_CheckTempdb since that had been requested by a few folks.

We hope these new updates can help you, especially if you’ve never used these stored procedures. Here are the updates for March 2026, with links to the GitHub repositories where you can download the latest versions.

Click through for those links.

Leave a Comment

Read-Write Ratios in SQL Server Databases

Louis Davidson resurrects an old article:

When I wrote this post, I had a very strong belief that my OLTP databases are heavy writes and OLAP heavy reads. But in many cases this isn’t exactly the true story. Why? Because you have to read a lot of data to update, delete and display data. Even inserts have reads to look for duplicates, for example. OLAP systems should likely be read more times than write, but loading data can be a lot of data moved around and reads maybe less if the queries are efficient. When I was looking for stuff to post, this post stood out as one that I really liked (other than the aforementioned title!)

I came upon this idea when I found a post by Jason Massie (whose handle was statisticsio back then) that I had referenced, and Kendal Van Dyke alerted me this post at SQL Saturday in Atlanta (this would have been back in 2008 or 2009). After reading that (long gone) post, I wrote this post using sys.dm_db_index_usage_stats, and added references to sys.dm_io_virtual_file_stats as well.

Click through for the script. My expectations going in would be that OLTP and OLAP servers would actually have fairly similar read-write ratios. We talk about OLTP being “write-heavy” and OLAP being “read-heavy” but in practice, you need a lot of write behavior to sustain a warehouse—especially one that isn’t just a “truncate and reload once a day” type of thing—and people care about reading the data from their OLTP systems. In practice, I see the split as more of optimizing for accuracy in data entry (OLTP) versus simplicity of reading data (OLAP).

Leave a Comment

Performance Monitor Full Installation Notes

Erik Darling has a pair of documentation videos for Performance Monitor. First up is the full dashboard installation:

In this video, I delve into the installation process of my free SQL Server performance monitoring tool, addressing a common issue where users have questions about setup despite having access to detailed documentation. I explain that there are two main installers: one for command line use and another with a graphical user interface (GUI). The command line installer offers various options such as reinstalling the database, choosing encryption levels, managing server certificates, and handling password input in different ways. Additionally, I cover how the tool runs through 52 installation scripts, including community dependencies like SP WhoIsActive and my own scripts for data collection and analysis.

After that, Erik shows off the user interface:

In this video, I delve into the exciting world of free SQL Server monitoring tools, specifically focusing on a dashboard I’ve developed called Performance Monitor. This tool is designed to make life easier for DBAs and developers by providing real-time insights into server performance, resource usage, blocking issues, and more. With a user-friendly interface that includes various tabs like an overview, resource metrics, and query analysis, the dashboard aims to simplify complex monitoring tasks. I also highlight some of its unique features such as the plan viewer, which leverages Microsoft’s SQL Server extension for Visual Studio Code to provide detailed execution plans with actionable insights. Whether you’re a seasoned DBA or just starting out, this tool can help you keep your SQL Server environment running smoothly and efficiently.

Click through to check out what Erik has to offer.

Leave a Comment

Power BI Version Control via Azure DevOps

Gilbert Quevauvilliers works with the on-again, off-again CI/CD solution Microsoft has to offer:

In this blog post is a way set up version control for Power BI semantic models (and reports) using the PBIP (Power BI Project) format, Azure DevOps (Azure Repos), and VS Code.

This approach treats your semantic model as readable text files (JSON/TMDL), enabling proper Git diffing, branching, merging, and collaboration—something binary .pbix files don’t support well.

Click through for the process.

Leave a Comment

Compressing Images in R

Yihui Xie announces a new package:

Last month, @bastistician opened an issue on the litedown repo pointing out that knitr has a hook_pngquant() function for compressing PNG plots from code chunks, but litedown lacks such a feature. He included a reasonable workaround—calling system2("pngquant", ...) with litedown::get_context("plot_files") in a chunk at the end of the vignette. It shrank his vignette from 80 KB to 54 KB, which is a 33% reduction. Not bad.

The catch, of course, is that it requires pngquant to be installed on the system. For R users, installing a system binary is more friction than it sounds: it is brew install pngquant on macOS, a separate package manager invocation on Linux, and hunting down a standalone executable on Windows. If you maintain a package that others will build, you are now asking all of them to do this—for every machine they use. By contrast, install.packages("tinyimg") works the same way everywhere, which is the kind of simplicity that makes a tool actually get used.

This is why I created tinyimg.

Read on for more details about how tinyimg works, how well it compresses, and how it integrates with litedown.

Leave a Comment

Apache Airflow Jobs in Fabric Data Factory

Mark Kromer makes an announcement:

The world of data integration is rapidly evolving, and staying up to date with the latest technologies is crucial for organizations seeking to make the most of their data assets. Available now are the newest innovations in Fabric Data Factory pipelines and Apache Airflow job orchestration, designed to empower data engineers, architects, and analytics professionals with greater efficiency, flexibility, and scalability.

Read on to see what’s newly available, including some preview functionality.

Leave a Comment

New Permissions in SQL Server 2025

Andreas Wolter digs into some new permissions:

This article provides a brief overview of the new permissions introduced with SQL Server 2025 and the few adjustments to existing ones.

Overall, the changes are subtle – which is not surprising. The past couple of years have focused heavily on SQL Database in Fabric, Entra ID integration, and incremental improvements rather than major changes to the core permission model.

All newly introduced permissions are tied to new functionality. The underlying permission model itself remains unchanged since its last significant evolution in SQL Server 2022.

Even so, click through to see what’s new and what granting EXECUTE across the board can net you in SQL Server 2025.

Leave a Comment

Auditing SQL Agent Jobs for a Migration

Andy Brownsword takes a peek:

Most SQL Server environments have more jobs, schedules, and hidden complexities than you realise. It’s only when you arrive at a migration and peek under the hood that the scale is clear.

Here we’ll pull out details from msdb to give a clear snapshot of what you’ll actually be dealing with. If you don’t understand the effort upfront, the migration will expose it.

The thrust of Andy’s post covers migration, but I think it’s a good query to run simply to get a better understanding of all of the jobs in your environment.

Leave a Comment