Press "Enter" to skip to content

Day: July 24, 2025

Retry Resiliency in Apache Kafka Pipelines

Published 2025-07-24 by Kevin Feasel

Ravi Teja Thutari explains the value of idempotence in moving data between systems:

In modern flight booking systems, streaming fare updates and reservations through distributed microservices is common. These pipelines must be retry-resilient, ensuring that transient failures or replays don’t cause duplicate bookings or stale pricing. A core strategy is idempotency: each event (e.g., a fare-update or booking command) carries a unique identifier so processing it more than once has no adverse effect. 

Read on to learn more. For reference, idempotence is a property of an operation where you can run through the operation as many times as you wish and will always end up at the same result. In the data operations world, this ties to the final state in a database. If I run a process once and it adds three rows to the database, I should be able to run the process a second time and end up with those exact three rows, no more, no fewer, and no different.

Leave a Comment

Copilots, MCP Servers, and Connection Strings

Published 2025-07-24 by Kevin Feasel

Chad Baldwin shares a warning:

Well, a few days ago, I ran into the result of one of those awkward pieces when combining the MSSQL extension for VS Code, MSSQL MCP Server and Copilot.

The short of it is…I asked Copilot to change the connection used by the MSSQL extension to use a particular database. I later asked Copilot to describe a table in the database (which uses the MSSQL MCP server), only for it to claim the table didn’t exist. I realized right away it was due to competing connections between the MSSQL extension and the MSSQL MCP Server configuration. It was also at that moment where I realized this situation could potentially be SO MUCH worse than simply not finding a table…

So let’s set up a worst case scenario and see what happens.

This is basically the equivalent of “Wait, that SSMS window was production? Uh-oh.” Not that this has ever happened to me, of course. Or any of you. Nope.

Leave a Comment

Optimizing Multi-Notebook Jobs in Microsoft Fabric and AWS Glue

Published 2025-07-24 by Kevin Feasel

Daniel Janik flips a switch:

Are your Azure Fabric pipelines with multiple notebooks running slower than you’d like? Are you paying for more Spark compute time than you should be? The culprit might be a simple setting that’s easy to miss. In this blog post, we’ll dive into the “For pipeline running multiple notebooks” setting in Azure Fabric and explain why enabling it can significantly improve your pipeline’s performance and reduce your costs.

Click through for this, as well as a comparison with AWS Glue and ways to perform something similar there.

Leave a Comment

A Deep Dive into IDENTITY Columns

Published 2025-07-24 by Kevin Feasel

Vlad Drumea performs a deep dive:

In SQL Server, IDENTITY is a column-level property that is used to provide an auto-incremented value for every new row inserted.

All you have to do is provide a seed value and an increment value when defining said column, and SQL Server will handle it from there.

Unlike sequences, identity columns do not require additional objects like default constraints or triggers to ensure the column is populated.

I’m glad that Vlad made a demo showing how @@IDENTITY works and how it can give you unexpected outputs if you’re not aware of a trigger working with a separate identity column. That one tends to get people.

Leave a Comment

Post-Install Configuration Tips for SQL Server

Published 2025-07-24 by Kevin Feasel

Kevin Hill shares some suggestions:

The SQL Server installer has gotten better: tempdb configuration, MAXDOP, and even max memory can now be configured during setup.

But don’t be fooled: there’s still a post-install checklist that can make or break your environment over time. If you’ve ever inherited a server that “just ran” for years and started getting slower over time you’ve likely seen what happens when this list gets ignored.

These are not in any particular order, but some do require a restart of the server or the SQL Server Engine service to take effect:

Click through for several tips. Most of them I agree without reservation. For a couple of them, like backup compression, there are specific circumstances in which I’d recommend against turning it on, namely when working with storage subsystems that deduplicate your data automatically. In that case, you might want to think harder about backup compression (or backup encryption, for that matter) before turning it on.

Leave a Comment