Press "Enter" to skip to content

Category: Cloud

Re-Migration and Data Engineering

Published 2026-06-11 by Kevin Feasel

Andy Leonard shares some thoughts:

Right-sizing didn’t always work out the way some clients were led to believe it would.

In nearly every instance, the right-sizing argument was presented (sold) as the solution to over-provisioning, or purchasing hardware to serve peak loads. The classic example was a US income tax service that needed more and faster compute available to meet increasing demand starting in late January and peaking in mid-April each calendar year. After mid-April, hardware that was beefy enough to handle that peak load sat mostly idle for the next 9 months.

I don’t think I’ve ever worked for a company where this scenario really made sense. Even in the e-commerce company where a sizable fraction of our total annual revenues happened over a 5-day period, the load was still significant enough the rest of the year that we made good use of our on-premises SQL Server hardware.

Leave a Comment

Automatic Index Compaction in Azure SQL

Published 2026-05-29 by Kevin Feasel

Chad Callihan takes a look at a preview feature:

There isn’t one set way to manage indexes. Maybe you use Ola Hallengren scripts. Maybe it’s something you put together yourself. Either way, there might be a big shift coming for SQL Server database administrators and how index management is handled.

Last month, Microsoft announced Automatic Index Compaction, which is in preview for Azure SQL Database, Azure SQL Managed Instance, and SQL Database in Fabric. Instead of utilizing something like Ola Hallengren scripts or your own homegrown setup to monitor and rebuild indexes, the database engine will continuously run in the background and handle indexes for you, hence the “automatic” in the name.

Read on to see how it works, as well as a note around page density and index fragmentation. But Jeff Moden makes a good point in the comments, so check that out.

Leave a Comment

An FAQ-Based Introduction to Data Factories in Azure

Published 2026-05-26 by Kevin Feasel

Koen Verbeeck answers some questions:

Is Microsoft Fabric replacing Azure Data Factory?

Officially, no. Or maybe not yet. At the time of writing, ADF still remains a separate product but it’s noticeable that more new features are added to Fabric than to ADF. There are still many customers using ADF, so Microsoft might keep the service around for a while. There’s also still a bit of a feature gap between the two services, but this becomes more narrower each month. Microsoft is offering migration scenarios from ADF to Fabric.

I picked this question because of how much the concept annoys me. There are three separate Data Factory code bases in Azure with overlapping but not matching functionality (which is how you can tell it’s multiple code bases and not just one code base reskinned). This can lead to a scenario where Person A says, “Oh, do this thing in Data Factory.” Person B then says, “But I can’t do that in Data Factory.” Person A’s response: “Oh, that’s weird, because I can do it in Data Factory.” This leads to necessary but somewhat absurd clarifications around how you need to use Microsoft Fabric Data Factory, not Azure Data Factory because, even though Microsoft Fabric Data Factory is hosted in Azure, it’s a different product.

And don’t get me started on the wide variety of KQL platforms, all of which are subtly different.

Leave a Comment

Visualizing Redshift Explain Plans

Published 2026-05-07 by Kevin Feasel

Eduardo Pivaral has created a tool:

PlanTrace takes your raw EXPLAIN output, parses it into a structured tree, and gives you three ways to reason about it — all in your browser, with no data ever leaving your machine.

Click through for a link. It looks a bit like Paste the Plan, in that it’s an online tool that accepts an explain (or execution) plan and generates a visual output for it.

Comments closed

Managed Identities in SQL Server 2025

Published 2026-05-06 by Kevin Feasel

Greg Low offers another security option for service management:

Those who have worked with SQL Server will understand the need to avoid storing passwords for accessing resources. Windows-based identities are fine for on-premises SQL Server systems, including those on cloud-based virtual machines (VMs), but are of no use when you need to access cloud-based resources like those in Azure.

Some Azure-based resources (including storage accounts) offer other access methods, such as shared access signatures (SAS), but these aren’t much of a step-up from passwords.

What’s really needed is for SQL Server to have its own Microsoft Entra based identity. These can be used directly with Azure-based resources – and that’s exactly where managed identities come in.

Click through to see how it works. Importantly, this is a feature that requires additional payment.

Comments closed

Microsoft Fabric Eventstream Network Security Features

Published 2026-04-22 by Kevin Feasel

Alex Lin looks at network security features:

Eventstream in Fabric Real-Time Intelligence stream data from both inside and outside the Fabric platform. When your external sources sit behind firewalls or in private networks, choosing the right network security feature is essential. This post breaks down the available options in Eventstream and helps you determine which one fits your scenario.

Click through for more information.

Comments closed

An Azure Bill Breakdown

Published 2026-04-17 by Kevin Feasel

Elaine Cahill takes us through an Azure monthly bill:

I received an Azure bill for the period Feb. 6th 2026 – March 5th 2026 that was $2.38 usage charges, with no tax added. Although a small amount, I decided it would be a good introductory example for anyone that has to understand and pay for Azure. My account is Pay-As-You-Go and I use it for learning, experiments and proof of concepts.

Click through for that primer. I think, on the whole, the way Azure shows billing is okay. The tricky part is when you want to reduce the bill. They’ll show you, for example, that you have D3 v2 or DS3 v2 virtual machines running in East US 2, but then you have to dig in and figure out which of your virtual machines are running that SKU. And there are some services that spin up VMs in the background, so you might see billing for that even if you didn’t directly create a VM of that SKU.

But with a bit of digging, you can at least gain an understanding of what’s costing money in Azure.

Comments closed

Syncing Logins across Failover Groups for Managed Instances

Published 2026-04-15 by Kevin Feasel

Andy Brownsword gets replicating:

Failover Groups for Managed Instances are a great option to replicate data, but they don’t replicate key instance elements – one of which is logins that live in the master database. If left unchecked, failovers leave systems unable to connect and panic ensues.

To alleviate this we’ll look at a script to synchronise logins and permissions across replicas.

Click through for a link to the script and an explanation of what’s going on with it.

Comments closed

Kafka Topic Management in Amazon MSK

Published 2026-04-03 by Kevin Feasel

Swapna Bandla, et al, dig into a managed service:

If you manage Apache Kafka today, you know the effort required to manage topics. Whether you use infrastructure as code (IaC) solutions or perform operations with admin clients, setting up topic management takes valuable time that could be spent on building streaming applications.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) now streamlines topic management by supporting new topic APIs and console integration. You can programmatically create, update, and delete Apache Kafka topics using familiar interfaces including AWS Command Line Interface (AWS CLI), AWS SDKs, and AWS CloudFormation. With these APIs, you can define topic properties such as replication factor and partition count and configuration settings like retention and cleanup policies. The Amazon MSK console integrates these APIs, bringing all topic operations to one place. You can now create or update topics with a few selections using guided defaults while gaining comprehensive visibility into topic configurations, partition-level information, and metrics. You can browse for topics within a cluster, review replication settings and partition counts, and go into individual topics to examine detailed configuration, partition-level information, and metrics. A unified dashboard consolidates partition topics and metrics in one view.

In this post, we show you how to use the new topic management capabilities of Amazon MSK to streamline your Apache Kafka operations. We demonstrate how to manage topics through the console, control access with AWS Identity and Access Management (IAM), and bring topic provisioning into your continuous integration and continuous delivery (CI/CD) pipelines.

Read on to see what the experience looks like using the MSK console.

Comments closed