Press "Enter" to skip to content

Category: Cloud

Troubleshooting Bad Request in ADF Pipelines

Published 2026-03-09 by Kevin Feasel

Koen Verbeeck said something bad:

A while ago I blogged about a use case where a pipeline fails during debugging with a BadRequest error, even though it validates successfully. If you’re wondering, this is the helpful error message that you get:

Click through for an image of the 400 Bad Request message, how Koen fixed it originally, and then a different scenario in which that 400 message popped up.

Ultimately, a 400 Bad Request comes down to “You sent me information that doesn’t make sense and I can’t fulfill your request, so fix it, dummy.” 400 status codes are very rude and insulting. Especially 418–that thing has a mouth like a sailor’s.

Comments closed

Creating Fabric Linked Service Parameters for ADO Deployment

Published 2026-03-06 by Kevin Feasel

Koen Verbeeck glues together several technologies:

Quite the title, so let me set the stage first. You have an Azure Data Factory instance (or Azure Synapse Pipelines) and you have a couple of linked services that point to Fabric artifacts such as a lakehouse or a warehouse. You want to deploy your ADF instance with an Azure Devops build/release pipeline to another environment (e.g. acceptance or production) and this means the linked services need to change as well because in those environments the lakehouse or warehouse are in a different workspace (and also have different object Ids).

When you want to deploy ADF, you typically use the ARM template that ADF automatically creates when you publish (when your instance is linked with a git repo). More information about this setup can be found in the documentation. To parameterize certain properties of a linked service, you can use custom parameterization of the ARM template. Anyway, long story short, I tried to parameterize the properties of the Fabric linked service. 

Read on to see how that went, as well as what you need to do to solve this issue.

Comments closed

Alerting People in Microsoft Teams from Data Factory Pipelines

Published 2026-03-06 by Kevin Feasel

Andy Brownsword sends a message:

Whether running Data Factory, Synapse, or Fabric pipelines, things go wrong – and the de facto response is to send an email. We’ve looked at sending emails from pipelines before, but at scale they can become noise and are easy to ignore.

A more effective option is to surface alerts where collaboration already exists, such as Teams.

In this post we’re going to start looking at using Teams and consolidate notifications into a channel. This functionality gives team members visibility, the ability to update in threads, and the option to tag people for a tighter response loop than typical emails bring.

Click through for the process.

Comments closed

Evaluating Power Query Programmatically in Microsoft Fabric

Published 2026-03-03 by Kevin Feasel

Mihir Wagle announces a new preview capability:

Power Query has long been at the center of data preparation across Microsoft products—from Excel and Power BI to Dataflows and Fabric. We’re introducing a major evolution: the ability to execute Power Query programmatically through a public API.

This capability turns Power Query into a programmable data transformation engine that can be invoked on demand through a REST API from notebooks, pipelines, and applications. Whether you’re orchestrating data pipelines, building custom data apps, or integrating Power Query into larger workflows, this API unlocks new flexibility and automation.

Click through for an overview of what’s available.

Comments closed

Using a Microsoft Fabric Variable Library in a Dataflow

Published 2026-02-13 by Kevin Feasel

Laura Graham-Brown shows another way to use variable libraries:

One of the popular low-code tools within Microsoft Fabric is the Gen2 Dataflow. Power BI report builders already know some Power Query. So armed with this knowledge is a popular starting point to load data into Microsoft Fabric. Adding values from the Variable Library in a Dataflow is an obvious plan to make it more future proof and to work better with Deployment pipelines.

I will confess the first time I tried these I could not get them to work till I read the instructions correctly. So they do work just understand the limitations!

To be fair, following instructions is one of the most challenging things to do, it seems.

Comments closed

Mirroring to OneLake without Public Internet Access

Published 2026-02-09 by Kevin Feasel

Paul Hernandez builds a (virtual) network:

Mirroring has been a transformative technology for data integrations tasks since the early Microsoft Fabric days. Moreover, this feature has been called “pain killer as a service” in community posts. In many projects, data sources to be mirrored are behind private networking and for security reasons they are not accessible using public internet. If you want to mirror, for example, an Azure SQL database, you’ll need a data gateway. According to the official docs: “If your Azure SQL Database is not publicly accessible and doesn’t allow Azure services to connect to it, you can set up virtual network data gateway or on-premises data gateway to mirror the data”.

In this post I’ll show you step-by-step how to set up connectivity to be able to use mirroring when Azure SQL allows only private access.

There are several steps involved, but the end result is worth it compared to not having the data at all or needing to make it accessible over the Internet.

Comments closed

SQL vs Azure Permissions

Published 2026-02-04 by Kevin Feasel

Rebecca Lewis continues a series on how Azure permissions and SQL Server (or Azure SQL Database) permissions are not the same thing:

Welcome to cloud permissions, where ‘Contributor’ doesn’t mean you can contribute and ‘Reader’ doesn’t mean you can read.

In my last post, I explained the management plane vs data plane split. This post is the promised follow-up for the minimum permission combinations for common DBA tasks. aka, what you need, how to verify it, and how to fix it when it fails.

I’m the guy pushing up my no-longer-existent glasses and saying “Well, actually…” to the first sentence, though stylistically, it’s a good one. But getting past the first sentence, there are some nice breakdowns of what it takes to do what you need to do on a cloud-hosted database.

2 Comments

Management Plane and Data Plane Permissions

Published 2026-01-30 by Kevin Feasel

Rebecca Lewis continues a series on cloud data platform security:

My last post ended with a promise to explain the management plane vs. data plane split in practical terms. The short story? There are two separate permission systems — one for managing resources, one for accessing data — and they don’t talk to each other. The good news: you already understand this model. You just don’t recognize it yet.

Click through for the explanation and a simile involving SQL Server’s security model.

Comments closed

Setting up Azure SQL Mirroring to Microsoft Fabric

Published 2026-01-28 by Kevin Feasel

Olivier Van Steenlandt troubleshoots an issue:

When setting up database mirroring from Azure SQL to Microsoft Fabric for one of my demo databases, I ran into an issue while trying to connect to my Azure SQL database.

As you can see in the screenshot above, it seems that a setting on my logical SQL Server in Azure is misconfigured. Let’s resolve that in a couple of steps.

Click through for the screenshot, the specific error, and how Olivier was able to get things working.

Comments closed