Press "Enter" to skip to content

Curated SQL Posts

Using the Azure Data Factory Self-Hosted Integration Runtime

Published 2023-07-14 by Kevin Feasel

Chen Hirsh hosts a runtime:

In Azure data factory (ADF), An integration runtime is a compute resource to run your pipelines on. When you run an application on your computer, it uses the computer resources, such as CPU and memory, to run its tasks. When you run activities in a pipeline in ADF, they also need resources to do their job, like copying data or writing a file, and these are provided by the integration runtime.

When you create an instance of ADF, you get a default integration runtime, hosted in the same region that you created ADF in. If you need, you can add your own integration runtimes, either on Azure, or you can download and install a self-hosted integration runtime (SHIR) on your own server.

Read on to understand when you would want to use a self-hosted integration runtime and the process to do so. This SHIR also applies to Synapse pipelines and is one of the few ways to move data out of a Synapse workspace with data exfiltration protection enabled.

Comments closed

Security Concepts: Who? and Where?

Published 2023-07-14 by Kevin Feasel

Kenneth Fisher is trying to figure out where he left his keys:

I was having a conversation with some friends the other day and Jen McCown (blog|twitter) asked about SQL Server security references and “What’s something that’s really difficult in SQL Server Security.” As happens sometimes I started thinking about this in the back of my head and I realized something. The two absolute hardest things that people run into with security (at least in my ever so humble opinion) are

  • Who: or Who am I when I try to access a resource?
  • Where: or Where am I when I try to access a resource? And Where is that resource?

Read on for Kenneth’s thoughts.

Comments closed

IS DISTINCT FROM in Snowflake

Published 2023-07-13 by Kevin Feasel

Kevin Wilkie remains distinct:

Now, the more fun – “new-ish” – version of the DISTINCT keyword.

Let’s take two values – A and B. Let’s define A = 7 and B = 2.

Snowflake will allow you to ask if A IS DISTINCT FROM B. Thankfully, in this case, it is.

Click through to see how this works. Also note that this syntax is available in SQL Server 2022.

Comments closed

Data Validation in Excel from a DAX Query

Published 2023-07-13 by Kevin Feasel

Gilbert Quevauvilliers builds a data validation tool in Excel:

I had a requirement where the customer wanted to make sure that the right user could only see the colour from the stock item assigned to them. Essentially using the Excel file for Row Level Security.

For the Row level security to work correctly it required the colour name to be an exact match.

For me to ensure that this would happen, I wanted to make sure I could get the data from my Power BI dataset, and then using the Excel data validation make sure that the user could either type in the value 100% correct or pick it from the drop-down list.

There are quite a few steps, but I can see business people being happy about the final outcome.

Comments closed

Creativity, Learning from Code, and the APPLY Operator

Published 2023-07-13 by Kevin Feasel

Rob Farley covers one of my favorite operators:

SQL Server 2005 was released, of course, in 2006, and I had been running the Adelaide SQL Server User Group since September 2005. Information about the new features had been coming through, and I’d been at TechEd Australia 2005 – my first since 1999. I was still an application developer at the time (well, a manager, but still getting my hands dirty), but saw the data as the most important part of my applications. When the T-SQL enhancements in SQL Server 2005 came through, there were two things that caught my eye (I know they were available in Oracle before SQL Server, but I was focusing more on the Microsoft platform by then). They were the OVER clause, and APPLY.

This is all part of a broader story about reading code to learn from it.

Comments closed

Users and Role Members for Azure SQL Databases

Published 2023-07-13 by Kevin Feasel

Peter Schott makes a list:

I ran into a concern to quickly audit all current users and role members for a set of Azure SQL databases, spread across multiple resource groups. Without an easy CMS concept or a way to quickly loop through an unknown set of servers, resource groups, and databases, that can be a little challenging. I have an account to use that should have access to all databases (but doesn’t) so put together some PowerShell that I could run locally to get that information and send the results to Excel.

Click through for a SQL script to get the data and a Powershell script to run this for each database and export the results into different tabs in Excel.

Comments closed

The Five Stages of Code Review Grief

Published 2023-07-13 by Kevin Feasel

Shane O’Neill has an experience:

You have code that keeps coming up as a high consumer of CPU on our systems? OK?

It had gotten to the stage where it kept appearing in sp_WhoIsActive? What’s that? And should it not be “whom is active”? Well, agree to disagree.

Let’s see the code so. Wow, that’s a small scroll bar! Yeah, that’s one of ours.

I should note that “Who is active?” is correct, as the sessions we are inquiring after are the subject of the question rather than the direct or indirect object, and there is no prepositional phrase which would affect the decision.

Comments closed

Shortcuts and Table Clones in Microsoft Fabric

Published 2023-07-13 by Kevin Feasel

Reitse Eskens takes a shortcut:

A few days ago, I heard the term table clone for the first time, it’s preview release date was to be confirmed and I had no idea what it was about. Two days later, a video emerged where the table clone was explained on a high level. At that point, I started to wonder what the differences are between a table clone and a shortcut. So let’s dig a little into that question!

First I’m going to create a shortcut, then a clone and finally compare the two.

Read on for the comparison, as these are quite different things.

Comments closed

Azure Stream Analytics No-Code Editor

Published 2023-07-12 by Kevin Feasel

Xu Jiang shows off a new designer:

Azure Stream Analytics is a fully managed stream processing engine designed to analyze and process large volumes of streaming data with sub-millisecond latencies. Using a SQL-like query language, it empowers you to analyze your streaming data efficiently. It only takes a few clicks to connect to multiple sources and sinks, creating a Stream Analytics job. 

The no-code editor offers an intuitive user experience that enables you to develop Stream Analytics jobs effortlessly, using drag-and-drop functionality, without having to write any code. It further simplifies Stream Analytics job development experience. With just a few clicks, you can quickly develop jobs to handle diverse scenarios in just minutes. It is available in the Azure Event Hubs portal, and now in Azure Stream Analytics portal as well.

Read on to see what it looks like and what you can do with it.

Comments closed