Press "Enter" to skip to content

Curated SQL Posts

Managing AI Context

Eugene Meidinger has a pair of articles. The first one covers the use of Markdown for building AI context:

When performing agentic development, most of your time will be spent either writing prompts or context files in Markdown (.md) files. This is a simple format used for documentation and notetaking, but which is also easy to read or write for humans and agents. You have almost certainly encountered Markdown before, but you may not have known it. Markdown is used by AI agents, in code documentation, and in some chat programs. Even Microsoft Teams supports a subset of Markdown.

Markdown is extremely common in more than just AI development work, so it’s definitely worth knowing.

The other is a guide to understanding and managing AI agent context:

The most important task for you during agentic development is to create, gather, and curate context. Context is anytext (or images) that the model ingests to produce a more relevant and useful response. Providing the right context at the right time helps agents produce better results; it’s about more than just writing markdown files:

Comments closed

Mirroring Data from SQL Server into Microsoft Fabric

Greg Low has a guide:

Mirroring in Microsoft Fabric is useful when you need to make SQL Server data available in Fabric. Perhaps in OneLake, for example, if you need to use the data for reporting, analytics, notebooks, or another Fabric workload.

However, it’s not suitable for everything, as I’ll explain in this complete guide to mirroring data from SQL Server to Microsoft Fabric.

Before we get started, a note about compatibility. Since SQL Server 2005, there’s been a type of database mirroring totally unrelated to Microsoft Fabric mirroring. I wish these types of terms weren’t so overloaded but, in this article, when I say mirroring, I’m talking only about Fabric mirroring.

Click through for a FAQ-style introduction.

Comments closed

Blind Spots and Troubleshooting a Cluster Problem

Alexander Arvidsson tells a story:

The patient was a misbehaving SQL Server 2014 running in a two-node cluster. It was your garden-variety cluster with a shared disk and a remote witness. In my experience, as long as the customer has enough know-how to maintain a cluster like this, it’s essentially bulletproof.

This specific cluster was not.

Click through for the story, as well as a reminder of why people in mission-critical situations tend to follow checklists and say the items aloud.

Comments closed

When Ints Overflow

Aaron Bertrand hits an outage in both directions:

The Stack Overflow database has a long lineage, and a lot more warts than what’s exposed in Stack Exchange Data Explorer (SEDE). Many of the core tables were created a decade before I joined the staff in 2021, and they grew far larger than could have been envisioned back then.

One of those tables is UserHistory. This table uses an identity column as the clustered index and primary key, and records all kinds of information about each user’s activity on the site. From changing your profile picture, to earning a new privilege, to changing a preference, to something as simple as logging in or out. Each event generates at least one new row in UserHistory. At peak popularity, this table would grow quite quickly and, as the application became more complex, more and more categories of activity and state would get written there.

Click through for the story of an outage, and then another outage with a very similar shape.

Comments closed

DBAs as Cost Centers

Warwick Rudd lays out the bitter truth:

That has been the problem for as long as there have been database administrators, and I don’t think we’ve ever really solved it.

Here’s what I mean. A DBA doing the job properly produces nothing you can see. The environment is up. The queries return. Nobody rings anybody at two in the morning. And because nothing appears to be happening, the people looking after it start to look like a line of cost with nothing attached to it.

In the traditional sense of how businesses think about this, they’re right. Nothing a DBA does makes money. What a DBA does is save money, which is a much harder thing to put in front of a board.

Warwick’s post moves in the direction of managed services. I’d move in the direction of understanding and vocalizing the benefits you bring to the table as a DBA.

Comments closed

Scatter and Sampling in Power BI Core vs Deneb

Kerry Kolosko shows some comparisons:

In almost a decade of building Power BI reports, a common blind spot I still see from colleagues regards data point limits in core visuals — largely unnoticed until someone maps their data and finds entire regions are missing.

Compare the following lasagne plot created using a Power BI core visual scatterplot with one created using a Power BI custom visual (Deneb).

Click through for the difference. The more I see Deneb, the more impressed I am with it. I’ve seen people do some wild things with it while still working just fine in Power BI. There’s even compatibility with Power BI Report Server, if you can believe it.

Comments closed

Dealing with Outages

Jeff Taylor tells a pair of stories:

It was a normal day at the start, checking on the servers, responding to email, then all of a sudden the office went dark and silent…we had just lost power!

Everyone started stirring and then getting up, checking that our phone system was on backup power, and someone called the power company to report it and get a status of when the power would be back on.

Extreme heat, smoke, and servers are not a great combination.

Comments closed

Implementing SCD Type 2 in Microsoft Fabric

Nikola Ilic reminds us that the Kimball model is alive and well:

Slowly Changing Dimensions (SCD) are one of the fundamental concepts in dimensional modeling. If you are not sure what dimensional modeling is, I suggest you first check the series of articles I wrote on data modeling some time ago.

And, when it comes to SCD in particular, SCD Type 2 is rightly considered “the queen” of the SCDs for analytical workloads. Without going into details (since the main goal of this article is to show you HOW to implement the SCD Type 2 in Microsoft Fabric), I’ll just briefly introduce the general concept behind the SCD Type 2.

Click through for a primer and examples in the warehouse and lakehouse.

Comments closed

Azure Databricks and Power BI Storage Modes

Chris Webb makes a choice:

In case you haven’t already seen the blog post on the Power BI blog or the discussion on LinkedIn, we at Microsoft published a new white paper last week to help you decide which storage mode to use when you’re using Power BI to create semantic models and reports on data stored in Azure Databricks. You can find the announcement and the link to the paper here.

Click through for Chris’s thoughts and check out the whitepaper.

Comments closed

Understanding the Receiver Operating Characteristic Curve

Ken Koon Wong digs into ROC and the area under the curve (AUC):

We see ROC-AUC so often with classification models, we know the higher the better, but there is always that, well it depends scenario. I’ve always wanted to know what the pitfall is, how to avoid it, and how to do better. Let’s go from the basics on how to code ROC-AUC from scratch to decision curve analysis!

Click through to learn how to calculate ROC and dig into the topic a bit. H/T R-Bloggers.

Comments closed