Press "Enter" to skip to content

Curated SQL Posts

Space-Filling Curves to Display Battles over Time

Richard Vale plots historical battles:

I recently learned about an interesting way of reducing the dimension of a data set. I’d like to illustrate it with a data set of about 550 historical battles in the UK, extracted from Wikidata using a SPARQL query. No doubt some readers will have seen this idea before, but it’s new to me and I found it pretty.

The following plot shows the raw data, with a crossed swords symbol for the location of each battle. It’s possible to see some patterns on the plot (for example, it probably wasn’t much fun to live on the borders of mediaeval Wales) but it could be clearer.

Click through to learn about Hilbert curves and how we can use them to map 2D information to a 1D space. I’d love to see the repro code for this, but Richard does explain the technique. H/T R-Bloggers.

Leave a Comment

Getting Visual Basic 6.0 to Run on Windows 11

Randolph West has a hobby:

In 1999 I released my first commercial software product. ShutOff 2000 was a Windows application that could log off or shut down Windows in multiple ways. It was successful enough that a serial number generator was produced by at least two hacker collectives. A badge of honour, really.

So now that I’m retired, I thought I’d go back and look at how different life is from the last release, which was as recent as 2014.

And then I realized I needed to install Visual Basic 6.0 on my 64-bit Windows 11 virtual machine.

This goes about as well as you’d expect, except that, by the end of it, Randolph actually has a working solution.

Leave a Comment

DML in Postgres

Grant Fritchey tries out some syntax:

Once your PostgreSQL tables are built, the next step is putting them to work. This guide walks through the core data operations every PostgreSQL user needs: inserting records with INSERT, retrieving them with SELECT, filtering and combining tables with WHERE and JOIN, and modifying or removing data with UPDATE and DELETE.

You’ll also learn how PostgreSQL handles IDENTITY columns, sequences, aliases, and referential integrity.

Click through for Grant’s notes.

Leave a Comment

AI Functions in the Fabric Warehouse

Koen Verbeeck shows off a few functions:

We are building a data platform using Microsoft Fabric and we are using the warehouse as the main database engine to store our data. We need to process unstructured data coming from various sources, such as logs, free-text fields in user forms, reviews etc. Our team is quite proficient in T-SQL, but the native T-SQL functions we know from SQL Server are limited in their ability to handle the unstructured text. Ideally, we would parse the data using AI tools, but we don’t know how to do this from the warehouse. What are our options?

Click through for the list. Based on Koen’s example, the AI_FIX_GRAMMAR one works about as well as Word’s grammar checker, which means that it does a decent job of fixing spelling but is awful for actual grammar.

Leave a Comment

The Pain of Azure Defender Billing

Steve Jones walks through a painful experience:

I hadn’t done much in the last couple of months on Azure. For some reason, I decided to check my bill. What I found was that there were charges for $35-ish/month. This is within my free credit limit as an MVP, but that seemed high. I have a few serverless dbs and some storage, but I was expecting more of a $15 charge.

Azure billing is probably 85% of what an ideal billing solution would look like. There’s a lot of detail and you can get the bill down to a pretty granular level. But the problem is, like Steve found out here, that it’s really hard to tie back individual line items to specific services in a consistent way. Sometimes it’s because of ancillary costs like networking bandwidth. Sometimes, it’s a problem where you can’t easily see what specific resource is costing you money. Either way, it turns a proper audit of the bill into a bit of an ordeal.

Leave a Comment

Truncating a Terabyte-Sized Table in a SQL Server Availability Group

Josh Darnell performs a test:

Years ago, I read a Q&A about the consequences (in terms of latency, log backups, etc.) of truncating a table with billions of rows in an Availability Group. I even wrote my own A to the Q with some measurements, although there was no AG involved in my test. Anyway, I was recently reminded of all that, and realized that AI might make it a lot easier to just test and see what happens.

Read on to see what Josh figured out.

Leave a Comment

sp_CheckActivity

Andrew Kelly announces a new version of an old tool:

I initially created the first version of this utility many years ago as an alternative to the immensely popular tool developed by Adam Machanic called sp_WhoIsActive. I wanted something that would suit most of my everyday needs for a snapshot type of view of the current activity on a SQL Server instance.

I know what you are thinking: why couldn’t I just use sp_WhoIsActive? 

Well, I have used it for many years and in some cases still do when a special need arises. The main motivator for creating sp_CheckActivity was not for lack of features but…it’s more like there can be too many.

Click through for more information about sp_CheckActivity and grab a copy from the GitHub repo if you’re so inclined.

Leave a Comment

Using Batch Mode on Rowstore Indexes in SQL Server

Mehdi Ghapanvari speeds things up:

SQL Server columnstore indexes can improve the performance of aggregation queries. However, there are many cases where they are not efficient. I discussed one of these scenarios in “When Using a Clustered Columnstore Index Does Not Make Sense.” In that article, I discussed that columnstore indexes have three main features, one of which is Batch Mode Processing. SQL Server 2019 introduced a new feature called Batch Mode on Rowstore, which enables batch mode processing on Rowstore tables. In this article, I will explore this feature and explain why it is valuable in real-world scenarios.

In Mehdi’s solution, the question comes up of “Who would run an aggregation query on an OLTP system?” The answer is “everybody.” I’ve never seen an OTLP system with zero aggregation work. Perhaps these systems really do exist but some level of hybrid operation is the default that I’ve experienced.

Leave a Comment

Installing SQL Server on Linux

Dualcore DBA performs an installation:

SQL Server on Linux – one of those things that you can’t believe actually exists – two worlds you never thought would collide. For years, Microsoft and Linux were very much apart, until the announcement in 2015 that Microsoft Loves Linux. SQL Server on Linux was launched in 2017 and whilst I won’t get into the differences between the closed and open source philosophies, being a fan of both Microsoft SQL Server and Linux, I felt like I had to see what SQL Server on Linux was like.

With this in mind, let’s walk through the install process.

I’ve found that I use containerized SQL Server way more often than directly installing it on Linux. But either way, I’m still glad that it’s possible.

Leave a Comment

Optimizing Power BI Semantic Models via Column Splitting

Phil Seamark provides some advice:

Column splitting is an idea worth testing when you are looking to reduce the size of a large semantic model to potentially reduce the SKU required to host the model. The good news is, the technique can also benefit the time to load data into a model and speed up your DAX queries.

Instead of storing each complete value in one column, split the column vertically and store it in smaller pieces. Then use a DAX measure to combine the pieces back together. Those pieces often repeat more frequently, which can help Power BI compress them.

What’s interesting is how similar this approach is to financial systems from back in the day, where all money was stored as two separate integers. In that case, it was for the sake of precision and correctness rather than performance, but it’s nice to see these techniques still kicking.

Leave a Comment