Press "Enter" to skip to content

Curated SQL Posts

Writing MICE from Scratch in R

Published 2026-11-05 by Kevin Feasel

Ken Koon Wong rewrites a package:

We used mulitple imputation by chained equations (MICE) quite often in imputing missing data in many occasion, but I never really understood the mechanism behind it except that it appears magical! šŸ§™ I want to see if I can build this from scratch as a mean to understand the foundation of MICE so that at least it will give me the illusion of understanding. šŸ¤£ Join me in my messy journey of trying to code it from the ground up (FYI, Iā€™m not coding OLS or MLE from the ground upā€¦), and letā€™s see if we can create some ā€œmagicā€ here! This is going to be a bumpy one with lots of coding, and maybe less of mathematical formulae. Iā€™ll try my best to provide a TL;DR for each section so that we can get a high level view as well. Buckle up! And oh, Happy Halloween! šŸŽƒ What a better time to look at magical recipes of statistics! Letā€™s go!

I really like this post. The end result isn’t exactly what the mice implementation gives, but it’s close enough to show real understanding of the algorithm. H/T R-Bloggers.

Leave a Comment

Testing AI Agent Skills

Published 2026-11-05 by Kevin Feasel

Eugene Meidinger and Ruben Van de Voorde try things out:

Writing a skill takes time, and skills are context that can help but also hurt the quality of the agent’s work. So how do you know if a skill actually helps? You can test it with evals. An eval is a prompt that you re-run under the same conditions, so that the only variable is what you deliberately change. You use the same model, starting context, and the same files (and wipe leftover memory from previous attempts). Then you compare: before and after an edit, with and without a skill, or two versions of a skill. Agents are non-deterministic, so you have to run the comparison multiple times to get a reliable difference; a few runs per condition is enough to start. Decide how a run is scored before looking at the results: use objective checks where you can (did it run the right command?), and for subjective calls write down beforehand what a good answer must include. Also run in a clean workspace: if the solution is already in another branch, the baseline run isn’t a real test. Some companies run large automated eval suites, but a few saved prompts and some notes are enough to start.

Click through for a testing technique and a warning around relying on a generative model to do the work for you.

Leave a Comment

Fabric Capacity Units and F-SKUs

Published 2026-11-05 by Kevin Feasel

Paul Turley provides an explanation:

If thereā€™s one concept that trips up nearly everyone moving into Fabric, itā€™s capacity units ā€” because a CU isnā€™t what most people assume it is.

Itā€™s compute, not seats.

A Capacity Unit has nothing to do with user counts or license tiers. It measures compute power, consumed in CU seconds as work actually runs. A report rendering, a pipeline moving data, a notebook chewing through a dataframe ā€” all of it draws from the same shared pool of CU seconds on your capacity. Think of it less like a parking pass and more like a utility meter ā€” it just keeps ticking based on whatā€™s actually being computed, no matter whoā€™s asking for it.

Read on for more information around CUs, as well as what F-SKUs provide and when you might want either.

Leave a Comment

Thoughts on the Databricks Lakehouse

Published 2026-11-05 by Kevin Feasel

Miles Cole writes from inside the company:

The lakehouse architecture introduced a simple but powerful idea: unify the robust data management of the data warehouse with the openness, flexibility, and decoupled storage of the data lake. In principle, a lakehouse can be implemented anywhere, on any object store, with any compute engine that supports open table formats. In this post I want to make the case for why I believe the lakehouse is best realized on the platform that invented the product category in the first place: Databricks.

Iā€™ll make the case with three points:

  1. Robust data management comes standard
  2. Flexibility without the complexity
  3. A track record of innovation

Click through for the argument.

Leave a Comment

The Importance of Testing Backups

Published 2026-11-05 by Kevin Feasel

Marlon Ribunal lays out a case:

After sharing my thoughts on SQL Server native backups using Ola Hallengrenā€™s Maintenance Solution, Mark Gribler left a comment that caught my attention:

ā€œThe proof of a backup is a restoreā€ (apologies to puddings)!!

I had to smile at that one.

By the way, I give Marlon +1 point for getting the aphorism correct. The correct aphorism is “The proof of the pudding is in the eating.” Ponder that while checking those backups.

Leave a Comment

Hash Keys and Synapse Tables

Published 2026-11-05 by Kevin Feasel

Andy Brownsword explains the importance of selecting an appropriate hash key:

In Synapse SQL Pools, Hash distributions allow us to somewhat control data distribution. Whilst this can deliver performance in the right scenarios, I noted last week that these tables should be monitored to avoid skew.

In this post I want to demonstrate how to identify the skew, and why a different distribution model may be better suited.

One of the key tricks is that your choice of hash key should be approximately unique. There’s no explicit uniqueness constraint, but having a column with high cardinality and relatively little skew leads to a good distribution.

Leave a Comment

Space-Filling Curves to Display Battles over Time

Published 2026-11-04 by Kevin Feasel

Richard Vale plots historical battles:

I recently learned about an interesting way of reducing the dimension of a data set. Iā€™d like to illustrate it with a data set of about 550 historical battles in the UK, extracted from Wikidata using a SPARQL query. No doubt some readers will have seen this idea before, but itā€™s new to me and I found it pretty.

The following plot shows the raw data, with a crossed swords symbol for the location of each battle. Itā€™s possible to see some patterns on the plot (for example, it probably wasnā€™t much fun to live on the borders of mediaeval Wales) but it could be clearer.

Click through to learn about Hilbert curves and how we can use them to map 2D information to a 1D space. I’d love to see the repro code for this, but Richard does explain the technique. H/T R-Bloggers.

Leave a Comment

Getting Visual Basic 6.0 to Run on Windows 11

Published 2026-11-04 by Kevin Feasel

Randolph West has a hobby:

In 1999 I released my first commercial software product. ShutOff 2000 was a Windows application that could log off or shut down Windows in multiple ways. It was successful enough that a serial number generator was produced by at least two hacker collectives. A badge of honour, really.

So now that Iā€™m retired, I thought Iā€™d go back and look at how different life is from the last release, which was as recent as 2014.

And then I realized I needed to install Visual Basic 6.0 on my 64-bit Windows 11 virtual machine.

This goes about as well as you’d expect, except that, by the end of it, Randolph actually has a working solution.

Leave a Comment

DML in Postgres

Published 2026-11-04 by Kevin Feasel

Grant Fritchey tries out some syntax:

Once your PostgreSQL tables are built, the next step is putting them to work. This guide walks through the core data operations every PostgreSQL user needs: inserting records with INSERT, retrieving them with SELECT, filtering and combining tables with WHERE and JOIN, and modifying or removing data with UPDATE and DELETE.

Youā€™ll also learn how PostgreSQL handles IDENTITY columns, sequences, aliases, and referential integrity.

Click through for Grant’s notes.

Leave a Comment

AI Functions in the Fabric Warehouse

Published 2026-11-04 by Kevin Feasel

Koen Verbeeck shows off a few functions:

We are building a data platform using Microsoft Fabric and we are using the warehouse as the main database engine to store our data. We need to process unstructured data coming from various sources, such as logs, free-text fields in user forms, reviews etc. Our team is quite proficient in T-SQL, but the native T-SQL functions we know from SQL Server are limited in their ability to handle the unstructured text. Ideally, we would parse the data using AI tools, but we donā€™t know how to do this from the warehouse. What are our options?

Click through for the list. Based on Koen’s example, the AI_FIX_GRAMMAR one works about as well as Word’s grammar checker, which means that it does a decent job of fixing spelling but is awful for actual grammar.

Leave a Comment