Press "Enter" to skip to content

Day: November 5, 2026

Writing MICE from Scratch in R

Published 2026-11-05 by Kevin Feasel

Ken Koon Wong rewrites a package:

We used mulitple imputation by chained equations (MICE) quite often in imputing missing data in many occasion, but I never really understood the mechanism behind it except that it appears magical! šŸ§™ I want to see if I can build this from scratch as a mean to understand the foundation of MICE so that at least it will give me the illusion of understanding. šŸ¤£ Join me in my messy journey of trying to code it from the ground up (FYI, Iā€™m not coding OLS or MLE from the ground upā€¦), and letā€™s see if we can create some ā€œmagicā€ here! This is going to be a bumpy one with lots of coding, and maybe less of mathematical formulae. Iā€™ll try my best to provide a TL;DR for each section so that we can get a high level view as well. Buckle up! And oh, Happy Halloween! šŸŽƒ What a better time to look at magical recipes of statistics! Letā€™s go!

I really like this post. The end result isn’t exactly what the mice implementation gives, but it’s close enough to show real understanding of the algorithm. H/T R-Bloggers.

Leave a Comment

Testing AI Agent Skills

Published 2026-11-05 by Kevin Feasel

Eugene Meidinger and Ruben Van de Voorde try things out:

Writing a skill takes time, and skills are context that can help but also hurt the quality of the agent’s work. So how do you know if a skill actually helps? You can test it with evals. An eval is a prompt that you re-run under the same conditions, so that the only variable is what you deliberately change. You use the same model, starting context, and the same files (and wipe leftover memory from previous attempts). Then you compare: before and after an edit, with and without a skill, or two versions of a skill. Agents are non-deterministic, so you have to run the comparison multiple times to get a reliable difference; a few runs per condition is enough to start. Decide how a run is scored before looking at the results: use objective checks where you can (did it run the right command?), and for subjective calls write down beforehand what a good answer must include. Also run in a clean workspace: if the solution is already in another branch, the baseline run isn’t a real test. Some companies run large automated eval suites, but a few saved prompts and some notes are enough to start.

Click through for a testing technique and a warning around relying on a generative model to do the work for you.

Leave a Comment

Fabric Capacity Units and F-SKUs

Published 2026-11-05 by Kevin Feasel

Paul Turley provides an explanation:

If thereā€™s one concept that trips up nearly everyone moving into Fabric, itā€™s capacity units ā€” because a CU isnā€™t what most people assume it is.

Itā€™s compute, not seats.

A Capacity Unit has nothing to do with user counts or license tiers. It measures compute power, consumed in CU seconds as work actually runs. A report rendering, a pipeline moving data, a notebook chewing through a dataframe ā€” all of it draws from the same shared pool of CU seconds on your capacity. Think of it less like a parking pass and more like a utility meter ā€” it just keeps ticking based on whatā€™s actually being computed, no matter whoā€™s asking for it.

Read on for more information around CUs, as well as what F-SKUs provide and when you might want either.

Leave a Comment

Thoughts on the Databricks Lakehouse

Published 2026-11-05 by Kevin Feasel

Miles Cole writes from inside the company:

The lakehouse architecture introduced a simple but powerful idea: unify the robust data management of the data warehouse with the openness, flexibility, and decoupled storage of the data lake. In principle, a lakehouse can be implemented anywhere, on any object store, with any compute engine that supports open table formats. In this post I want to make the case for why I believe the lakehouse is best realized on the platform that invented the product category in the first place: Databricks.

Iā€™ll make the case with three points:

  1. Robust data management comes standard
  2. Flexibility without the complexity
  3. A track record of innovation

Click through for the argument.

Leave a Comment

The Importance of Testing Backups

Published 2026-11-05 by Kevin Feasel

Marlon Ribunal lays out a case:

After sharing my thoughts on SQL Server native backups using Ola Hallengrenā€™s Maintenance Solution, Mark Gribler left a comment that caught my attention:

ā€œThe proof of a backup is a restoreā€ (apologies to puddings)!!

I had to smile at that one.

By the way, I give Marlon +1 point for getting the aphorism correct. The correct aphorism is “The proof of the pudding is in the eating.” Ponder that while checking those backups.

Leave a Comment

Hash Keys and Synapse Tables

Published 2026-11-05 by Kevin Feasel

Andy Brownsword explains the importance of selecting an appropriate hash key:

In Synapse SQL Pools, Hash distributions allow us to somewhat control data distribution. Whilst this can deliver performance in the right scenarios, I noted last week that these tables should be monitored to avoid skew.

In this post I want to demonstrate how to identify the skew, and why a different distribution model may be better suited.

One of the key tricks is that your choice of hash key should be approximately unique. There’s no explicit uniqueness constraint, but having a column with high cardinality and relatively little skew leads to a good distribution.

Leave a Comment