Press "Enter" to skip to content

Category: Data Science

Writing MICE from Scratch in R

Published 2026-11-05 by Kevin Feasel

Ken Koon Wong rewrites a package:

We used mulitple imputation by chained equations (MICE) quite often in imputing missing data in many occasion, but I never really understood the mechanism behind it except that it appears magical! šŸ§™ I want to see if I can build this from scratch as a mean to understand the foundation of MICE so that at least it will give me the illusion of understanding. šŸ¤£ Join me in my messy journey of trying to code it from the ground up (FYI, Iā€™m not coding OLS or MLE from the ground upā€¦), and letā€™s see if we can create some ā€œmagicā€ here! This is going to be a bumpy one with lots of coding, and maybe less of mathematical formulae. Iā€™ll try my best to provide a TL;DR for each section so that we can get a high level view as well. Buckle up! And oh, Happy Halloween! šŸŽƒ What a better time to look at magical recipes of statistics! Letā€™s go!

I really like this post. The end result isn’t exactly what the mice implementation gives, but it’s close enough to show real understanding of the algorithm. H/T R-Bloggers.

Leave a Comment

Defining a Data Science Pit of Success

Published 2026-10-09 by Kevin Feasel

Bruno Rodrigues thinks about robustness in data science:

Rico Mariani, a long-time performance engineer at Microsoft, coined the idea of the pit of success. A system has a pit of success when the natural, lazy way of using it leads to good outcomes. You donā€™t need heroics or perfect memory. You fall into the right result, and climbing out to do something wrong takes deliberate effort.

I came across this framing in a talk that I was recently recommended and I further recommend it to anyone interested in the topic: Functional architecture – The pits of success – Mark Seemann. It put a name to something I had been circling around for years.

This is worth a careful read if you’re tied in with a data science team. H/T R-Bloggers.

Comments closed

Tuning DiskANN

Published 2026-09-16 by Kevin Feasel

Mala Mahadevan wraps up a three-parter on how DiskANN works:

In Part 1, we saw how ā€˜Vamanaā€™ represents vectors as nodes, connects them with edges, and uses greedy graph traversal to avoid comparing a query against every vector in the dataset. In Part 2, we investigated how PQ gave us compact representations for inexpensive distance calculations, caching kept useful graph nodes close, how SSDs provided capacity, and beam search allowed multiple promising nodes to be expanded together.

In this last part, we will be exploring how we use the parameters to tune DiskANN.

Click through to learn more about these parameters and what they do.

Comments closed

Understanding the Receiver Operating Characteristic Curve

Published 2026-09-04 by Kevin Feasel

Ken Koon Wong digs into ROC and the area under the curve (AUC):

We see ROC-AUC so often with classification models, we know the higher the better, but there is always that, well it depends scenario. Iā€™ve always wanted to know what the pitfall is, how to avoid it, and how to do better. Letā€™s go from the basics on how to code ROC-AUC from scratch to decision curve analysis!

Click through to learn how to calculate ROC and dig into the topic a bit. H/T R-Bloggers.

Comments closed

Runge-Kutta in SQL Server

Published 2026-09-01 by Kevin Feasel

Sebastiao Pereira implements a formula:

Runge-Kutta method it is an effective numerical interactive technique used to solve ordinary differential equations, used in physics, engineering, aerospace, ballistics, circuit simulation, epidemiology, chemical kinetics, games, and others. Is it possible to create in SQL Server without the use of external tools?

Click through for the solution.

Comments closed

Understanding the DiskANN Algorithm

Published 2026-08-28 by Kevin Feasel

Mala Mahadevan digs into an algorithm:

DiskANN is meant to help with searching a billion-vector dataset from a single machine using SSDs.

Microsoft Researchā€™s original work demonstrated a billion-point index on a workstation with 64 GB of RAM and an SSD, while targeting high recall and low query latency. Before we get to ā€˜why ssdā€™ and details of storage, we need to understand the basics of vector search and underlying terminologies.

Click through for a depiction of the paper and quite a few examples of how the mechanisms work.

Comments closed

Implementing the Gauss Kronod Quadrature Formula in SQL Server

Published 2026-08-07 by Kevin Feasel

Sebastiao Pereira implements a formula:

Gauss-Kronrod quadrature is a numerical integration method that extends the Gaussian quadrature providing high accuracy and a built-in error estimation. It is based on the work of Carl Friedrich Gauss and Alexander Kronrod. The Kronrod method reuses all Gauss nodes adding extra points obtaining one integral estimate for Gauss and another for Kronrod and the differences gives the estimated error.

Click through to see how you can do this in T-SQL.

Comments closed

GPopt for R

Published 2026-07-30 by Kevin Feasel

Thierry Moudiki looks at an R port of a Python package:

Keep in mind that this package is for Machine Learning hyperparameter tuning: the global minimum wonā€™t always be found, but this isnā€™t an issue, since it means you arenā€™t overfitting the training set.

Itā€™s ported the same way as nnetsauce for R was: with uv to create an isolated Python virtual environment containing the Python GPopt package, and reticulate to call into it from R. Every function in this R package is a thin wrapper that returns the underlying Python object; the general rule is: object accesses with .ā€™s in Python are replaced by $ā€™s in R.

Click through for the instructions and examples of how it works. H/T R-Bloggers.

Comments closed