Press "Enter" to skip to content

Category: R

Ways to Impute Missing Data in R

The DataScience+ desk compares methods:

Missing data is the daily reality of medical and epidemiological work: a patient skips a lab draw, a questionnaire item is left blank. So in this tutorial I use real health-survey data and compare three current imputation packages on it: mice (multiple imputation, still the reference method for statistical inference), missRanger (random-forest imputation, the fast successor of missForest), and VIM (k-nearest-neighbour imputation). Before any imputing I use naniar, the modern, ggplot-native toolkit for missing data, to picture where the gaps are. I also include the method most analyses actually use by default, dropping incomplete rows, because seeing why it fails is half the point.

I start from complete records, delete values myself, and check each package against the truth I hid.

Click through for the method and different ways to impute data to fill in the gaps. The text of the article looks AI-generated, though the recommendations end up being pretty solid (at least in my experience with imputation and working with mice).

Leave a Comment

Solving the Jug Problem with Bezout’s Identity

Tomaz Kastrun shifts liquids:

With DFS we can solve this with Bezout’s identity; which in general is a theorem which relates two arbitraty integers with their greatests common divisor; and used in algebraic language, finding common zeros of n-polznomials in n-indeterminates. So the common zeros equals the product of the degrees of the polynomials.

The idea, to refresh your memory, is that you have three jugs of different sizes. Tomaz uses 16L, 11L, and 7L. Using just those, how can you get 8L into each of the two larger jugs? Click through for the answer.

Leave a Comment

TheseusPlot 0.3.0 Released

Koji Makiyama announces an update to an R package:

TheseusPlot is an R package that decomposes differences in a rate metric between two groups into subgroup-level contributions and visualizes the results as a “Theseus Plot”.

For example, when a click-through rate, conversion rate, or retention rate differs between two time periods or groups, TheseusPlot helps answer questions such as: which subgroup contributed most to the difference?

I love the name and I think the plot concept is interesting, especially in the e-commerce context that Koji describes. H/T R-Bloggers.

Comments closed

Snapshot Testing in R

Jakub Sobolewski drills into a particular form of testing:

Snapshot testing is not about screenshots.

Most people meet it through UI regression tests: render a component, save a picture, fail the build when the picture changes. So the technique gets filed away as “the thing that compares images.” That is one use. But not the only one.

The mechanic underneath is general. Capture some output, save it to a file, and on every later run compare fresh output against the saved copy. The output can be a plot. It can also be console text, a log, a data frame, an error message, or a deeply nested list. Anything you can serialize, you can snapshot.

Read on to see how you can perform snapshot testing, using examples in R to demonstrate. H/T R-Bloggers.

Comments closed

Noise in CRAN Package Additions

Joseph Rickert shows a consequence of lowering the bar for application development:

If you are reading this post on R-bloggers, you will probably know that I have been publishing my selection of the “Top 40” new R packages on CRAN for quite some time. I did this first as part of my work at Revolution Analytics, then on R Views for RStudio and Posit, and now here on R Works. It used to take about a day’s worth of pleasurable work spread out over a month to select forty interesting packages. For a hundred or so packages, I could look at all of the package webpages, download and play with a small number of them. Now, the “Top 40” has become a real hamster-on-the-wheel project. The following plot shows my count of the number of new packages to make it to CRAN since I began publishing on R Works.

Click through to see what Joseph has laid out. The part that surprises me is, historically, CRAN was pretty difficult to get a package into and you typically needed to jump through a certain number of quality gates. I suppose that has to have changed given what Joseph notes around the lack of documentation in many of these new packages.. But it could be that my understanding of it was wrong H/T R-Bloggers.

Comments closed

The Ulam Prime Spiral

Tomaz Kastrun re-creates a classic:

Stanislaw Ulam, Los Alamos, 1963 was bored in a meeting and he started dooddling integers in a spiral and circled the primes. Diagonal lines appeared. He later showed it to Martin Gardner, to Ulam surprise, Gardner published his findings in Scientific American. We are still confused to this day.

Click through for a demonstration of this in action.

Comments closed

Adding Patterns to ggplot2 Plots

Zhenguo Zhang adds some patterns:

Adding patterns to plots is a great way to improve accessibility (making plots colorblind-friendly) and to add an extra dimension of information. The ggpattern package provides a rich set of tools to achieve this in ggplot2.

I’m personally not the biggest fan of patterns. I see them as a point of necessity when dealing with grayscale circumstances, such as printing out a chart in an academic journal. But it’s very easy to overdo patterns and end up making a mess of the visual.

But one side note about color vision deficiency and plots: make sure that your plots are monochrome-friendly because somebody probably will try to print out your chart or view it on a grayscale-only device. Or might actually be monochromatic.

Comments closed

Probabilistic Time Series Cross-Validation in R

Thierry Moudiki checks an interval:

A previous post introduced the crossvalidation package for R. This time, the focus is on probabilistic forecasting — evaluating not just how accurate point forecasts are, but how well-calibrated prediction intervals are, using empirical coverage rates and Winkler scores – and crossvalidation.

Click through for the code and not much additional commentary. H/T R-Bloggers.

Comments closed