Category: Data Science

Steven Sanderson lays out some updates:

One of the standout additions is the introduction of util_log_ts(). This function seems like a game-changer, providing a streamlined way to log time series data. This is incredibly useful, especially when dealing with extensive datasets, making the whole process more efficient and user-friendly. This is a helper function for auto_stationarize().

There’s a lot in this update and the blog post also includes several examples of automating stationarity and ARIMA.

Comments closed

Steven Sanderson isn’t content with regressing against a single variable:

Multiple linear regression is a powerful statistical method that allows us to examine the relationship between a dependent variable and multiple independent variables.

Read on to see how you can do this in R, as well as some of the types of things you should think about along the way, including the concept of multicollinearity.

Comments closed

Steven Sanderson builds a prediction interval:

Prediction intervals are a powerful tool for understanding the uncertainty of your predictions. They allow you to specify a range of values within which you are confident that the true value will fall. This can be useful for many tasks, such as setting realistic goals, making informed decisions, and communicating your findings to others.

In this blog post, we will show you how to create a prediction interval in R using the mtcars dataset. The mtcars dataset is a built-in dataset in R that contains information about fuel economy, weight, displacement, and other characteristics of 32 cars.

Click through to see an example based on linear regression.

Comments closed

Michael Mayer lays some groundwork:

SHAP is the predominant way to interpret black-box ML models, especially for tree-based models with the blazingly fast TreeSHAP algorithm.

For general models, two slower SHAP algorithms exist:

1. Permutation SHAP (Štrumbelj and Kononenko, 2010)
2. Kernel SHAP (Lundberg and Lee, 2017)

Read on to understand more about these two forms of SHAP, as well as how they compare in two datasets of differing levels of difficulty.

Comments closed

Steven Sanderson shows off a new function:

If you’re an R enthusiast like me, you know that data manipulation is at the core of everything we do. The ability to transform your data swiftly and efficiently can make or break your data analysis projects. That’s why I’m thrilled to introduce a game-changing function in TidyDensity, my very own R library. Say hello to convert_to_ts()!

In the world of data analysis, time series data is like a treasure chest of insights waiting to be unlocked. Whether you’re tracking stock prices, monitoring patient data, or analyzing the temperature over the years, having your data in a time series format is a crucial step in the process. With convert_to_ts(), that process just got a whole lot easier.

Click through to see how it works and what you can do with it.

Comments closed

Bryan Shalloway explains how odds and probabilities intertwine:

However human understanding of odds predates our formal understanding of probability. You can find references to odds dating back to Shakespeare:

Knew that we ventured on such dangerous seas
That if we wrought out life ’twas ten to one;
– Shakespeare’s Henry IV, Part II, 1597

Yet, in most common settings, modern society has largely supplanted odds for probabilities. You can imagine if Shakespeare were writing today the line might end “’twas ten out of eleven.”

Read the whole thing.

Comments closed

Steven Sanderson tests a distribution fit:

There are two main ways to fit a gamma distribution to a dataset in R:

1. Maximum likelihood estimation (MLE): This method estimates the parameters of the gamma distribution that are most likely to have produced the observed data.
2. Method of moments: This method estimates the parameters of the gamma distribution by equating the sample mean and variance to the theoretical mean and variance of the gamma distribution.

Click through to see which technique Steven uses and an example of how it all works.

Comments closed

Steven Sanderson unleashes the power of the triangle:

The Triangular Distribution is a continuous probability distribution with a triangular shape, hence the name. It is defined by three parameters: min, max, and mode. These parameters determine the range of values the distribution can take and the most likely value within that range. In mathematical terms, the probability density function (PDF) of the Triangular Distribution is given by:

Read on to see the definition, as well as how you can use the four functions around the Triangular Distribution.

By the way, the best-known case of the Triangular Distribution is combining the results of two fair dice, which gives us a peak at the number 7 (1/6 of the time) for a pair of fair, six-sided dice and moving symmetrically down from there, so p(6) = p(8), p(5) = p(9), and so on.

Comments closed

Steven Sanderson isn’t satisfied with the binomial:

The multinomial distribution is a probability distribution that describes the probability of obtaining a specific number of counts for k different outcomes, when each outcome has a fixed probability of occurring.

In R, we can use the rmultinom() function to simulate random samples from a multinomial distribution, and the dmultinom() function to calculate the probability of a specific outcome.

Click through to see how you can build a multinomial distribution and what the difference is between rmultinom() and dmultinom().

Comments closed

Steven Sanderson digs into the uniform distribution:

Randomness is an essential part of many statistical and machine learning tasks. In R, there are a number of functions that can be used to generate random numbers, but the runif() function is the most commonly used.

Something mildly embarrassing for me is that it took me a while to figure out why they call the command runif(). That’s because, at first, I didn’t pronounce it r unif but rather run if.

In reality, *unif() means “uniform distribution” and r stands for “random number.” There are several other functions based on the uniform distribution and Steven looks at those as well in this post.

Comments closed