A Fine Slice Of SQL Server
Steven Sanderson makes use of data.table:
I’m thrilled to announce a major upgrade to the TidyDensity package that’s sure to accelerate your data analysis workflows. We’ve integrated the lightning-fast
data.tablepackage for generating tidy distribution data, resulting in a jaw-dropping 30% speed boost.
The data.table package is so much faster than its competition in so many cases, yet I really don’t like its syntax.
Comments closedPeter Laurinec gives us an overview:
Clustering is a very popular technique in data science because of its unsupervised characteristic – we don’t need true labels of groups in data. In this blog post, I will give you a “quick” survey of various clustering methods applied to synthetic but also real datasets.
Read on for a quick description of what clustering is and a few use cases. Then, Peter dives into a variety of techniques and important things you should know about them. H/T R-Bloggers.
Comments closedSteven Sanderson charts performance:
Statistical analysis often involves calculating various measures on large datasets. Speed and efficiency are crucial, especially when dealing with real-time analytics or massive data volumes. The TidyDensity package in R provides a set of fast cumulative functions for common statistical measures like mean, standard deviation, skewness, and kurtosis. But just how fast are these cumulative functions compared to doing the computations directly? In this post, I benchmark the cumulative functions against the base R implementations using the rbenchmark package.
Click through for the functions under test and how they fare.
Comments closedSteven Sanderson unleashes the power of the triangle:
Welcome back, fellow data enthusiasts! Today, we embark on an exciting journey into the world of statistical distributions with a special focus on the latest addition to the TidyDensity package – the triangular distribution. Tightly packed and versatile, this distribution brings a unique flavor to your data simulations and analyses. In this blog post, we’ll delve into the functions provided, understand their arguments, and explore the wonders of the triangular distribution.
Read on to learn what the triangular distribution is and how you can use work with it in TidyDensity.
Comments closedSteven Sanderson has an update to the TidyDensity package:
The latest release of the
TidyDensityR package brings some major changes and improvements that open up new possibilities for statistical analysis and data visualization. Version 1.3.0 includes breaking changes, new features, and a host of minor fixes and improvements that enhance performance and usability. Let’s dive into what’s new!
Read on for that change list and how you can get a copy of the TidyDensity R package.
Steven Sanderson groups by month and year:
Taming the beast of daily data can be daunting. While it captures every detail, sometimes you need a bird’s-eye view. Enter aggregation, your secret weapon for transforming daily data into monthly and yearly insights. In this post, we’ll dive into the world of R, where you’ll wield powerful tools like
dplyrandlubridateto master this data wrangling art.
Click through for examples of summarizing daily data into monthly and annual data. One thing to keep in mind, however, is that the monthly aggregation in these examples is just month, so if you have July 2023 and July 2024 data, you’ll get a row back for July. It’s all about understanding what the grain of your data is, as well as your desired grain.
Comments closedMichael Mayer has some ‘splainin to do:
Let’s explain a {tidymodels} random forest by classic explainability methods (permutation importance, partial dependence plots (PDP), Friedman’s H statistics), and also fancy SHAP.
Disclaimer: {hstats}, {kernelshap} and {shapviz} are three of my own packages.
What I really appreciate in here is that Michael includes classic methods here. It can be easy to say “Oh, this is old and therefore no longer relevant.” But that would be quite wrong.
Comments closedSteven Sanderson performs a moving average:
Ever felt those data points were a bit too jittery? Smoothing out trends and revealing underlying patterns is a breeze with rolling averages in R. Ready to roll? Let’s dive in!
Read on to see one way to do this in R.
Comments closedSteven Sanderson had me thinking of LOESS but then, bam!, snuck this in on me:
Locally Weighted Scatterplot Smoothing, or Lowess, is a powerful technique for capturing trends in noisy data. It’s particularly useful when dealing with datasets that exhibit complex patterns that might be missed by other methods. So, let’s get our hands dirty and start coding!
Read on for an example of LOWESS smoothing, which actually is a little different from LOESS. If you’re interested in learning more about the differences between LOESS and LOWESS, this Stack Exchange question and answer page is really good.
Comments closedJohn MacKintosh has eyes for .I:
In this post I’m using a small extract from the SIMD2020 dataset to figure out what the special operator
.Idoes.Files and code are on github if you’re interested
Click through for a demo-laden post. H/T R-Bloggers.
Comments closed