A Fine Slice Of SQL Server
Steven Sanderson performs feature selection:
As an R programmer, one of the fundamental tasks you’ll encounter is manipulating data frames. Whether you’re cleaning messy data or preparing it for analysis, knowing how to drop unnecessary columns is a valuable skill. In this guide, we’ll walk through the process of dropping columns from data frames in R, using simple examples to demystify the process.
Read on for three ways of doing this.
Leave a CommentSteven Sanderson searches for subsets:
In data analysis, there often arises a need to extract the top N values within each group of a dataset. Whether you’re dealing with sales data, survey responses, or any other type of grouped data, identifying the top performers or outliers within each group can provide valuable insights. In this tutorial, we’ll explore how to accomplish this task using three popular R packages: dplyr, data.table, and base R. By the end of this guide, you’ll have a solid understanding of various approaches to selecting top N values by group in R.
Read on for the three examples.
Leave a CommentRuss Hyde shares items of interest:
R 4.4.0 (“Puppy Cup”) was released on the 24th April 2024 and it is a beauty. In time-honoured tradition, here we summarise some of the changes that caught our eyes. R 4.4.0 introduces some cool features (one of which is experimental) and makes one of our favourite {rlang} operators available in base R. There are a few things you might need to be aware of regarding handling
NULLandcomplexvalues.The full changelog can be found at the r-release ‘NEWS’ page and if you want to keep up to date with developments in base R, have a look at the r-devel ‘NEWS’ page.
Read on for a big note on tail-call recursion, an operator for coalescing, and a few more neat features.
Leave a CommentSteven Sanderson wants to check for row existence:
Hello, fellow R users! Today, we’re going to explore a common scenario you might encounter when working with data frames: checking if a row from one data frame exists in another. This is a handy skill that can help you compare datasets and verify data integrity.
There are several ways to do it. Click through for two of those ways.
Leave a CommentSteven Sanderson takes a slice:
Explanation: – We use
nrow(my_df)to get the total number of rows in the data frame. – Then, we use indexing ([nrow(my_df), ]) to extract the last row.
Read on for a simple example of getting the last row using base R, dplyr, and data.table. Then, we kick it up a notch to get the second-to-last row, with the idea being that you could substitute “second-to-last” with any arbitrary number.
Leave a CommentKeith Goldfield updates the simstudy package:
A researcher reached out to me a few weeks ago. They were trying to generate longitudinal data that included irregularly spaced follow-up periods. The default periods generated by the function
addPeriodsin thesimstudypackage are {0,1,2,…,n−1}{0,1,2,…,n−1}, where there are n total periods. However, when follow-up periods required more specificity, such as {0,90,180,365}{0,90,180,365} days from baseline, users had to manually add them. Originally, I had intended to incorporate this feature into the function, but unfortunately it slipped through the cracks. Thanks to the clear motivation provided by the researcher, I’ve implemented this enhancement. Users can now replace the default vector with their desired set of follow-up periods using the new argument periodVec. This addition is available in the development version ofsimstudyon GitHub.
Read on to see how it works. H/T R-Bloggers.
Leave a CommentSteven Sanderson performs a test:
In the world of statistics and data analysis, understanding and accurately estimating the parameters of probability distributions is crucial. One such distribution is the chi-square distribution, often encountered in various statistical analyses. In this blog post, we’ll dive into how we can estimate the degrees of freedom (“df”) and the non-centrality parameter (“ncp”) of a chi-square distribution using R programming language.
Read on to learn more about the process of estimation while I grumble something about Bayesian analysis being better.
Leave a CommentSteven Sanderson now has two problems:
Regular expressions, or regex, are incredibly powerful tools for pattern matching and extracting specific information from text data. Today, we’ll explore how to harness the might of regex in R with a practical example.
Let’s dive into a scenario where we have data that needs cleaning and extracting numerical values from strings. Our data, stored in a dataframe named
df, consists of four columns (x1,x2,x3,x4) with strings containing numerical values along with percentage values enclosed in parentheses. Our goal is to extract these numerical values and compute a total for each row.
Click through for a worked-out example.
Leave a CommentSteven Sanderson grabs some rows:
Imagine you want to analyze the fuel efficiency (miles per gallon) of a particular car. Here’s how to grab a single row by its index (row number):
Read on for that example, as well as examples covering two other cases: multiple rows based on index value, and ranges of indices.
Leave a CommentArt Steinmetz runs a comparison:
As we start working with larger and larger datasets, the basic tools of the tidyverse start to look a little slow. In the last few years several packages more suited to large datasets have emerged. Some of these are column, rather than row, oriented. Some use parallel processing. Some are vector optimized. Speedy databases that have made their way into the R ecosystem are data.table, arrow, polars and duckdb. All of these are available for Python as well. Each of these carries with it its own interface and learning curve. duckdb, for example is a dialect of SQL, an entirely different language so our dplyr code above has to look like this in SQL:
Read on for a detailed comparison. Your mileage may vary, etc., but I’m pleasantly surprised with the results, given that I like the Tidyverse for its ease of use compared to base R and other alternatives like raw data.table. H/T R-Bloggers.
Leave a Comment