Kevin Feasel


Hadoop, R, Spark

John Mount shows off replyr, which is dplyr for remote, distributed data sets (think SparkR or sparklyr):

Suppose we had a large data set hosted on a Spark cluster that we wished to work with using dplyr and sparklyr (for this article we will simulate such using data loaded into Spark from the nycflights13 package).

We will work a trivial example: taking a quick peek at your data. The analyst should always be able to and willing to look at the data.

It is easy to look at the top of the data, or any specific set of rows of the data.

Read on for more details.

Related Posts

R Data Frames And stringsAsFactors

John Mount recommends setting stringsAsFactors = FALSE for data frames in R: R often uses a concept of factors to re-encode strings. This can be too early and too aggressive. Sometimes a string is just a string. Tibbles have this set by default.  For an explanation as to why it defaults to TRUE for data frames, Roger […]

Read More

Cassandra To Kafka Connect

Mike Barlotta shows how to feed data into Kafka from Cassandra via Kafka Connect.  Part one involves basic setup: Modeling data in Cassandra must be done around the queries that are needed to access the data (see this article for details). Typically this means that there will be one table for each query and data (in our […]

Read More


March 2017
« Feb Apr »