Yitao Li announces sparklyr 1.3:
sparklyr1.3 is now available on CRAN, with the following major new features:– Higher-order Functions to easily manipulate arrays and structs
– Support for Apache Avro, a row-oriented data serialization framework
– Custom Serialization using R functions to read and write any data format
– Other Improvements such as compatibility with EMR 6.0 & Spark 3.0, and initial support for Flint time series library
Between this and the work from the Spark side, we are seeing some nice quality of life improvements for Spark and R.