Anubhav Tarar has a word count app for Spark 2.0:
Now you have to perform the given steps:
-
Create a spark session from org.apache.spark.sql.sparksession api and specify your master and app name
-
Using the sparksession.read.txt method, read from the file wordcount.txt the return value of this method in a dataset. In case you don’t know what a data set looks like you can learn from this link.
-
Split this dataset of type string with white space and create a map which contains the occurence of each word in that data set.
-
Create a class prettyPrintMap for printing the result to console.
This Hello World app is a bit longer than the sheer minimum code necessary, as it includes a class for formatting results and some error handling.