Amit Kulkarni shows how to make Azure Data Lake Store the default file system for a Hadoop cluster:
So to give a concrete example, if the default file system was
hdfs://123.23.12.4344:9000
then the/user/filename.txt
would resolve tohdfs://123.23.12.4344:9000/user/filename.txt
.Why does the default file system matter? The first answer to this is purely convenience. It is a heck lot easier to simply say
/events/sensor1/
thanadl://amitadls.azuredatalakestore.net/
in code and configurations. Secondly, many components in Hadoop use relative paths by default. For instance there are a fixed set of places, specified by relative paths, where various applications generate their log files. Finally, many ISV applications running on Hadoop specify important locations by relative paths.
Read on to see how.