Changing the replication factor of an existing file in HDFS
In this recipe, we are going to take a look at how to change the replication factor of a file in HDFS. The default replication factor is 3.
Getting ready
To perform this recipe, you should already have a running Hadoop cluster.
How to do it...
Sometimes. there might be a need to increase or decrease the replication factor of a specific file in HDFS. In this case, we'll use the setrep command.
Once you list the file, it will show you the read/write permissions on this file, and the very next parameter is the replication factor. We have the replication factor set to 3 for our cluster, hence, you the number is 3.
Let's change it to 2 using this command:
hadoop fs -setrep -w 2 /mydir1/LICENSE.txt
It will wait till the replication is adjusted. Once done, you can verify this again by running the ls command:
Once the setrep command is executed, NameNode will be notified, and then NameNode decides whether the replicas need to be increased or decreased from certain DataNode. When you are using the –w command, sometimes, this process may take too long if the file size is too big.