- Hadoop Real-World Solutions Cookbook(Second Edition)
- Tanmay Deshpande
- 218字
- 2021-07-09 20:02:50
Exporting HDFS data to a local machine
In this recipe, we are going to export/copy data from HDFS to the local machine.
Getting ready
To perform this recipe, you should already have a running Hadoop cluster.
How to do it...
Performing this recipe is as simple as copying data from one folder to the other. There are a couple of ways in which you can export data from HDFS to the local machine.
- Using the
copyToLocal
command, you'll get this code:hadoop fs -copyToLocal /mydir1/LICENSE.txt /home/ubuntu
- Using the
get
command, you'll get this code:hadoop fs -get/mydir1/LICENSE.txt /home/ubuntu
How it works...
When you use HDFS copyToLocal
or the get
command, the following things occur:
- First of all, the client contacts
NameNode
because it needs a specific file in HDFS. NameNode
then checks whether such a file exists in itsFSImage
. If the file is not present, the error code is returned to the client.- If the file exists,
NameNode
checks the metadata for blocks and replica placements inDataNodes
. NameNode
then directly pointsDataNode
from where the blocks would be given to the client one by one. The data is directly copied fromDataNode
to the client machine. and it never goes throughNameNode
to avoid bottlenecks.- Thus, the file is exported to the local machine from HDFS.
推薦閱讀
- 計算機應用復習與練習
- 影視后期制作(Avid Media Composer 5.0)
- C語言開發(fā)技術詳解
- Blender Compositing and Post Processing
- JavaScript典型應用與最佳實踐
- Building a BeagleBone Black Super Cluster
- Linux內核精析
- 寒江獨釣:Windows內核安全編程
- 未來學徒:讀懂人工智能飛馳時代
- 電腦故障排除與維護終極技巧金典
- Unreal Development Kit Game Design Cookbook
- Hadoop Beginner's Guide
- 傳感技術基礎與技能實訓
- SQL Server 2019 Administrator's Guide
- Linux Administration Cookbook