- Hadoop Real-World Solutions Cookbook(Second Edition)
- Tanmay Deshpande
- 218字
- 2021-07-09 20:02:50
Exporting HDFS data to a local machine
In this recipe, we are going to export/copy data from HDFS to the local machine.
Getting ready
To perform this recipe, you should already have a running Hadoop cluster.
How to do it...
Performing this recipe is as simple as copying data from one folder to the other. There are a couple of ways in which you can export data from HDFS to the local machine.
- Using the
copyToLocal
command, you'll get this code:hadoop fs -copyToLocal /mydir1/LICENSE.txt /home/ubuntu
- Using the
get
command, you'll get this code:hadoop fs -get/mydir1/LICENSE.txt /home/ubuntu
How it works...
When you use HDFS copyToLocal
or the get
command, the following things occur:
- First of all, the client contacts
NameNode
because it needs a specific file in HDFS. NameNode
then checks whether such a file exists in itsFSImage
. If the file is not present, the error code is returned to the client.- If the file exists,
NameNode
checks the metadata for blocks and replica placements inDataNodes
. NameNode
then directly pointsDataNode
from where the blocks would be given to the client one by one. The data is directly copied fromDataNode
to the client machine. and it never goes throughNameNode
to avoid bottlenecks.- Thus, the file is exported to the local machine from HDFS.
推薦閱讀
- 工業機器人技術及應用
- 基于LabWindows/CVI的虛擬儀器設計與應用
- Hands-On Cloud Solutions with Azure
- Zabbix Network Monitoring(Second Edition)
- 21天學通Java Web開發
- Nginx高性能Web服務器詳解
- Visual FoxPro數據庫基礎及應用
- Mastering ServiceNow Scripting
- 電腦日常使用與維護322問
- 大數據技術基礎:基于Hadoop與Spark
- 貫通開源Web圖形與報表技術全集
- Oracle 11g Anti-hacker's Cookbook
- 自適應學習:人工智能時代的教育革命
- 單片機C語言編程實踐
- 中老年人學數碼照片后期處理