- Hadoop Real-World Solutions Cookbook(Second Edition)
- Tanmay Deshpande
- 218字
- 2021-07-09 20:02:50
Exporting HDFS data to a local machine
In this recipe, we are going to export/copy data from HDFS to the local machine.
Getting ready
To perform this recipe, you should already have a running Hadoop cluster.
How to do it...
Performing this recipe is as simple as copying data from one folder to the other. There are a couple of ways in which you can export data from HDFS to the local machine.
- Using the
copyToLocal
command, you'll get this code:hadoop fs -copyToLocal /mydir1/LICENSE.txt /home/ubuntu
- Using the
get
command, you'll get this code:hadoop fs -get/mydir1/LICENSE.txt /home/ubuntu
How it works...
When you use HDFS copyToLocal
or the get
command, the following things occur:
- First of all, the client contacts
NameNode
because it needs a specific file in HDFS. NameNode
then checks whether such a file exists in itsFSImage
. If the file is not present, the error code is returned to the client.- If the file exists,
NameNode
checks the metadata for blocks and replica placements inDataNodes
. NameNode
then directly pointsDataNode
from where the blocks would be given to the client one by one. The data is directly copied fromDataNode
to the client machine. and it never goes throughNameNode
to avoid bottlenecks.- Thus, the file is exported to the local machine from HDFS.
推薦閱讀
- Big Data Analytics with Hadoop 3
- ABB工業機器人編程全集
- Dreamweaver CS3+Flash CS3+Fireworks CS3創意網站構建實例詳解
- Java開發技術全程指南
- 輕松學Java
- CorelDRAW X4中文版平面設計50例
- 大學計算機應用基礎
- The Python Workshop
- Docker on Amazon Web Services
- 云計算和大數據的應用
- 精通LabVIEW程序設計
- MATLAB-Simulink系統仿真超級學習手冊
- Photoshop CS4數碼照片處理入門、進階與提高
- PowerPoint 2003中文演示文稿5日通
- EDA技術及其創新實踐(Verilog HDL版)