- Hadoop Real-World Solutions Cookbook(Second Edition)
- Tanmay Deshpande
- 218字
- 2021-07-09 20:02:50
Exporting HDFS data to a local machine
In this recipe, we are going to export/copy data from HDFS to the local machine.
Getting ready
To perform this recipe, you should already have a running Hadoop cluster.
How to do it...
Performing this recipe is as simple as copying data from one folder to the other. There are a couple of ways in which you can export data from HDFS to the local machine.
- Using the
copyToLocal
command, you'll get this code:hadoop fs -copyToLocal /mydir1/LICENSE.txt /home/ubuntu
- Using the
get
command, you'll get this code:hadoop fs -get/mydir1/LICENSE.txt /home/ubuntu
How it works...
When you use HDFS copyToLocal
or the get
command, the following things occur:
- First of all, the client contacts
NameNode
because it needs a specific file in HDFS. NameNode
then checks whether such a file exists in itsFSImage
. If the file is not present, the error code is returned to the client.- If the file exists,
NameNode
checks the metadata for blocks and replica placements inDataNodes
. NameNode
then directly pointsDataNode
from where the blocks would be given to the client one by one. The data is directly copied fromDataNode
to the client machine. and it never goes throughNameNode
to avoid bottlenecks.- Thus, the file is exported to the local machine from HDFS.
推薦閱讀
- 大學(xué)計(jì)算機(jī)信息技術(shù)導(dǎo)論
- Word 2003、Excel 2003、PowerPoint 2003上機(jī)指導(dǎo)與練習(xí)
- Mastering Hadoop 3
- 人工免疫算法改進(jìn)及其應(yīng)用
- 數(shù)據(jù)運(yùn)營之路:掘金數(shù)據(jù)化時代
- Visual C# 2008開發(fā)技術(shù)實(shí)例詳解
- 數(shù)控銑削(加工中心)編程與加工
- 網(wǎng)絡(luò)綜合布線技術(shù)
- 傳感器與新聞
- SAP Business Intelligence Quick Start Guide
- 簡明學(xué)中文版Photoshop
- Serverless Design Patterns and Best Practices
- 伺服與運(yùn)動控制系統(tǒng)設(shè)計(jì)
- 網(wǎng)絡(luò)信息安全項(xiàng)目教程
- 深度學(xué)習(xí)之模型優(yōu)化:核心算法與案例實(shí)踐