書名: PySpark Cookbook作者名: Denny Lee Tomasz Drabas本章字?jǐn)?shù): 106字更新時間: 2021-06-18 19:06:29
How to do it...
There are five major steps we will undertake to install Spark from sources (check the highlighted portions of the code):
- Download the sources from Spark's website
- Unpack the archive
- Build
- Move to the final destination
- Create the necessary environmental variables
The skeleton for our code looks as follows (see the Chapter01/installFromSource.sh file):
#!/bin/bash
# Shell script for installing Spark from sources
#
# PySpark Cookbook
# Author: Tomasz Drabas, Denny Lee
# Version: 0.1
# Date: 12/2/2017
_spark_source="http://mirrors.ocf.berkeley.edu/apache/spark/spark-2.3.1/spark-2.3.1.tgz"
_spark_archive=$( echo "$_spark_source" | awk -F '/' '{print $NF}' )
_spark_dir=$( echo "${_spark_archive%.*}" )
_spark_destination="/opt/spark"
...
checkOS
printHeader
downloadThePackage
unpack
build
moveTheBinaries
setSparkEnvironmentVariables
cleanUp
推薦閱讀
- C語言程序設(shè)計基礎(chǔ)與實驗指導(dǎo)
- C# 從入門到項目實踐(超值版)
- Programming ArcGIS 10.1 with Python Cookbook
- 區(qū)塊鏈:以太坊DApp開發(fā)實戰(zhàn)
- Building Mobile Applications Using Kendo UI Mobile and ASP.NET Web API
- INSTANT Mercurial SCM Essentials How-to
- 嚴(yán)密系統(tǒng)設(shè)計:方法、趨勢與挑戰(zhàn)
- Python算法從菜鳥到達(dá)人
- Mastering JBoss Enterprise Application Platform 7
- Symfony2 Essentials
- Hands-On Functional Programming with TypeScript
- 零基礎(chǔ)學(xué)Python網(wǎng)絡(luò)爬蟲案例實戰(zhàn)全流程詳解(入門與提高篇)
- Getting Started with Python
- 深入解析Java編譯器:源碼剖析與實例詳解
- HTML+CSS+JavaScript網(wǎng)頁制作:從入門到精通(第4版)