- Hadoop MapReduce v2 Cookbook（Second Edition）
- Thilina Gunarathne
- 360字
- 2021-07-23 20:32:54
Saving money using Amazon EC2 Spot Instances to execute EMR job flows
Amazon EC2 Spot Instances allow us to purchase underutilized EC2 compute resources at a significant discount. The prices of Spot Instances change depending on the demand. We can submit bids for the Spot Instances and we receive the requested compute instances if our bid exceeds the current Spot Instance price. Amazon bills these instances based on the actual Spot Instance price, which can be lower than your bid. Amazon will terminate your instances, if the Spot Instance price exceeds your bid. However, Amazon does not charge for partial Spot Instance hours if Amazon terminated your instances. You can find more information on Amazon EC2 Spot Instances at http://aws.amazon.com/ec2/spot-instances/.
Amazon EMR supports using Spot Instances both as master as well as worker compute instances. Spot Instances are ideal to execute nontime critical computations such as batch jobs.
How to do it...
The following steps show you how to use Amazon EC2 Spot Instances with Amazon Elastic MapReduce to execute the WordCount MapReduce application:
- Follow steps 1 to 9 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.
- Configure your EMR cluster to use Spot Instances in the Hardware Configuration section. (Refer to step 10 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe). In the Hardware Configuration section, select the Request Spot checkboxes next to the instance types.
- Specify your bid price in the Bid price textboxes. You can find the Spot Instance pricing history in the Spot Requests window of the Amazon EC2 console (https://console.aws.amazon.com/ec2).
- Follow steps 11 to 16 of the Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.
There's more...
You can also run the EMR computations on a combination of traditional EC2 on-demand instances and EC2 Spot Instances, safe guarding your computation against possible Spot Instance terminations.
Since Amazon bills the Spot Instances using the current Spot price irrespective of your bid price, it is a good practice to not set the Spot Instance price too low to avoid the risk of frequent terminations.
See also
The Running Hadoop MapReduce v2 computations using Amazon Elastic MapReduce recipe.
- Oracle從入門到精通（第3版）
- Mastering NetBeans
- Spring Cloud Alibaba微服務架構設計與開發實戰
- 算法精粹：經典計算機科學問題的Java實現
- Ext JS Data-driven Application Design
- 你必須知道的204個Visual C++開發問題
- HTML5+CSS3網站設計教程
- Windows Server 2016 Automation with PowerShell Cookbook（Second Edition）
- 大數據分析與應用實戰：統計機器學習之數據導向編程
- 學習OpenCV 4：基于Python的算法實戰
- MySQL從入門到精通（軟件開發視頻大講堂）
- Python機器學習之金融風險管理
- RESTful Web Clients：基于超媒體的可復用客戶端
- 大學計算機基礎實訓教程
- SFML Game Development