- Mastering Apache Storm
- Ankit Jain
- 258字
- 2021-07-02 20:32:33
Configure parallelism at the code level
Storm provides an API to set the number of worker processes, number of executors, and number of tasks at the code level. The following section shows how we can configure parallelism at the code level.
We can set the number of worker processes at the code level by using the setNumWorkers method of the org.apache.storm.Config class. Here is the code snippet to show these settings in practice:
Config conf = new Config(); conf.setNumWorkers(3);
In the previous chapter, we configured the number of workers as three. Storm will assign the three workers for the SampleStormTopology and SampleStormClusterTopology topology.
We can set the number of executors at the code level by passing the parallelism_hint argument in the setSpout(args,args,parallelism_hint) or setBolt(args,args,parallelism_hint) methods of the org.apache.storm.topology.TopologyBuilder class. Here is the code snippet to show these settings in practice:
builder.setSpout("SampleSpout", new SampleSpout(), 2); // set the bolt class builder.setBolt("SampleBolt", new SampleBolt(), 4).shuffleGrouping("SampleSpout");
In the previous chapter, we set parallelism_hint=2 for SampleSpout and parallelism_hint=4 for SampleBolt. At the time of execution, Storm will assign two executors for SampleSpout and four executors for SampleBolt.
We can configure the number of tasks that can execute inside the executors. Here is the code snippet to show these settings in practice:
builder.setSpout("SampleSpout", new SampleSpout(), 2).setNumTasks(4);
In the preceding code, we have configured the two executors and four tasks of SampleSpout. For SampleSpout, Storm will assign two tasks per executor. By default, Storm will run one task per executor if the user does not set the number of tasks at the code level.
- SPSS數據挖掘與案例分析應用實踐
- Cocos2d Cross-Platform Game Development Cookbook(Second Edition)
- 黑客攻防從入門到精通(實戰(zhàn)秘笈版)
- 精通JavaScript+jQuery:100%動態(tài)網頁設計密碼
- R語言經典實例(原書第2版)
- Mastering Ubuntu Server
- Java應用開發(fā)技術實例教程
- 零基礎輕松學SQL Server 2016
- INSTANT Passbook App Development for iOS How-to
- 基于ARM Cortex-M4F內核的MSP432 MCU開發(fā)實踐
- Node Cookbook(Second Edition)
- JavaScript動態(tài)網頁編程
- SQL Server 2008 R2數據庫技術及應用(第3版)
- Python函數式編程(第2版)
- 進入IT企業(yè)必讀的324個Java面試題