官术网_书友最值得收藏!

  • Mastering Apache Storm
  • Ankit Jain
  • 258字
  • 2021-07-02 20:32:33

Configure parallelism at the code level

Storm provides an API to set the number of worker processes, number of executors, and number of tasks at the code level. The following section shows how we can configure parallelism at the code level.

We can set the number of worker processes at the code level by using the setNumWorkers method of the org.apache.storm.Config class. Here is the code snippet to show these settings in practice:

Config conf = new Config(); 
conf.setNumWorkers(3); 

In the previous chapter, we configured the number of workers as three. Storm will assign the three workers for the SampleStormTopology and SampleStormClusterTopology topology.

We can set the number of executors at the code level by passing the parallelism_hint argument in the setSpout(args,args,parallelism_hint) or setBolt(args,args,parallelism_hint) methods of the org.apache.storm.topology.TopologyBuilder class. Here is the code snippet to show these settings in practice:

builder.setSpout("SampleSpout", new SampleSpout(), 2); 
// set the bolt class 
builder.setBolt("SampleBolt", new SampleBolt(), 4).shuffleGrouping("SampleSpout"); 

In the previous chapter, we set parallelism_hint=2 for SampleSpout and parallelism_hint=4 for SampleBolt. At the time of execution, Storm will assign two executors for SampleSpout and four executors for SampleBolt.

We can configure the number of tasks that can execute inside the executors. Here is the code snippet to show these settings in practice:

builder.setSpout("SampleSpout", new SampleSpout(), 2).setNumTasks(4); 

In the preceding code, we have configured the two executors and four tasks of SampleSpout. For SampleSpout, Storm will assign two tasks per executor. By default, Storm will run one task per executor if the user does not set the number of tasks at the code level.

主站蜘蛛池模板: 慈溪市| 青岛市| 庐江县| 尼勒克县| 石楼县| 运城市| 和静县| 固始县| 滦南县| 瑞丽市| 咸宁市| 临猗县| 平舆县| 东乌| 彭山县| 东乌珠穆沁旗| 镇沅| 安溪县| 同心县| 武乡县| 宁陕县| 房产| 商都县| 高邮市| 亚东县| 海城市| 宁城县| 巴林右旗| 梅州市| 尖扎县| 芮城县| 墨江| 石景山区| 手游| 元朗区| 剑河县| 宜州市| 淮滨县| 北流市| 岑巩县| 鹤峰县|