官术网_书友最值得收藏!

SingleMapperCombinerReducer job

SingleMapperReducer jobs are used in aggregation use cases. A combiner, also known as a semi-reducer, is an optional class that operates by accepting the inputs from the map class and thereafter passing the output key/value pairs to the reducer class. The purpose of the combiner is to reduce the workload of the reducer:

In the MapReduce program, 25% of the work is done in the map stage, which is also known as the data preparation stage, which works in parallel. At the same time, 75% of the work is done in the reduce stage, which is known as the calculation stage, and is not parallel. Therefore, it is slower than the map phase. To reduce time, some work in the reduce phase can be done in the combiner phase.

For example, if we have a combiner, then we will send (Boston, 66) from a mapper, which sees (Boston, 22), (Boston, 24), (Boston, 20) as input records, instead of sending three inpidual key/pair records across the network.

主站蜘蛛池模板: 瑞金市| 杭锦后旗| 富顺县| 凯里市| 呼图壁县| 肇源县| 宁波市| 菏泽市| 德江县| 四会市| 米易县| 南川市| 正阳县| 闽清县| 文成县| 台南县| 佛冈县| 葫芦岛市| 抚顺县| 萨嘎县| 百色市| 新民市| 增城市| 积石山| 习水县| 巴彦淖尔市| 兴义市| 论坛| 金阳县| 林西县| 武邑县| 吉水县| 内黄县| 察隅县| 达日县| 通河县| 洛宁县| 沈丘县| 贵港市| 新兴县| 荔波县|