MapReduce的原理-I Then consider the problem of counting the number of occurrences of each word in the following sentences: The weather is good. Today is good. Good weather is good. Today has good weather
MapReduce的原理-1 Then consider the problem of coun6ng the number of occurrences of each word in the following sentences: The weather is good. Today is good. Good weather is good. Today has good weather
MapReduce的原理-l 目标数据 初始kw 初始kv 初始kw 初始kw 键佳对 键值对 数据划分 键值对 键值对 Map Map 0●年年 Map Map (the,1) (today,1) (g0od,1) (today,1) (weather,1) (is,1) (weather,1) (ha5,1) (is,1 (good,1) (s,1j (good,1) (g00d,1) (good,1) (weather,1) Combiner Combiner Combiner Combiner (the,1) (weather,1) (today,1) 中间结果 (good,2) (today,1) (is,1) (is,1 (weather, (has,1) (good,1) (g0od,1) 1) (good,1) (is,1) veather,1】 Partitioner Partitioner Barrier Partitioner Partitioner (good,1) (is,1) (weather, (good,1) (is,1 1) (good,2) (is,1) (weather, (good,1) (has,1) Reduce Reduce Reduce 1) (weather, (is,3) 1) (weather, (good,5) (has,1) 3) (the,1) (the,1) (today,1) 计算结果 (today,2) (today,1)
MapReduce的原理-1 Barrier (good, 1) (good, 1) (good,2) (good,1) Partitioner Partitioner Partitioner Partitioner (is, 1) (is, 1) (is, 1) (has, 1) (weather, 1) (weather, 1) (weather, 1) (the, 1) (today, 1) (today,1) 目标数据 计算结果 …… 数据划分 Map 初始kv 键值对 初始kv 键值对 初始kv 键值对 初始kv 键值对 Map Map Map 中间结果 (the, 1) (weather, 1) (is, 1) (good, 1) Combiner Combiner Combiner Combiner (the, 1) (weather, 1) (is, 1) (good, 1) (today, 1) (is, 1) (good, 1) (good, 1) (weather, 1) (is, 1) (good, 1) (today, 1) (has, 1) (good, 1) (weather, 1) (today, 1) (is, 1) (good, 1) (good, 2) (weather, 1) (is, 1) (today, 1) (has, 1) (good, 1) (weather, 1) Reduce Reduce Reduce (good, 5) (is, 3) (has, 1) (weather, 3) (the, 1) (today, 2)