It seems odd to want mappers and reducers to have the same type. Map/reduce is popular for limiting the scope of computations so they can execute in parallel with shared-nothing, which imposes very different roles on them. A mapper should be a pure function which takes a single input value (it can't rely on seeing more of the input than that) and produces a collection (not necessarily a list but some monad like that which the framework can join over) of values having an equivalence relation. A reducer should be a pure function which takes a collection of all values in one equivalence class (it can rely on seeing all of them from any mapper, but not any other values) and produces whatever (again, some collection monad so the framework can join over the collection-of-collections).
1 comment
[ 2.4 ms ] story [ 12.1 ms ] thread