Geek Answers Handbook

Parallel code in nested loops

You always hear that functional code is inherently easier to parallelize than non-functional code, so I decided to write a function that does the following:

Given line input, sum the number of unique characters for each line. So, given the input, [ "aaaaa"; "bbb"; "ccccccc"; "abbbc" ]our method will return a: 6; b: 6; c: 8.

Here is what I wrote:

(* seq<#seq<char>> -> Map<char,int> *)
let wordFrequency input =
    input
    |> Seq.fold (fun acc text ->
        (* This inner loop can be processed on its own thread *)
        text
        |> Seq.choose (fun char -> if Char.IsLetter char then Some(char) else None)
        |> Seq.fold (fun (acc : Map<_,_>) item ->
            match acc.TryFind(item) with
            | Some(count) -> acc.Add(item, count + 1)
            | None -> acc.Add(item, 1))
            acc
        ) Map.empty

This code is perfectly parallelizable because each line in inputcan be processed in its own thread. This is not as simple as it seems, since the inner dot adds elements to the map shared between all inputs.

, , - . Async?

+5
4

, , , . , " ", ( - ). , , , , , , ( ). , . , " ", . ( , , , , , 26 az .) , -, " F # , .

#light

let input = [| "aaaaa"; "bbb"; "ccccccc"; "abbbc" |]

// first discover all unique letters used
let Letters str = 
    str |> Seq.fold (fun set c -> Set.add c set) Set.empty 
let allLetters = 
    input |> Array.map (fun str -> 
        async { return Letters str })
    |> Async.Parallel 
    |> Async.Run     
    |> Set.union_all // note, this step is single-threaded, 
        // if input has many strings, can improve this

// Now count each letter on a separate thread
let CountLetter letter =
    let mutable count = 0
    for str in input do
        for c in str do
            if letter = c then
                count <- count + 1
    letter, count
let result = 
    allLetters |> Seq.map (fun c ->
        async { return CountLetter c })
    |> Async.Parallel 
    |> Async.Run

// print results
for letter,count in result do
    printfn "%c : %d" letter count

" ", , , - . , , .

+2

:

let wordFrequency =
  Seq.concat >> Seq.filter System.Char.IsLetter >> Seq.countBy id >> Map.ofSeq

, PSeq FSharp.PowerPack.Parallel.Seq DLL Seq :

let wordFrequency =
  Seq.concat >> PSeq.filter System.Char.IsLetter >> PSeq.countBy id >> Map.ofSeq

, , 5.5 , 4.75 0.66 . 7,2 × 8- .

+3

, .

, PLINQ .

+1

F #, . map/reduce:

n = card (& Sigma;) - & sigma; & Sigma;.

:

n , i- , & sigma; i .

:

n . - .

- ; , , , , .

0

More articles:


All Articles