Processing vec in parallel: how to do it safely or without using unstable functions?

Question

I have a massive vector that I want to be able to load / act in parallel, for example. load the first hundreds of thousands of indexes into one stream, the next into another, and so on. Since this will be a very hot part of the code, I came up with the following proof of unsafe concept code to do this without arcs and mutexes:

let mut data:Vec<u32> = vec![1u32, 2, 3]; let head = data.as_mut_ptr(); let mut guards = (0..3).map(|i| unsafe { let mut target = std::ptr::Unique::new(head.offset(i)); let guard = spawn(move || { std::ptr::write(target.get_mut(), 10 + i as u32); }); guard }); 

Is there anything I missed here that could cause it to explode?

In this case, #![feature(unique)] , so I don’t see how to use this in stable. Is there a way to do such things in a stable way (ideally safe without using the Arc and Mutex pointers and overhead)?

Also, looking at the documentation for Unique , he says

It also means that the pointer reference must not be changed without a unique path to a unique reference.

I do not understand what the "unique path" means.

Answer 1

To do this, you can use an external library, for example. simple_parallel (disclaimer, I wrote it) allows you to write:

 extern crate simple_parallel; let mut data = vec![1u32, 2, 3, 4, 5]; let mut pool = simple_parallel::Pool::new(4); pool.for_(data.chunks_mut(3), |target| { // do stuff with `target` }) 

chunks and chunks_mut methods are an ideal way to split a vector / slice T into fragments of the same size: they respectively return an iterator over elements like &[T] and &mut [T] .

Answer 2

Today, the rayon box is the de facto standard for this kind of thing:

 use rayon::prelude::*; fn main() { let mut data = vec![1, 2, 3]; data.par_iter_mut() .enumerate() .for_each(|(i, x)| *x = 10 + i as u32); assert_eq!(vec![10, 11, 12], data); } 

Note that this is just one line, different from the single-threaded version using standard iterators, which will replace par_iter_mut with iter_mut .

See also Writing a Small Ray Tracer in Rust and Zig .