Working with fragments of structures simultaneously using links

I have JSON I need to do some processing. It uses a slice, which I need to somehow reference to change the structure of the Room at the end of the function. How can I work with this structure at the same time as a reference type?

http://play.golang.org/p/wRhd1sDqtb

type Window struct { Height int64 `json:"Height"` Width int64 `json:"Width"` } type Room struct { Windows []Window `json:"Windows"` } func main() { js := []byte(`{"Windows":[{"Height":10,"Width":20},{"Height":10,"Width":20}]}`) fmt.Printf("Should have 2 windows: %v\n", string(js)) var room Room _ = json.Unmarshal(js, &room) var wg sync.WaitGroup // Add many windows to room for i := 0; i < 10; i++ { wg.Add(1) go func() { defer wg.Done() addWindow(room.Windows) }() } wg.Wait() js, _ = json.Marshal(room) fmt.Printf("Sould have 12 windows: %v\n", string(js)) } func addWindow(windows []Window) { window := Window{1, 1} // Do some expensive calculations fmt.Printf("Adding %v to %v\n", window, windows) windows = append(windows, window) } 
+1
go
source share
2 answers

There are two different problems in your logic: the first is how the slice itself is handled, and the second concerns the actual concurrency problems.

To manipulate fragmentation, simply passing the slice by value as a parameter will mean that you cannot mutate the slice so that the call site sees it when the slice needs to be enlarged or the support array is redistributed to accommodate the new data that you add. There are two general ways to handle this.

Returning a new slice:

 func addWindow(windows []Window) []Window { return append(windows, Window{1, 1}) } room.Windows = addWindow(room.Windows) 

Or by providing a mutable parameter on which the call site maintains a link to:

 func addWindow(room *Room) { room.Windows = append(room.Windows, Window{1, 1}) } 

In the second problem, you have to make sure that the values ​​are not changed at the same time in an unsafe way. There are many ways to solve this problem:

Use channel

Instead of directly manipulating the room, you can ask the windows that N goroutines should prepare and return their results to a checkpoint without checking. For example, you might have:

 windows := make(chan Window, N) for i := 0; i < N; i++ { go createWindow(windows) } for i := 0; i < N; i++ { room.Windows = append(room.Windows, <-windows) } 

and addWindow will look something like this:

 func createWindow(windows chan Window) { windows <- Window{1, 1} } 

Thus, the creation is simultaneous, but the actual manipulation of the room is not.

Add Mutex Field

It is also typical to have a private mutex field in the type itself, for example:

 type Room struct { m sync.Mutex Windows []Window } 

Then, whenever you manipulate concurrency-sensitive fields, protect the exclusive area with the mutex:

 room.m.Lock() room.Windows = append(room.Windows, window) room.m.Unlock() 

Ideally, the use of such a mutex should remain encapsulated next to the type itself, so it is easy to determine how it is used. For this reason, you will often see that a mutex is used from methods of the type itself ( room.addWindow , for example).

If you have panicky logic in an exclusive (secure) area, it might be a good idea to defer an Unlock call right after Lock . Many people simply put one right after the other, even in simple operations, so they don’t need to determine whether it is safe or not. This may be a good idea if you are not sure.

VERY IMPORTANT: In most cases, it is a bad idea to copy a structure with a mutex field by value. Use a pointer to the original value instead. The reason for this is that internally the mutex relies on the address of its fields so as not to change for the correct operation of atomic operations.

Add Global Mutex

In more unusual circumstances, which most likely do not apply to the case that you are trying to process, but which you are well aware of, you can protect the logic yourself and not protect the data. One way to do this is with the mutex global variable, with something around the lines:

 var addWindowMutex sync.Mutex func addWindow(room *Room) { addWindowMutex.Lock() room.Windows = append(room.Windows, Window{1, 1}) addWindowMutex.Unlock() } 

This addWindow method addWindow itself protected, regardless of who calls it. The advantage of this approach is that you do not depend on the implementation of the room for this. The disadvantage is that only a single goroutine falls into the exclusive region, regardless of how many rooms are processed in parallel (which is not the case with the previous solution).

In doing so, remember that reading room.Windows or any data that mutates in the exclusive area must also be protected in case there is still concurrency to change it.

Finally, like some incorrect feedback, check these error values. Ignoring errors is a very bad practice, be it just an example or serious code. You will catch errors many times, even when you create sample code.

+19
source share
 package main import ( "encoding/json" "fmt" "sync" ) type Window struct { Height int64 `json:"Height"` Width int64 `json:"Width"` } type Room struct { mu sync.Mutex Windows []Window `json:"Windows"` } func main() { js := []byte(`{"Windows":[{"Height":10,"Width":20},{"Height":10,"Width":20}]}`) fmt.Printf("Should have 2 windows: %v\n", string(js)) var room Room _ = json.Unmarshal(js, &room) var wg sync.WaitGroup // Add meny windows to room for i := 0; i < 10; i++ { wg.Add(1) go func() { defer wg.Done() addWindow(&room) }() } wg.Wait() js, _ = json.Marshal(room) fmt.Printf("Sould have 12 windows: %v\n", string(js)) } func addWindow(r *Room) { window := Window{1, 1} fmt.Printf("Adding %v to %v\n", window, r.Windows) r.mu.Lock() defer r.mu.Unlock() r.Windows = append(r.Windows, window) } Should have 2 windows: {"Windows":[{"Height":10,"Width":20},{"Height":10,"Width":20}]} Adding {1 1} to [{10 20} {10 20}] Adding {1 1} to [{10 20} {10 20} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1}] Adding {1 1} to [{10 20} {10 20} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1} {1 1}] Sould have 12 windows: {"Windows":[{"Height":10,"Width":20},{"Height":10,"Width":20},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1},{"Height":1,"Width":1}]} 
0
source share

All Articles