Shared memory and distributed memory, as well as multithreaded and multiprocessor

I myself study parallel programming. I wonder if distributed memory is always multi-processor, and multi-threading is shared memory? if the multiprocessor can be both for distributed memory and for shared memory? Thank you and welcome!

+4
source share
1 answer

Yes, yes, and yes, in a sense

In a distributed memory system, different CPUs have their own memory systems. Access from another CPU is likely to be slower or with a more limited consistency model, if at all possible. This will be more typical for a multiprocessor messaging.

Using multiple threads for parallel programming is more of a software paradigm than a hardware problem, but you're right, using the term thread actually indicates that one shared memory is being used, and it may or may not include the actual multiple processors. It may not even include multiple kernel threads, in which case the threads will not run in parallel.

I do not quite understand the meaning of the last question. Of course, saying “distributed memory” or “shared memory” means “distributed processors” and “shared by processors,” so I suggest that these terms only apply to multiprocessor or potentially multiprocessor systems. If we are talking about several processes in software, I assume that this is largely a requirement for systems with distributed memory and essentially a requirement (they can be called threads) for a shared memory system.

I must add that systems with distributed memory but with cache-coherence exist and are a type of shared memory multiprocessor architecture called NUMA. Only a few years ago, these machines were insane parallel computing, but now Intel Core i7 processors have brought NUMA to the mainstream.

+5
source

All Articles