The difference between parallel programs, parallel programs, and distributed programs

Question

I just studied research on the structure of programming; In particular, I studied parallel programs. I came across several articles where the distinction between parallel programs, parallel programs, and distributed programs seems intimidating.

My understanding is this:

simultaneous programs: simultaneously performs several tasks or gives an idea about it

parallel programs: algorithms for solving such problems can simultaneously perform some related tasks

distributed programs: has more in common with available resources than the inherent parallelism in the corresponding algorithm. To solve the problem, more than one computing resource is available.

Can someone point me to correct and have a detailed explanation of these conditions?

Answer 1

These terms are really related and sometimes overlap: Parallel programming is a subset of parallel programming, the purpose of which is to use several simultaneously executed objects (threads, processes) to speed up some calculations. Distributed computing is a very broad topic that, as you said, focuses on using resources to solve a problem. These resources may be on the same machine or may be geographically distributed or heterogeneous. It is difficult to draw a clear line between parallel and distributed computing. Parallel computing may or may not be distributed depending on the resources used, while distributed computing may or may not be used to solve the problem in parallel.

In my opinion, the subtle difference between these terms comes from the layers of abstraction on which they are considered. When the main attention is paid to the interaction of individual computing objects according to some common data, we are talking about parallel programming. When a key aspect speeds up computation using multiple units, we have parallel computations. Finally, if the focus is on how to manage resources to solve a problem, then we are talking about distributed computing.