Multiple threads inside a docker container

Question

I need to create N threads inside a docker container. I am going to get a list of elements and then split it into pieces, and each thread will process each fragment.

So, I use a docker container with one process and N threads. Is this a good practice in docker? I think so, because we have, for example, the apacha web server that handle connections that cover streams.

Or would it be better to spawn N containers each for each piece? If so, what is the correct way to do this?

Answer 1

The container as such has nothing to do with the calculation you need to perform. The question you are posting is, should I have several processes executing my processing, or several threads spawned by the same process executing the processing?

A container is a platform for launching an application in the required environment. Period. This means that you will run the process inside the container to run your business logic. Several containers simply mean several processes and, as reported, you should use several threads, not several processes, as spawning of a new process (in your case as a container) will consume more resources , and also require more memory, etc. . Therefore, it is better to have only one container that will spawn multiple threads in order to do the job for you.

However, it also depends on the configuration of the host computer on which the container is running. If it makes sense to create multiple containers with multiple threads due to the multi-core capabilities of the underlying hardware, you should do this.

Answer 2

Short answer:

Run the program as one docker container. Think of a docker container as a lightweight sandbox, akin to a virtual environment where you can run a program / service. This service can run multiple threads, all started from the parent program - this is another service running in the same docker container.

Explanation:

Suppose you have a program that spawns threads for this kind of work: this program can be a pool of threads to do some computation in many pieces, or it can be a web server such as apache. It could even be some python code that creates a process pool that performs chunch calculations. In all these cases, all threads and processes belong to the master process, which can be considered as one program or service. This single program is launched with a single user command, the command you will be in Dockerfile ENTRYPOINT.

For example, you can start the apache server container using the official apache image on docker hub docker hub ref ):

docker run -dit --name my-apache-app -v "$PWD":/usr/local/apache2/htdocs/ httpd:2.4 

And this will lead to the fact that the apache web server will become the only container, regardless of how many threads it runs, which can be easily attributed to when the operator wants to stop it, restart it, delete it, etc., using docker commands. And this is more convenient, since we do not need to worry about installing installation volumes, opening ports and connecting multiple containers so that they exchange information with each other.

So, the main thing is that you want to create a container for each instance of the service. For example, if you want to run duplicate instances of the parent process. run apache on two machines as part of a load-balanced configuration, then you will run two containers, one on each host.

In addition, as an alternative, if you have a precedent when you needed to perform various tasks in a batch system, where each task required the installation of certain libraries, then this type of use would be useful for isolating the environment, it would be achieved from running various containers. But this is not what you asked, your question specifically mentioned that the web server spawns threads and processes that use threads to work in pieces, and for these cases you create one container for the service / program.