Maximum thread for a process on Linux

Question

I wrote a simple program to calculate the maximum number of threads that a process can have on linux (Centos 5). here is the code:

int main() { pthread_t thrd[400]; for(int i=0;i<400;i++) { int err=pthread_create(&thrd[i],NULL,thread,(void*)i); if(err!=0) cout << "thread creation failed: " << i <<" error code: " << err << endl; } return 0; } void * thread(void* i) { sleep(100);//make the thread still alive return 0; } 

I realized that the maximum number for threads is only 300 !? What if I need more? I should mention that pthread_create returns 12 as an error code.

Thanks before

Answer 1

The thread limit for linux is limited , and you can change it by completing the required limit to /proc/sys/kernel/threads-max . The default value is calculated from the available system memory. In addition to this limit, there is another limit: /proc/sys/vm/max_map_count , which limits the maximum mmapped segments and at least the latest kernels will write memory to the stream. It should be safe to increase this limit if you hit it.

The limitation you click on is the lack of virtual memory on a 32-bit operating system. Install 64-bit Linux, if your hardware supports it, and everything will be fine. I can easily start 30,000 threads with a stack size of 8 MB. The system has one Core 2 Duo + 8 GB system memory (I use 5 GB at the same time for other things), and it works with 64-bit Ubuntu with a 2.6.32 kernel. Please note that overcommit memory (/ proc / sys / vm / overcommit_memory) must be enabled, because otherwise the system will need at least 240 GB of memory (the sum of real memory and swap space).

If you need a lot of threads and you cannot use a 64-bit system, the only choice is to minimize the memory usage in the stream to save virtual memory. Start by querying as a small stack that you can live with.

Answer 2

The limitations of your system may not allow you to map the stacks of all the required threads. Take a look at /proc/sys/vm/max_map_count and see this answer . I am not 100% sure that this is your problem, because most people run into problems in much more threads .

Answer 3

I also ran into the same problem when the number of threads crosses a certain threshold. This was because of the user-level limit (the number of processes that the user can start at a time) set to 1024 in the /etc/security/limits.conf file.

so check your /etc/security/limits.conf and find the entry: -

username - / soft / hard -nproc 1024

change it to a few larger values ​​at 100k (sudo privileges / root is required) and it should work for you.

For more information on security policies, see http://linux.die.net/man/5/limits.conf .

Answer 4

check the stack size per thread using ulimit, in my case Redhat Linux 2.6:

  ulimit -a ... stack size (kbytes, -s) 10240 

Each of your threads will receive this amount of memory (10 MB) assigned to it. With a 32-bit program and a maximum address space of 4 GB, this is a maximum of 4096 MB / 10 MB = 409 streams! The minus code of the program, minus heaps of space, is likely to lead to your observed maximum. from 300 threads.

You have to raise this by compiling a 64-bit application or installing ulimit -s 8192 or even ulimit -s 4096. But if appropriate, this is another discussion ...

Answer 5

You also won't run out of memory unless you reduce the default stack size. Its 10 MB in our version of Linux.

EDIT: Error code 12 = from memory, so I believe the 1 MB stack is still too big for you. Compiled for 32 bits, I can get a 100 kilobyte stack to give me 30k streams. For 30k streams, I get an error code of 11, which means it is no longer allowed by threads. The 1 MB stack gives me about 4 thousand. Streams before the error code 12. 10 MB gives me 427 threads. 100MB gives me 42 threads. 1 GB gives me 4 ... We have a 64-bit OS with 64 GB of RAM. Is your OS 32 bit? When I compile for 64-bit, I can use any stack size I want and get the thread limit.

I also noticed that if I go over the profiling material (Tools | Profiling) for netbeans and run from ide ... I can only get 400 threads. Weird Netbeans also dies if you use all threads.

Here is a test application that you can run:

 #include <stdio.h> #include <stdlib.h> #include <pthread.h> #include <signal.h> // this prevents the compiler from reordering code over this COMPILER_BARRIER // this doesnt do anything #define COMPILER_BARRIER() __asm__ __volatile__ ("" ::: "memory") sigset_t _fSigSet; volatile int _cActive = 0; pthread_t thrd[1000000]; void * thread(void *i) { int nSig, cActive; cActive = __sync_fetch_and_add(&_cActive, 1); COMPILER_BARRIER(); // make sure the active count is incremented before sigwait // sigwait is a handy way to sleep a thread and wake it on command sigwait(&_fSigSet, &nSig); //make the thread still alive COMPILER_BARRIER(); // make sure the active count is decrimented after sigwait cActive = __sync_fetch_and_add(&_cActive, -1); //printf("%d(%d) ", i, cActive); return 0; } int main(int argc, char** argv) { pthread_attr_t attr; int cThreadRequest, cThreads, i, err, cActive, cbStack; cbStack = (argc > 1) ? atoi(argv[1]) : 0x100000; cThreadRequest = (argc > 2) ? atoi(argv[2]) : 30000; sigemptyset(&_fSigSet); sigaddset(&_fSigSet, SIGUSR1); sigaddset(&_fSigSet, SIGSEGV); printf("Start\n"); pthread_attr_init(&attr); if ((err = pthread_attr_setstacksize(&attr, cbStack)) != 0) printf("pthread_attr_setstacksize failed: err: %d %s\n", err, strerror(err)); for (i = 0; i < cThreadRequest; i++) { if ((err = pthread_create(&thrd[i], &attr, thread, (void*)i)) != 0) { printf("pthread_create failed on thread %d, error code: %d %s\n", i, err, strerror(err)); break; } } cThreads = i; printf("\n"); // wait for threads to all be created, although we might not wait for // all threads to make it through sigwait while (1) { cActive = _cActive; if (cActive == cThreads) break; printf("Waiting A %d/%d,", cActive, cThreads); sched_yield(); } // wake em all up so they exit for (i = 0; i < cThreads; i++) pthread_kill(thrd[i], SIGUSR1); // wait for them all to exit, although we might be able to exit before // the last thread returns while (1) { cActive = _cActive; if (!cActive) break; printf("Waiting B %d/%d,", cActive, cThreads); sched_yield(); } printf("\nDone. Threads requested: %d. Threads created: %d. StackSize=%lfmb\n", cThreadRequest, cThreads, (double)cbStack/0x100000); return 0; }