-Xss sets the default virtual memory for each new thread stack. (That is, for each new thread that is not given an explicit stack size through its constructor.)
There is no way to limit the combined stack sizes for all threads.
If the -Xss down from 512K to -Xss allows your program to create more threads, then your program must create a lot of threads.
What are all these streams?
Are you sure you cannot solve the problem using a limited number of threads in the thread pool?
source share