Python processes do not join

Question

I can’t insert my code snippet, so I’ll explain scenario 1. I have a list of about 50,000 shares that need some evaluation 2. These stocks are in MultiProcess.Queue 3. I create several processes, each process will receive a batch of 50 out of line and do something. 4. in the main thread, I have a check that looks like this

 anymore_to_process = True

 while anymore_to_process:
     if (stock_queue.qsize() == 0):
         anymore_to_process = False

for jobs in stock_jobs:
    jobs.join()

5. however, this does not work when I process records of 50,000. If I process 500 shares, this works fine.

what am I doing wrong? Why processes do not connect when I process a lot of stocks.

I know that it is difficult to answer without looking at my code ..... but if you can give me some pointers it will be very helpful.

Answer 1

The problem is solved - I took a cue from the question @dano. I really wrote in another queue that blocked processes. I decided that the problem was resolved.