I want to process a very large object itertools.product. The problem is this:
import string
from itertools import product
text = string.lowercase[:] + string.uppercase[:] + '0123456789'
items = product(text, repeat=5)
for item in items:
#do something
I know the length items 62**5. If I want to process elements itemswhose indices range from 300000to 600000, how do I do this?
I tried converting the list itertools.productto a python list, for example:
items = list(product(text, repeat=5))[300000:600000+1]
for item in items:
but, it seems, the conversion has consumed a large amount of memory, since I have been waiting for this conversion for a long time and finally abandoned it.
I have this requirement because I want to do this in python gevent, so I want to cut large itertool.productinto small elements for gevent spawn.
source
share