How to fix the "AttributeError: __exit__" issue in multiprocessing in Python?

Question

I tried to rewrite some csv code to be able to run it on multiple cores in Python 3.2.2. I tried to use the Multiprocessor Pool object, which I adapted from working examples (and already worked for me for another part of my project). I came across an error message that was difficult for me to decrypt and fix.

Mistake:

 Traceback (most recent call last): File "parser5_nodots_parallel.py", line 256, in <module> MG,ppl = csv2graph(r) File "parser5_nodots_parallel.py", line 245, in csv2graph node_chunks) File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/multiprocessing/pool.py", line 552, in get raise self._value AttributeError: __exit__ 

Relevant Code:

 import csv import time import datetime import re from operator import itemgetter from multiprocessing import Pool import itertools def chunks(l,n): """Divide a list of nodes 'l' in 'n' chunks""" l_c = iter(l) while 1: x = tuple(itertools.islice(l_c,n)) if not x: return yield x def csv2nodes(r): strptime = time.strptime mktime = time.mktime l = [] ppl = set() pattern = re.compile(r"""[A-Za-z0-9"/]+?(?=[,\n])""") for row in r: with pattern.findall(row) as f: cell = int(f[3]) id = int(f[2]) st = mktime(strptime(f[0],'%d/%m/%Y')) ed = mktime(strptime(f[1],'%d/%m/%Y')) # collect list l.append([(id,cell,{1:st,2: ed})]) # collect separate sets ppl.add(id) return (l,ppl) def csv2graph(source): MG=nx.MultiGraph() # Remember that I use integers for edge attributes, to save space! Dic above. # start: 1 # end: 2 p = Pool() node_divisor = len(p._pool) node_chunks = list(chunks(source,int(len(source)/int(node_divisor)))) num_chunks = len(node_chunks) pedgelists = p.map(csv2nodes, node_chunks) ll = [] ppl = set() for l in pedgelists: ll.append(l[0]) ppl.update(l[1]) MG.add_edges_from(ll) return (MG,ppl) with open('/Users/laszlosandor/Dropbox/peers_prisons/python/codetenus_test.txt','r') as source: r = source.readlines() MG,ppl = csv2graph(r) 

What a good way to fix this?

Answer 1

The problem is in this line:

 with pattern.findall(row) as f: 

You are using with expression. Requires an object with __enter__ and __exit__ . But pattern.findall returns a list with attempts to save __exit__ , but cannot find it and throws an error. Just use

 f = pattern.findall(row) 

instead.

Answer 2

The problem is not this question, but the first troubleshooting step for the general "AttributeError: __exit__" should be sure that there are brackets, for example

 with SomeEnterExitObject() as foo: #works because a new object is referenced... 

not

 with SomeEnterExitObject as foo: #AttributeError because the class is referenced 

Catches me from time to time and I end here -__-

Answer 3

An error also occurs when trying to use

 with multiprocessing.Pool() as pool: # ... 

with a version of Python that is too old (like Python 2.x) and does not support the use of with with multiprocessor pools.

(See this answer at https://stackoverflow.com/a/3/3737/ ... for another question)

Answer 4

The reason for this error is that the Flask application is already running, it did not close, and in the middle of this we try to start another instance with: app.app_context (): #Code Before using this with the operator, we need to make sure that the area of ​​the previous running application is closed.

Answer 5

Yes, to get around this in my case, I just assigned my "with" variable to a regular variable that had a shorter name. Worked for me.