TL, dr:
This seems to be a problem with Cython , there is a hacker way that does the trick, but is not reliable, you can use it for one-time affairs until this problem is fixed. *
Change the source of line_profiler :
I can’t be 100% sure of this, but it works, you need to load the source for line_profiler and go python_trace_callback . After the code = <object>py_frame.f_code from the current execution frame ( code = <object>py_frame.f_code ), add the following:
if what == PyTrace_LINE or what == PyTrace_RETURN: code = <object>py_frame.f_code # Add entry for code object with different address if and only if it doesn't already # exist **but** the name of the function is in the code_map if code not in self.code_map and code.co_name in {co.co_name for co in self.code_map}: for co in self.code_map: # make condition as strict as necessary cond = co.co_name == code.co_name and co.co_code == code.co_code if cond: del self.code_map[co] self.code_map[code] = {}
This will replace the code object in self.code_map one currently executing, which matches its name and the contents of co.co_code . co.co_code b'' for Cython , so essentially in the spokes of Cython functions with that name. Here it can become more reliable and map more attributes of the code object (for example, file name).
Then you can build it using python setup.py build_ext and install using sudo python setup.py install . I am currently creating it using python setup.py build_ext --inplace to work with it locally, and I suggest you do it. If you build it using --inplace , navigate to the folder containing the source for line_profiler before import type it.
So, in the folder containing the built-in shared library for line_profiler , I installed the cyclosure.pyx file containing your functions:
def outer_func(int n): def inner_func(int c): cdef int i for i in range(n): c+=i return c return inner_func
And the equivalent of setup_cyclosure.py script to create it:
from distutils.core import setup from distutils.extension import Extension from Cython.Build import cythonize from Cython.Compiler.Options import directive_defaults directive_defaults['binding'] = True directive_defaults['linetrace'] = True extensions = [Extension("cyclosure", ["cyclosure.pyx"], define_macros=[('CYTHON_TRACE', '1')])] setup(name = 'Testing', ext_modules = cythonize(extensions))
As before, the build was done using python setup_cyclosure.py build_ext --inplace .
Starting your interpreter from the current folder and issuing the following results gives the desired results:
>>> import line_profiler >>> from cyclosure import outer_func >>> f = outer_func(5) >>> prof = line_profiler.LineProfiler(f) >>> prof.runcall(f, 5) 15 >>> prof.print_stats() Timer unit: 1e-06 s Total time: 1.2e-05 s File: cyclosure.pyx Function: inner_func at line 2 Line # Hits Time Per Hit % Time Line Contents ============================================================== 2 def inner_func(int c): 3 cdef int i 4 1 5 5.0 41.7 for i in range(n): 5 5 6 1.2 50.0 c+=i 6 1 1 1.0 8.3 return c
Problem with IPython %%cython :
Trying to run this from IPython leads to an unsuccessful situation. At runtime, the code object does not save the path to the file where it was defined, it just saved the file name. Since I just throw the code object into the self.code_map dictionary, and since the code objects have read-only attributes, we lose track of the file path when using it from IPython (since it stores files generated from %%cython in a temporary directory).
Because of this, you get profiling statistics for your code, but you don’t get content for the content. One could force file names to be copied between two code objects, but this is another problem.
* Problem:
The problem is that for some reason when working with nested and / or private functions, there is an abnormality with the address of the code object when it is created and when interpreted in one of the Pythons frames. The problem you encountered is caused by the following unsatisfied condition :
if code in self.code_map:
That was weird. Creating your function in IPython and adding it to LineProfiler really added it to the self.code_map dictionary:
prof = line_profiler.LineProfiler(f) prof.code_map Out[16]: {<code object inner_func at 0x7f5c65418f60, file "/home/jim/.cache/ipython/cython/_cython_magic_1b89b9cdda195f485ebb96a104617e9c.pyx", line 2>: {}}
When it came time to check the previous condition, and the current code object was torn from the current execution frame using code = <object>py_frame.f_code , the address of the code object was different:
# this was obtained with a basic print(code) in _line_profiler.pyx code object inner_func at 0x7f7a54e26150
indicates that it has been recreated. This only happens with Cython and when a function is defined inside another function. Either this, or something that I'm completely absent.