Scipy Sparse Matrices and Cyton

Question

I need to perform a set of operations on a scipy sparse matrix in a method Cython.

To apply them effectively, I need access to the view lil_matrix. In data representation, lil (linked list with sparse matrix) in python is used list of listswith different lengths.

How can I effectively pass a list of a list of different lengths in cython (without copying)? Is there any other way to access lil matrices in cython?

Answer 1

The example below iterates over lil_matrixand calculates the sum for each row.

. Cython , . ...

import time

import numpy as np
cimport numpy as np

from scipy.sparse import lil_matrix

cdef iter_over_lil_matrix(m):
    cdef list sums, data_row
    sums = []
    for data_row in m.data:
        s = 0
        for value in data_row:
            s += value
        sums.append(s)
    return sums

def main():
    a = np.random.random((1e4*1e4))
    a[a>0.1] = 0
    a = a.reshape(1e4,1e4)
    m = lil_matrix(a)

    t0 = time.clock()
    sums = iter_over_lil_matrix(m)
    t1 = time.clock()
    print 'Cython lil_matrix Time', t1-t0

    t0 = time.clock()
    array_sums = a.sum(axis=1)
    t1 = time.clock()
    print 'Numpy ndarray Time', t1-t0

    t0 = time.clock()
    lil_sums = m.sum(axis=1)
    t1 = time.clock()
    print 'lil_matrix Time', t1-t0

    mcsr = m.tocsr()
    t0 = time.clock()
    csr_sums = mcsr.sum(axis=1)
    t1 = time.clock()
    print 'csr_matrix Time', t1-t0

    assert np.allclose(array_sums, sums)
    assert np.allclose(array_sums, np.asarray(lil_sums).flatten())
    assert np.allclose(array_sums, np.asarray(csr_sums).flatten())

- 2 , NumPy: D, , lil_matrix.sum(), csr_matrix() , @hpaulj . , csr_matrix.sum() , .

Cython lil_matrix Time 0.183935034665
Numpy ndarray Time 0.106583238273
lil_matrix Time 2.47158218631
csr_matrix Time 0.0140050888745

, :

• for i in range(len(m.data)): data_row = m.data[i]
• np.ndarray[object, ndim=1] data data=m.data

, :

• boundscheck wraparound