Problems installing nimfa (Python matrix factorization library)

Question

I have a large (~ 25000 x 1000) matrix for factorization. I wrote my own numpy-based code, but it is inefficient and continues to cause a memory error.

I am trying to install and use nimfa ( http://nimfa.biolab.si/ ) and the installation process (tried easy_install, pip and downloading and running git) does not show any errors. But when I try to call it using import nimfa , I get the following error. I checked the nimfa prerequisites and did not mention anything except numpy and scipy.

I am on Windows 8 and am using Python 2.7.5 with numpy and scipy installed. I also tried installing (and subsequently uninstalling) minGW and doing this.

Any ideas?

 Traceback (most recent call last): File "<pyshell#0>", line 1, in <module> import nimfa File "C:\Python27\lib\site-packages\nimfa-1.0-py2.7.egg\nimfa\__init__.py", line 18, in <module> from mf_run import * File "C:\Python27\lib\site-packages\nimfa-1.0-py2.7.egg\nimfa\mf_run.py", line 26, in <module> from utils import * File "C:\Python27\lib\site-packages\nimfa-1.0-py2.7.egg\nimfa\utils\__init__.py", line 8, in <module> import linalg File "C:\Python27\lib\site-packages\nimfa-1.0-py2.7.egg\nimfa\utils\linalg.py", line 15, in <module> import scipy.sparse.linalg as sla File "C:\Python27\lib\site-packages\scipy\sparse\linalg\__init__.py", line 100, in <module> from .isolve import * File "C:\Python27\lib\site-packages\scipy\sparse\linalg\isolve\__init__.py", line 6, in <module> from .iterative import * File "C:\Python27\lib\site-packages\scipy\sparse\linalg\isolve\iterative.py", line 7, in <module> from . import _iterative ImportError: DLL load failed: The specified module could not be found.` 

Answer 1

If your goal is to factorize the matrix, rather than using nimfa to do this, I would suggest instead of dask , Dask is designed to provide the ability to perform operations on data objects that are placed on disk but not in memory with minimal changes to the code. Working example:

 import dask.array as da import numpy as np import dask mtx = da.from_array(np.random.normal(size=(25000, 1000)), chunks=(250, 20)) q, r = np.linalg.qr(mtx) 

You may need to adjust the chunks parameter to suit your computing resources (see frequently asked questions for recommendations on this).