MemoryError when creating a very large numpy array

Question

I am trying to create a very large array with zero number of zeros and then copy the values ​​from another array to a large array of zeros. I use Pycharm and I keep getting: MemoryError even when I try to create an array. This is how I tried to create an array of zeros:

 import numpy as np last_array = np.zeros((211148,211148)) 

I tried to increase the memory heap in Pycharm from 750 m to 1024 m according to this question: https://superuser.com/questions/919204/how-can-i-increase-the-memory-heap-in-pycharm , but this doesn't seem to help.

Let me know if you would like further clarification. Thanks!

Answer 1

Look at using the sparse array capabilities in scipy:
scipy.sparse documentation .

The scipy.sparse library has a set of examples and tutorials:
Exceptional Lectures: Sparse Matrices in SciPy

This can help you solve memory problems, as well as make things faster.

---

To create an empty sparse array with values ​​at specific positions, as you requested in your comment:

Is there a way to create an empty array with values ​​in certain positions, for example: last_array [211147] [9], but would it remain empty everywhere?

 from scipy.sparse import * values = [42] row_ind = [211147] col_ind = [9] last_array = csc_matrix((values, (row_ind, col_ind)), shape=(211148,211148)) print(last_array[211147,9])