Python, fix corrupted npy file. ValueError: the total size of the new array must be unchanged

Question

I run a program that saves data in an npy file every 20 seconds, but overwrites the old one to update the file.

Yesterday, after a day of data collection, the program crashed, which led to a human error when someone ran the keyboard at the wrong time.

Now, after loading the npy file, I get a ValueError error message: the total size of the new array should be unchanged

I have already tried

with open ("test.npy", "rb") as npy: a = np.load (npy)

but the error remains.

I think it happens that there are no lines in the npy file, but I do not know how to fix it. The npy file is still the size in MB that it should have, so there is something discreet.

Does anyone know how to fix my file and get the data that is there?

Otherwise, I am losing a day of data.

Any help is much appreciated!

Edit: Here you can find a broken file and an example of a good one that should be exactly the same except for the numbers. http://perswww.kuleuven.be/~u0077049/

Form (1024, 3649) Dtype - dtype ('float64')

Answer 1

The npy format is very simple . It begins with a description of ascii data (data type, array size), followed by a flat binary dump. Header data is always a multiple of 16 bytes, which is a multiple of float64 (8 bytes).

In this example, the header is 80 bytes. In your case, you should try.

import numpy as np

# write npy file
a = 10**np.array(np.linspace(-10, 10, 51), dtype=np.float64)
np.save('foo.npy', a)

# read back
b = np.fromfile('foo.npy', dtype=np.float64)
print(b[10:])

reshape, (Fortran).

; , 29892688 , - 29892609, .. 79 . ( , ) 79 , , .