I have a very large floating point dataset:
In[25]:= Dimensions[daylightImgd]
Out[25]= {18, 2002, 2989}
In[26]:= daylightImgd[[1, 1]][[1 ;; 10]]
Out[26]= {0.0122293, 0.0104803, 0.0103955, 0.0115533, 0.0118063, \
0.0120648, 0.0122957, 0.011398, 0.0117426, 0.0119997}
I can save the entire array of images on disk using DumpSave a la:
DumpSave["thisWorks.mx", daylightImgd]
Dropping this giant (861 megabytes) takes less than 10 seconds. If I reduce these images, a la:
downsample[image_, f_] := Module[{w, h}, h = Dimensions[image][[1]];
w = Dimensions[image][[2]];
Table[image[[i, j]], {i, 1, h, f}, {j, 1, w, f}]]
In[26]:= daylightImgdDown = downsample[#, 4] & /@ daylightImgd;
In[27]:= Dimensions[daylightImgdDown]
Out[27]= {18, 500, 748}
In[28]:= daylightImgdDown[[1, 1]][[1 ;; 10]]
Out[28]= {0.0122293, 0.0118063, 0.0117426, 0.0119349, 0.0109443, \
0.0121632, 0.0121304, 0.00681408, 0.0101728, 0.00603242}
Then I can no longer tremble; a thing hangs and rotates forever - or at least for many minutes until I kill it, and maxes CPU:
In[31]:= DumpSave["broken.mx", daylightImgdDown]; (* Hangs forever *)
As far as I can tell, everything is as it should be: thumbnails have the correct sizes; you can build them through ArrayPlot and everything looks great; manually enumerating the first line looks great. In short, everything looks the same as with images without snapshots, but DumpSave hangs on a much smaller dataset.
reference
:
. , , .
, downsample [] . , , 18 - , - , Map, 3d- ( PackedArrayQ), .
, , 3d-, 3D- , DumpSave . , , , DumpSave, , ByteCount. , :
In[42]:= downsample3[image_, f_] :=
Module[{w, h}, h = Dimensions[image][[1]];
w = Dimensions[image][[2]];
Developer`ToPackedArray@Table[image[[i, j]], {i, 1, h, f}, {j, 1, w, f}]]
In[43]:= daylightImgdDown = downsample3[#, downsampleSize] & /@ daylightImgd;
In[44]:= ByteCount[daylightImgdDown]
Out[44]= 53966192
In[45]:= Developer`PackedArrayQ[daylightImgdDown]
Out[45]= False
In[46]:= dd = Developer`ToPackedArray[daylightImgdDown];
In[47]:= Developer`PackedArrayQ[dd]
Out[47]= True
In[48]:= ByteCount[dd]
Out[48]= 53963844
In[49]:= DumpSave["daylightImgdDown.mx", dd]; (* works now! *)
.