What is the most efficient way to manage large amounts of data (height data) and replace this huge array?

Question

I need to quickly find this data and access all this data. Unfortunately, I also need to save memory (some of them will call OutofMemoryExceptions)

short[,,] data = new short[8000,8000,2];

I tried to do the following:

• tried jagged array - same memory problems
• tried to hack smaller arrays - still having memory problems
• only permission is to efficiently match this data with a memory mapped file, or is there any other way to do this?

Answer 1

How about a database ? , .

NoSQL. (, , ) , .

, - , ( , MongoDB) . , , - .

Redis Membase . , , . , .

, . , Zynga Membase Redis VMWare.

Answer 2

, ?... , , ?

mip-mapping LoD-, ? , .

? ? 64-?

/, , ++, ?

, - , ...?

Answer 3

, . , , .NET . ( ), , , . - , , .

Answer 4

? ? , , , .

, 100% ? - , , - . . , Gigs, - , SqlDataReader.

TL; DR: , O (ref) , , .

, !

Answer 5

.NET 32- , 16 . , , ints int , -.

. :

• 64- . , , (, ). 8 TERAbytes ( ).

• , , IO .