Get a fragment of an array without creating a new copy

Question

I am trying to find a solution / workaround for slicing extremely large arrays without creating new copies. Here is my problem.

Suppose I have a large double / int array of size 100 million or more. I store many different arrays representing different things in one extremely large array in order to significantly save on memory usage. Therefore, instead of having 1 million arrays each of size 100, I have one array of size 100 million. I store indexes (start and end) to keep track of my data.

I want to get thousands of fragments of size 100. If I use the Arrays.copyOfRange () method to get the fragments, it defeats the goal of placing it in just one large array, since each fragment is a new copy.

I have legacy code (over a million lines written over the years by many people) that works with its own data (which are smaller arrays). It is not possible to modify existing code for working with indexes (beginning, end) in a large array.

If I could somehow return the original array so that the returned array is a reference (or claims to be a role), where index 0 is some arbitrary index in the original large array, that would be great.

In C / C ++, I can easily return a pointer with a special offset and length with which the call code can work.

What are my options in Java?

Edit: I considered the following similar question, but it does not contain an answer to my question. How to get auxiliary array array in Java without copying data?

Answer 1

Your best option is to store the slice indices in a separate structure, such as an array storing these indices.

Thus, you do not create large arrays that are a section of the entire data array.

Answer 2

For an array of int values, you can enclose an IntBuffer . You can also wrap a slice of an array.

 int[] largeArray = . . . // create a slice containing the elements 100 through 149 (50 elements): IntBuffer slice = IntBuffer.wrap(largeArray, 100, 50); 

Answer 3

Can you create your own object that stores the index, size and reference to the original array?

 class CustomizedArray { int startIndex; int size; int[] originalArray; public CustomizedArray(int startIndex, int size, int[] originalArray) { this.startIndex = startIndex; this.size = size; this.originalArray = originalArray; } public int getIndex(int index) { int originalIndex = startIndex+index; if(index <0 || originalIndex >= startIndex+size) { throw new IndexOutOfBoundException(); } return originalArray[originalIndex]; } 

You can then save the CustomizedArray in some larger structure.

Answer 4

How to create a wrapper class containing references to your source array and your start index, and using an instance of this wrapper to access the source array.

The code below may not be syntactically correct, but it should give you this idea.

 public class ArraySlice(){ private int startIndex; private int[] originalArray; //getters-setters public ArraySlice(int[] originalArray, int startIndex){ //Initialize } public int get(int index){ return originalArray[startIndex+index] } }