Dynamically switch between stack and heap

Question

Suppose I am writing a simple buffer class. This buffer will act as a simple wrapper for a standard array of C objects. It also needs to be backward compatible to work with existing functions that accept simple arrays as input.

The goal is to make this buffer efficient in both speed and memory usage. Since stack allocation is always faster than a heap, I want to allocate everything on the stack to a certain threshold, and if it grows, redistribute on the heap. How can this be done effectively?

I researched, and apparently std :: string does something similar. I just don't know how to do this. The closest solution I had was something like (pseudocode not compiled):

template <typename T, int MinSize> class Buffer { public: void Push(const T& t) { ++_size; if (_size > MinSize && _heap == NULL) { // allocate _heap and copy contents from stack // _stack is unused and wasted memory } else if (_heap != NULL) { // we already allocated _heap, append to it, re-allocate if needed } else { // still got room on stack, append to _stack } } void Pop() { --_size; if (_size <= MinSize && _heap != NULL) { // no need for _heap anymore // copy values to _stack, de-allocate _heap } else if (_heap != NULL) { // pop from heap } else { // pop from stack } } private: T _stack[MinSize]; T* _heap; int _size; }; 

As you can see, _stack is just lost space when the buffer grows above MinSize . In addition, push and pop can be especially expensive if the Buffer is large enough. Another solution was to keep the first few items always on the stack and put an overflow in the heap. But that would mean that Buffer cannot be "converted" to a simple array.

Is there a better solution? If this is done in std :: string, can someone please indicate how or provide some resources?

Answer 1

I would suggest using the _data pointer instead of _heap , which always refers to your data store. _heap == NULL will become _data == _stack , etc., but in all situations that do not change the length of the data, you could avoid the difference in the case.

The _capacity element is not included in the current sketch to track the selected space. To do this, you will need to implement the “add to it, redistribute if necessary” part if you do not want to redistribute for each change in the length of the container allocated by the heap.

You might also consider freeing up heap space until your data hits the stack. Otherwise, there may be applications that add and remove one element only at this boundary, causing a distribution each time. Therefore, either implement hysteresis , or do not free up heap space at all as soon as you allocate it. In general, I would say that freeing up heap memory should go along with a shrinking heap of memory. Both of you can do this automatically, in response to a specific function call, for example, shrink_to_fit , or not at all, but there is little point in doing one and not the other in a similar situation.

In addition, I believe that your decision is almost everything you can hope for. Perhaps specify a default value for MinSize . If MinSize is small to avoid, then wasting this space will not be a big problem, right?

Of course, ultimately it all depends on your actual application, since many unused stack allocations of this form can have an adverse effect, for example. to stack memory caching. Given the fact that default allocators can be pretty smart, you should probably check to see if you really get anything from this optimization for this application.

Answer 2

I'm not sure if a new data structure is needed here. It seems to me that you really need a new distributor that will be used with any structure that you consider to be the best.

In C ++ 03, this would be relatively difficult, but C ++ 11 now requires STL containers to work with state-controlled controllers, so you could ideally create a dispenser with a small stack for your own use ... and use this as an argument std::vector<> .

Example (using template aliases)

 template <typename T, size_t N = 8> using SmallVector = std::vector<T, SmallAllocator<T, N>>; 

This way, you will benefit from all the confidence and optimization that came with the std::vector implementation, and you just highlight the distribution layer, which was the goal at first glance. It seems.