Creating objects with dynamic size

Question

Deleted the C tag, seeing that this was causing some confusion (it should not have started with this, sorry for any inconvenience there. C response is still welcome :)

In several things that I did, I found the need to create objects with dynamic size and static size, where the static part is your main members of the object, however the dynamic part is an array / buffer attached directly to the class, preserving memory continuity, thereby reducing the number of necessary distributions (these are objects that are not redistributable), and reducing fragmentation (although it may be more difficult to find a block with a sufficiently large size as the lower side, but this is much less common - whether this should happen at all - than fragmentation of the heap. It is also useful for embedded devices where memory is very expensive (however, I am not doing anything for embedded devices at present) and things like std :: string should be avoided or not used , as in the case of trivial unions.

In general, as I would do, it would (ab) use malloc (std :: string is not used specifically and for various reasons):

struct TextCache { uint_32 fFlags; uint_16 nXpos; uint_16 nYpos; TextCache* pNext; char pBuffer[0]; }; TextCache* pCache = (TextCache*)malloc(sizeof(TextCache) + (sizeof(char) * nLength)); 

This, however, is not too good with me, since at first I would like to do it using a new one, and therefore in a C ++ environment, and secondly, it looks awful: P

So, the next step was the C ++ boilerplate variable:

 template <const size_t nSize> struct TextCache { uint_32 fFlags; uint_16 nXpos; uint_16 nYpos; TextCache<nSize>* pNext; char pBuffer[nSize]; }; 

This has a problem that saving a pointer to a variable-sized object becomes "impossible", so the following work around:

 class DynamicObject {}; template <const size_t nSize> struct TextCache : DynamicObject {...}; 

However, this still requires casting, and the presence of pointers to DynamicObjects around the place becomes ambiguous when another object with a dynamic size follows from it (it also looks terrible and may suffer from an error that causes empty classes to still have size, although this is probably an archaic, extinct mistake ...).

Then there was the following:

 class DynamicObject { void* operator new(size_t nSize, size_t nLength) { return malloc(nSize + nLength); } }; struct TextCache : DynamicObject {...}; 

which looks much better, but will interfere with objects that already have overloads of new ones (this may even affect the placement of new ones ...).

Finally, I came up with a new investigation:

 inline TextCache* CreateTextCache(size_t nLength) { char* pNew = new char[sizeof(TextCache) + nLength]; return new(pNew) TextCache; } 

This, however, is probably the worst idea so far for a number of reasons.

So are there any better ways to do this? or will one of the above versions be better or at least improved? Is it considered safe and / or bad programming practice?

---

As I said above, I try to avoid double distributions, because there should not be two distributions for this, and this leads to greatly simplified writing (serialization) of these things to files. The only exception to the double allocation requirement that I have is when the overhead is mostly zero. the only reason I ran into this is where I sequentially allocate memory from a fixed buffer ( using this system that I ran into) however this is also a special exception to prevent over-dense copying.

Answer 1

I would compromise with the DynamicObject concept. Everything that does not depend on size falls into the base class.

 struct TextBase { uint_32 fFlags; uint_16 nXpos; uint_16 nYpos; TextBase* pNext; }; template <const size_t nSize> struct TextCache : public TextBase { char pBuffer[nSize]; }; 

This should reduce the casting required.

Answer 2

C99 blesses "struct hack" - aka a flexible member of an array.

§6.7.2.1 Specifications for structure and union

¶16 As a special case, the last structure element with more than one named element may have an incomplete array type; this is called a flexible array element. With two exceptions, the flexible element of the array is ignored. Firstly, the size of the structure must be equal to the offset of the last element of the identical structure, which replaces the flexible element of the array with an array of unspecified length. ¹⁰⁶⁾ Secondly, when a. (or →) the operator has a left operand, which is a (pointer to) structure with a flexible array element and the right operand calls this member, it behaves as if this member was replaced with the longest array (with the same element type) that would not create a structure more than access to an object; the offset of the array must remain equal to the flexible element of the array, even if it differs from the size of the replacement array. If this array does not have any elements, it behaves as if it had one element, but the behavior is undefined if there is any attempt to access this element or generate a pointer in the past.

¶17 EXAMPLE Suppose all array elements are aligned the same after declarations:

 struct s { int n; double d[]; }; struct ss { int n; double d[1]; }; 

three expressions:

  sizeof (struct s) offsetof(struct s, d) offsetof(struct ss, d) 

have the same meaning. The struct s structure has a flexible array element d.

¹⁰⁶⁾ The length is not specified to take into account the fact that implementations can give array members different alignments in their length.

Otherwise, use two separate distributions — one for the master data in the structure and one for the added data.

Answer 3

This may look like a heretic in terms of the economic thinking of C or C ++ programmers, but last time I had a similar problem to solve the problem. I decided to place a static buffer of a fixed size in my structure and access it through a pointer. If this structure was larger than my static buffer, then the direction indicator was then dynamically allocated (and the internal buffer was not used). It was very simple to implement and solve the problems that you raised, for example, fragmentation, since the static buffer was used in more than 95% of the actual use, and the remaining 5% needed really large buffers, so I did not really care about a small loss of the internal buffer .

Answer 4

I believe that in C ++ it is technically Undefined Behavior (due to alignment problems), although I suspect that it could be done to work, probably for every existing implementation.

But why do this?

Answer 5

You can use placement new in C ++:

 char *buff = new char[sizeof(TextCache) + (sizeof(char) * nLength)]; TextCache *pCache = new (buff) TextCache; 

The only caveat is that you need to remove buff instead of pCache , and if pCache has a destructor, you will have to call it manually.

If you plan to access this extra area using pBuffer , I would recommend doing this:

 struct TextCache { ... char *pBuffer; }; ... char *buff = new char[sizeof(TextCache) + (sizeof(char) * nLength)]; TextCache *pCache = new (buff) TextCache; pCache->pBuffer = new (buff + sizeof(TextCache)) char[nLength]; ... delete [] buff; 

Answer 6

There is nothing wrong with managing your own memory.

 template<typename DerivedType, typename ElemType> struct appended_array { ElemType* buffer; int length; ~appended_array() { for(int i = 0; i < length; i++) buffer->~ElemType(); char* ptr = (char*)this - sizeof(DerivedType); delete[] ptr; } static inline DerivedType* Create(int extra) { char* newbuf = new char[sizeof(DerivedType) + (extra * sizeof(ElemType))]; DerivedType* ptr = new (newbuf) DerivedType(); ElemType* extrabuf = (ElemType*)newbuf[sizeof(DerivedType)]; for(int i = 0; i < extra; i++) new (&extrabuf[i]) ElemType(); ptr->lenghth = extra; ptr->buffer = extrabuf; return ptr; } }; struct TextCache : appended_array<TextCache, char> { uint_32 fFlags; uint_16 nXpos; uint_16 nYpos; TextCache* pNext; // IT A MIRACLE! We have a buffer of size length and pointed to by buffer of type char that automagically appears for us in the Create function. }; 

However, you should consider that this optimization is premature, and there are ways to improve it, for example, to have a pool of objects or a managed heap. In addition, I did not expect any kind of alignment, however, I understand that sizeof () returns the alignment size. In addition, it will be a bitch to maintain a non-trivial design. In addition, this is completely untested. Managed heap is the best idea. But you should not be afraid to manage your own memory - if you have custom memory requirements, you need to manage your own memory.

It just occurred to me that I destroyed, but did not delete the "extra" memory.