Working with char buffers

Question

As a C ++ programmer, I sometimes have to deal with memory buffers using methods from C. For example:

char buffer[512]; sprintf(buffer, "Hello %s!", userName.c_str()); 

Or on Windows:

 TCHAR buffer[MAX_PATH+1]; // edit: +1 added ::GetCurrentDirectory(sizeof(buffer)/sizeof(TCHAR), &buffer[0]); 

The above example is how I usually create local buffers (local array with char array). However, there are many possible options, and therefore I am very interested in your answers to the following questions:

• Passing a buffer as &buffer[0] better programming style than passing buffer ? (I prefer &buffer[0] .)
• Is there a maximum size that is considered safe for buffered stacks?
  • Update: I mean, for example, the highest value that can be considered safe for cross-platform desktop applications on Mac, Windows, Linux (non-mobile) desktop computers.
• Is the static buffer faster ( static char buffer[N]; )? Are there any other arguments for or against him?
• When using static buffers, you can use the const char * return type. Is this a (generally) good or bad idea? (I understand that the caller will need to make his own copy to avoid the next call changing the previous return value.)
• How about using static char * buffer = new char[N]; by never deleting the buffer and not using it for every call.
• I understand that heap allocation should be used when (1) deals with large buffers or (2) the maximum buffer size is unknown at compile time. Are there any other factors that play in the stack / heap allocation decision?
• If you prefer options sprintf_s , memcpy_s , ...? (Visual Studio has long been trying to convince me of this, but I want to get a second opinion: p)

Answer 1

I assume that your interest is shown primarily in terms of performance, since solutions such as vector, string, wstring, etc., as a rule, will work even for interacting with C APIs. I recommend learning how to use them and how to use them effectively. If you really need it, you can even write your own memory allocator to make them super fast. If you are sure that they are not what you need, you still have no excuse for not writing a simple wrapper to handle these string buffers using RAII for dynamic cases.

With this in mind:

Passing a buffer as a buffer [0] is a better programming style than a buffer transition? (I prefer the buffer [0].)

Not. I would find this style a little less useful (admittedly subjective here) since you cannot use it to pass a null buffer, and so you have to make exceptions in your style to pass pointers to arrays that may be empty . However, this is necessary if you are passing data from std :: vector to the C API, waiting for a pointer.

Is there a maximum size considered safe for dedicated stack buffers?

It depends on your platform and compiler settings. A simple rule of thumb: if you doubt that your code will overflow the stack, write it in the wrong way.

Is the static buffer (static charbuffer [N];) faster? Are there any other arguments for or against him?

Yes, there is a big argument against this, and this means that your function ceases to be repeated. If your application becomes multithreaded, these functions will not be thread safe. Even in a single-threaded application, sharing the same buffer while recursively calling these functions can lead to problems.

How about using a static buffer char * = new char [N]; and never delete the buffer? (Reusing the same call buffer.)

We still have the same problems with reconnecting.

I understand that heap allocation should be used when (1) deals with large buffers or (2) the maximum size buffer is unknown at compile time. Are there other factors that play into the decision about stack / heap allocation?

Stacking destroys objects on the stack. This is especially important for exception safety. Thus, even if you allocate memory on the heap inside a function, you should usually manage the object on the stack (for example: smart pointer) .///@ see section RAII.

If you prefer sprintf_s, memcpy_s, ... options? (Visual Studio tried to convince me of this for a long time, but I want a second opinion: p)

MS was right about these functions, which are safer alternatives, since they have no problems with buffer overflows, but if you write such code as it is (without writing options for other platforms), your code will marry Microsoft. since it will not be portable.

When using static buffers, you can use return type const char *. Is this a (generally) good or bad idea? (I realize that the caller to make his own copy to avoid the next call will change the previous return value.)

I would say in almost every case, you want to use const char * for return types for a function that returns a pointer to a character buffer. For a function returning a mutable char *, it is usually confusing and problematic. Or it returns an address to global / static data that it should not use first (see Re-inclusion above), local class data (if it is a method), in which case its return destroys the class’s ability to maintain invariants, allowing clients to intervene into this, but they like it (for example: the stored string should always be valid) or return the memory indicated by the pointer passed to the function (the only case where it could reasonably be said that mutable char * should be returned).

Answer 2

• Stay away from static buffers if you ever want to reuse your code.

• use snprintf () instead of sprintf () so you can control buffer overflows.

• You never know how much stack space is left in the context of your call, so the size is not technically “safe”. You play most of the time to play most of the time. But one day it will benefit you. I use a rule of thumb to never stack arrays.

• Do the client collect the buffer and pass it and its size to your function. This makes him a repeat participant and leaves no ambiguity about who needs to control the life of the buffer.

• If you are dealing with string data, double-check your string functions to make sure they end, especially when they get to the end of the buffer. The C library is very incompatible when it comes to handling line endings for various functions.

Answer 3

• It is up to you to simply make buffer more concise, but if it was vector , you still need to do &buffer[0] .
• Depends on your intended platform.
• Does it matter? Have you decided that this is a problem? Write the code that is easiest to read and maintain before escaping anxiety if you can confuse it with something faster. But for what it's worth, stack allocation is really fast (you just change the value of the stack pointer.)
• You should use std::string . If performance becomes a problem, you can reduce dynamic allocations simply by returning the internal buffer. But the std::string return interface is much better and safer, and performance is your last problem.
• This is a memory leak. Many will argue that everything is in order, since the OS frees it anyway, but I feel the terrible practice of just leaking things. Use static std::vector , you should never do any highlighting! If you put yourself in a position where you can leak (because it needs to be done explicitly), you are doing it wrong.
• I think yours (1) and (2) just cover it. Dynamic allocation is almost always slower than stack allocation, but you should be more concerned about what makes sense in your situation.
• You should not use them at all. Use std::string , std::stringstream , std::copy , etc.

Answer 4

You have a lot of questions! I will do my best to answer the couple and give you a place to search for others.

Is there a maximum size that is considered safe for stacked distributed buffers?

Yes, but the size of the stack depends on the platform you are working on. See When are you worried about stack size? for a very similar question.

It is a static buffer char [N]; Faster? Are there any other arguments for or against him?

The value of static depends on where the buffer is declared, but I assume that you are talking about static declared inside a function, so it is initialized only once. In functions called many times, using static buffers may be a good idea to prevent, but otherwise keep in mind that allocating buffers is a cheap operation. In addition, static buffers are much more difficult to work with when working with multiple threads.

For answers to most of your other questions, see Large Buffers vs. Large Static Buffers, is there an advantage? .

Answer 5

Do buffers have a better programming style as & buffer [0] than traversing a buffer? (I prefer & buffer [0].)

&buffer[0] makes the code less readable to me. I have to pause for a second and wonder why someone used it instead of just going through buffer . Sometimes you need to use &buffer[0] (if buffer is std::vector ), but otherwise stick to the standard C style.

Is there a maximum size that is considered safe for stacked distributed buffers?

I doubt there is a practical limit if you use the stack reasonably enough. I have never had any problems in my development.

If I read MSDN correctly, streams on Windows by default are 1 MB of stack size. This is customizable. Other platforms have other limitations.

It is a static buffer char [N]; Faster? Are there any other arguments for or against?

On the one hand, this can reduce the need to commit memory pages for the stack, so your application can run faster. Switching to a BSS segment or equivalent, on the other hand, can reduce cache locality compared to the stack, so your application may run slower. I seriously doubt that you notice the difference anyway.

Using static not streaming when using the stack. This is a huge stack advantage. (Even if you don’t think you will be multithreaded, why make life harder if this changes in the future?)

When using static buffers, you can return the return function with the return type const char *. Is that a good idea? (I understand that the caller will need to make his own copy to avoid the next call changing the previous return value.)

The correctness constant is always a good thing.

Returned pointers to static buffers are error prone; a later call can change it, another thread can change it, etc. Instead, use std::string or other automatic allocated memory (even if your function should work internally with char buffers, such as your GetCurrentDirectory example).

How about using static char * buffer = new char [N]; and never delete the buffer? (Reusing the same buffer for each call.)

Less efficient than just using static char buffer[N] , since you need heap allocation.

I understand that heap allocation should be used when (1) deals with large buffers or (2) the maximum buffer size is unknown at compile time. Are there any other factors that play in the stack / heap allocation decision?

See Justin Ardini's answer.

If you prefer options sprintf_s, memcpy_s, ...? (Visual Studio has long been trying to convince me of this, but I need a second opinion: p)

This is a matter of some debate. Personally, I think that these functions are a good idea, and if you focus exclusively on Windows, there is an advantage in choosing your preferred approach to Windows and using these functions. (And they’re pretty easy to override if you later need to focus on something other than Windows, if you don’t rely on their behavior to handle errors.) Others believe that Secure CRT functions are no more secure than using C correctly and enter other disadvantages; Wikipedia links with several arguments against them.

Answer 6

If the function gives you a way of knowing how many characters it will return, use it. The GetCurrentDirectory example example is a good example:

 DWORD length = ::GetCurrentDirectory(0, NULL); 

Then you can use a dynamically allocated array (either a string or a vector) to get the result:

 std::vector<TCHAR> buffer(length, 0); // assert(buffer.capacity() >= length); // should always be true GetCurrentDirectory(length, &buffer[0]); 

Answer 7

1) buffer and &buffer[0] should be equivalent.

2) Stack size limits will depend on your platform. For most simple functions, my personal rule of thumb is that something over ~ 256KB is declared dynamically; there is no real rhyme or reason for this number, although this is just my own agreement, and it is currently included in the default stack sizes for all platforms for which I am developing.

3) Static buffers are not faster or slower (for all purposes and tasks). The only difference is the access control mechanism. The compiler usually puts static data in a separate section of the binary file than non-static data, but there are no noticeable / significant advantages or penalties. The only real way to say for sure is to write a program in both directions and time (since many aspects of speed related to this depend on your platform / compiler).

4) Do not return the const pointer if the caller needs to modify it (which defeats the const point). Use const for function parameters and return types if and only if they are not intended to be modified. If the caller needs to change the value, the best option would be for the caller to pass the function to the pointer to the previously allocated buffer (along with the buffer size) and the function to write data to this buffer.

5) Reusing a buffer can lead to increased performance for larger buffers due to bypassing overhead that is involved in a malloc / free or new / delete call each time. However, you run the risk of accidentally using old data if you forget to flush the buffer each time or try to run two copies of the function in parallel. Again, the only real way to find out for sure is to try both methods and measure how long it takes to execute the code.

6) Another factor in stack / heap allocation is the definition of the area. The stack variable goes out of scope when the function in which it lives is returned, but the variable dynamically allocated on the heap can be returned safely to the caller or available the next time the function is called (a la strtok ).

7) I would recommend using sprintf_s , memcpy_s and friends. They are not part of the standard library and are not portable. The more you use these functions, the more additional work you will have when you want to run your code on another platform or use a different compiler.

Answer 8

• A buffer or buffer [0] is exactly the same. You can even write Buffer + 0. Personally, I prefer to just write Buffer (and I think most developers also prefer this), but this is your personal choice.
• The maximum depends on how big and how deep your stack is. If you already use 100 functions on the stack, the maximum size will be less. If you can use C ++, you can write a buffer class that dynamically selects whether to use the stack (for small sizes) or the heap (for large sizes). You will find the code below.
• A static buffer is faster because the compiler will reserve space in advance for you. The stack buffer is also fast. For a stack buffer, an application simply needs to increment the stack pointer. For the heap buffer, the memory manager must find free space, ask the operating system about the new memory, then free it again, do some bookkeeping, ...
• If possible, use C ++ strings to avoid memory leaks. Otherwise, the caller must know whether he should free memory after this or not. The downside is that C ++ strings are slower than static buffers (since they are allocated on the heap).
• I would not use memory allocation for global variables. When are you going to remove it? , ( )?
• , . , , (, , ). , , char -pointer.
• sprintf, , , , , . , snprintf, .

/:

 template<size_t BUFSIZE,typename eltType=char> class DynamicBuffer { private: const static size_t MAXSIZE=1000; public: DynamicBuffer() : m_pointer(0) {if (BUFSIZE>=MAXSIZE) m_pointer = new eltType[BUFSIZE];} ~DynamicBuffer() {if (BUFSIZE>=MAXSIZE) delete[] m_pointer;}; operator eltType * () { return BUFSIZE>=MAXSIZE ? m_pointer : m_buffer; } operator const eltType * () const { return BUFSIZE>=MAXSIZE ? m_pointer : m_buffer; } private: eltType m_buffer[BUFSIZE<MAXSIZE?BUFSIZE:1]; eltType *m_pointer; }; 

Answer 9

Is passing the buffer as &buffer[0] better programming style than passing buffer? (I prefer &buffer[0].)

. : buffer + index &buffer[index] , .

Is there a maximum size that is considered safe for stack allocated buffers?

. , , , , - .

Is static char buffer[N]; faster? Are there any other arguments for or against it?

, . . :

When using static buffers you can have your function return have the const char * return type. Is this a good idea? (I do realize that the caller will need to make his own copy to avoid that the next call would change the previous return value.)

, , :

• ​​ ( char buf[100] ): , . / (, ).

• ​​ static , . strtok .

What about using static char * buffer = new char[N]; and never deleting the buffer? (Reusing the same buffer each call.)

, , .

I understand that heap allocation should be used when (1) dealing with large buffers or (2) maximum buffer size is unknown at compile time. Are there any other factors that play in the stack/heap allocation decision?

, ( ).

Should you prefer the sprintf_s, memcpy_s, ... variants? (Visual Studio has been trying to convince me of this for a long time, but I want a second opinion :p )

, : . :

 // this is not tested - it is just an example #ifdef _WINDOWS #define SPRINTF sprintf_s #else #define SPRINTF sprintf #endif