Why doesn't my program overflow the stack when I allocate an array of 11 MB char and the top limit of the stack is 10 MB?

Question

I have two simple C ++ programs and two questions. I am working on CentOS 5.2 and my dev environment is as follows:

• g ++ (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50)
• ulimit -s output: 10,240 (kilobytes), i.e. 10 MB

Program No. 1:

main.cpp:

int main(int argc, char * argv[]) { char buf[1024*1024*11] = {0}; return 0; } 

(Compiled with "g ++ -g main.cpp")

The program allocates 1024 * 1024 * 11 bytes (i.e. 11 MB) on the stack, but it does not crash. After I changed the allocation size to 1024 * 1024 * 12 (that is, 12 MB), the program crashes. I think this should be caused by a stack overflow. But why doesn't the program crash when the allocation size is 11 MB, which is also larger than the upper limit of 10 MB?

Program No. 2:

main.cpp:

 #include <iostream> int main(int argc, char * argv[]) { char buf[1024*1024*11] = {0}; std::cout << "*** separation ***" << std::endl; char buf2[1024*1024] = {0}; return 0; } 

(Compiled with "g ++ -g main.cpp")

This program will crash the program because it allocates 12 MB of bytes on the stack. However, according to the main dump file (see below), the crash occurs on buf, but not buf2. It shouldn’t fail with buf2, because it is known from program # 1 that the allocation of char buf [1024 * 1024 * 11] is OK, so after allocating another 1024 * 1024 bytes, will the stack overflow?

I think there should be some fairly fundamental concepts that I have not created a solid understanding. But what is it?

Appendix: core-dump information generated by program # 2:

 Core was generated by `./a.out'. Program terminated with signal 11, Segmentation fault. [New process 16433] #0 0x08048715 in main () at main.cpp:5 5 char buf[1024*1024*11] = {0}; 

Answer 1

You mistakenly believe that stack distributions occur where they appear in your code. Each time you have local variables whose size is known at compile time, space for them will be allocated together when you enter the function. Then only local variables with dynamic size (VLA and alloca) are assigned.

In addition, an error occurs as soon as you write to memory, and not when it is first allocated. Most likely, buf is before buf2 on the stack, and overflow thus occurs in buf, not buf2.

Answer 2

To analyze such secrets, it is always useful to look at the generated code. I assume that your particular version of the compiler does something different, because my segfaults are with -O0, but not with -O1.

From your program # 1 with g ++ ac -g -O0 and then objdump -S a.out

 int main(int argc, char * argv[]) { 8048484: 55 push %ebp 8048485: 89 e5 mov %esp,%ebp 

This is a standard stack stack. Nothing is visible here.

  8048487: 83 e4 f0 and $0xfffffff0,%esp 

Align the stack to a multiple of 16 just in case.

  804848a: 81 ec 30 00 b0 00 sub $0xb00030,%esp 

Allocate 0xB00030 bytes of stack space. This is 1024 * 1024 * 11 + 48 bytes. There is no memory access, so no exception. An additional 48 bytes use the internal use of the compiler.

  8048490: 8b 45 0c mov 0xc(%ebp),%eax 8048493: 89 44 24 1c mov %eax,0x1c(%esp) <--- SEGFAULTS 

The first time access to the stack goes beyond ulimit, so it segfaults.

  8048497: 65 a1 14 00 00 00 mov %gs:0x14,%eax 

Thiis is a stack protector.

  804849d: 89 84 24 2c 00 b0 00 mov %eax,0xb0002c(%esp) 80484a4: 31 c0 xor %eax,%eax char buf[1024*1024*11] = {0}; 80484a6: 8d 44 24 2c lea 0x2c(%esp),%eax 80484aa: ba 00 00 b0 00 mov $0xb00000,%edx 80484af: 89 54 24 08 mov %edx,0x8(%esp) 80484b3: c7 44 24 04 00 00 00 movl $0x0,0x4(%esp) 80484ba: 00 80484bb: 89 04 24 mov %eax,(%esp) 80484be: e8 d1 fe ff ff call 8048394 <memset@plt> 

Initialize an array by calling memset

  return 0; 80484c3: b8 00 00 00 00 mov $0x0,%eax } 

As you can see, segfault happens when accessing internal variables because they fall below a large array (they should be because there is a stack protector to detect stack splitting).

If you compile with optimization, the compiler notices that you are not doing anything with the array and optimizing it. So nothing sigseg.

Your GCC version is probably a bit super small in non-optimized mode and removes the array. We can analyze it further if you publish the output of objdump -S a.out .

Answer 3

When defining local variables on the stack, there is no real memory allocation, as is done on the heap. Stack memory allocation is the easiest way to change the address of the stack pointer (which will be used by the called functions) to reserve the required memory.

I suspect that this operation of changing the stack pointer is performed only once at the beginning of the function in order to reserve space for the entire local variable used (by matching its change once for the local variable). This explains why an error in your program # 2 occurs during the first distribution.

Answer 4

Both of your programs should ideally provide segfault.

Usually, when a function is introduced, all the variables defined in it allocate memory on the stack. However, this also depends on the level of optimization with which the code is compiled.

The optimization level, designated as -O0 during compilation, does not indicate a lack of optimization at all. This is also the default optimization level with which the code is compiled. The above programs, when compiled with -O0, give segfault.

However, when compiling programs using higher optimization levels, the compiler notices that the variable that is defined is not used in the function. Therefore, it removes the variable definition from assembly language code. As a result, your programs will not produce any segfault.