Resizing char * on the fly - why does this code * work *?

Question

I am experimenting with malloc and realloc and came up with code for the following problem:

I want to create a string of unknown size without setting any restrictions. I could ask the user for nr characters, but I rather resize str as the user type of each character.

So, I tried to do this using malloc + realloc, and the idea was that every time the user enters a new char, I use realloc to request a +1 piece of memory to store the char.

While trying to implement this, I made a mistake and ended up doing the following:

int main () { /* this simulates the source of the chars... */ /* in reality I would do getch or getchar in the while loop below... */ char source[10]; int i, j; for (i=0, j=65; i<10; i++, j++) { source[i] = j; } /* relevant code starts here */ char *str = malloc(2 * sizeof(char)); /* space for 1 char + '\0' */ int current_size = 1; i = 0; while(i<10) { char temp = source[i]; str[current_size-1] = temp; str[current_size] = '\0'; current_size++; printf("new str = '%s' | len = %d\n", str, strlen(str)); i++; } printf("\nstr final = %s\n", str); return 0; } 

Please note that the realloc part is not yet implemented.

I compiled and executed this code and got the following output

 new str = 'A' | len = 1 new str = 'AB' | len = 2 new str = 'ABC' | len = 3 new str = 'ABCD' | len = 4 new str = 'ABCDE' | len = 5 new str = 'ABCDEF' | len = 6 new str = 'ABCDEFG' | len = 7 new str = 'ABCDEFGH' | len = 8 new str = 'ABCDEFGHI' | len = 9 new str = 'ABCDEFGHIJ' | len = 10 

I found these results strange because I expected the program to crash: str has room for 2 characters, and the code adds more than 2 characters to str without asking for more memory. In my opinion, this means that I write to a memory that I do not own, so it should give a runtime error.

So ... Why does this work?

(The compiler is GCC 4.3.4.)

Thanks in advance.

EDIT: One of the commenters suggesting that calling free () can lead to an error signal. I tried calling free () with the above code, and no error occurred while executing the code. However, after adding more elements to the original array, as well as to call for free, the following error was received:

* glibc ./prog detected: free (): invalid next size (fast): 0x09d67008 **

Answer 1

Since you write allocated memory, your code has undefined behavior .

The fact that the code did not crash once (or even many times) does not change it.

undefined behavior does not mean that the code should crash. In your case, some memory happens right after str , which you rewrite. The actual effects of overwriting this memory are unknown (you could change the value of some other variable, corrupt the heap, launch a nuclear strike, etc.).

Answer 2

it looks like from glibc-2.14, the memory allocation will be allocated as size as follows and it will set the border, so when allocating 2-byte size "char * str = malloc (2 * sizeof (char))", it seems that the allocated memory is at least 16 bytes, so you can add more elements and then cause a program error.

 struct _bucket_dir bucket_dir[] = { { 16, (struct bucket_desc *) 0}, { 32, (struct bucket_desc *) 0}, { 64, (struct bucket_desc *) 0}, { 128, (struct bucket_desc *) 0}, { 256, (struct bucket_desc *) 0}, { 512, (struct bucket_desc *) 0}, { 1024, (struct bucket_desc *) 0}, { 2048, (struct bucket_desc *) 0}, { 4096, (struct bucket_desc *) 0}, { 0, (struct bucket_desc *) 0}}; /* End of list marker */ void *malloc(unsigned int len) { struct _bucket_dir *bdir; struct bucket_desc *bdesc; void *retval; /* * First we search the bucket_dir to find the right bucket change * for this request. */ for (bdir = bucket_dir; bdir->size; bdir++) if (bdir->size >= len) break; if (!bdir->size) { printk("malloc called with impossibly large argument (%d)\n", len); panic("malloc: bad arg"); } /* * Now we search for a bucket descriptor which has free space */ cli(); /* Avoid race conditions */ for (bdesc = bdir->chain; bdesc; bdesc = bdesc->next) if (bdesc->freeptr) break; /* * If we didn't find a bucket with free space, then we'll * allocate a new one. */ if (!bdesc) { char *cp; int i; if (!free_bucket_desc) init_bucket_desc(); bdesc = free_bucket_desc; free_bucket_desc = bdesc->next; bdesc->refcnt = 0; bdesc->bucket_size = bdir->size; bdesc->page = bdesc->freeptr = (void *) cp = get_free_page(); if (!cp) panic("Out of memory in kernel malloc()"); /* Set up the chain of free objects */ for (i=PAGE_SIZE/bdir->size; i > 1; i--) { *((char **) cp) = cp + bdir->size; cp += bdir->size; } *((char **) cp) = 0; bdesc->next = bdir->chain; /* OK, link it in! */ bdir->chain = bdesc; } retval = (void *) bdesc->freeptr; bdesc->freeptr = *((void **) retval); bdesc->refcnt++; sti(); /* OK, we're safe again */ return(retval); } 

Answer 3

[We assume that the behavior is undefined for this operation]

Heap integrity is often checked when calling realloc or free, and not every time you write, you probably haven't redefined enough to get a crash.

Please note that you did not call for free at the end, if you want, you are likely to get a failure.

Answer 4

Besides the fact that you run undefined behavior, which includes but does not require a “crash”, I think that your application really owns the memory that you write.

In modern operating systems, memory is processed in pages , pieces of memory more than two bytes. AFAIK malloc queries the OS for full pages and shares them internally, if necessary. (Note: implementation dependent, but I think at least glibc works that way.) That way, the OS allows you to write to memory, because it's technically yours. Inside, malloc usually divides the page and supplies parts of it for each request. This way you can overwrite another variable in your heap. Or write outside, according to the malloc view, memory is still waiting to be requested.

I expect an accident only when you try to write a page that has not yet been allocated from the OS or marked as read-only.

Answer 5

To add to the previous answer, there really is no place for 2, it is just a pointer in memory. Somewhere, malloc remembers that he allocated space for two characters, but this is malloc's internal work.

You can try the following little experiment to see how it works:

Create another pointer immediately after the first.

 char *str2 = str + 5; /* or you could simply malloc another */ char *str2 = malloc(2); printf("str=%d, str2=%d\n",str,str2); /* to eyeball the pointers actually received and note the difference in the two pointers. You will need to raise source length to at least that much to see the results below */ 

and enter another printf in the loop after the first:

 printf("new str2 = '%s' | len = %d\n", str2, strlen(str2)); 

Sooner or later, str2 will also start showing the same letters.

NTN