How restrictions on working with shared memory on Linux

Question

I studied Linux kernel limitations in shared memory

/proc/sys/kernel/shmall 

indicates the maximum number of pages that can be highlighted. Given this number as x and page size as p. I assume that the "x * p" byte is the limit of the total system memory.

Now I have written a small program to create a shared memory segment, and I am attached to this shared memory segment twice below

 shm_id = shmget(IPC_PRIVATE, 4*sizeof(int), IPC_CREAT | 0666); if (shm_id < 0) { printf("shmget error\n"); exit(1); } printf("\n The shared memory created is %d",shm_id); ptr = shmat(shm_id,NULL,0); ptr_info = shmat(shm_id,NULL,0); 

In the above program, ptr and ptr_info were different. Thus, shared memory maps to two virtual addresses in my process address space.

When I do ipcs , it looks like

 ... 0x00000000 1638416 sun 666 16000000 2 ... 

Now go to the shmall x * p limit mentioned above in my question. Does this limit apply to the sum of all virtual memory allocated for each shared memory segment? or is this limit applicable to physical memory?

There is only one physical memory (shared memory) and from the above program, when I do 2 shmat , it is twice the amount of memory allocated in my process address space. So, this limit will hit soon if the continuous shmat in one shared memory segment?

Answer 1

The limit applies only to physical memory, that is, to real shared memory allocated for all segments, since shmat() simply maps the allocated segment to the process address space.

You can trace it in the kernel, there is only one place where this limit is noted - in newseg() function that allocates new segments ( ns->shm_ctlall comparison). shmat() implementation is busy with a lot of things, but actually it does not have a shmall restriction, so you can map one segment as much as possible (well, the address space is also limited, but in practice you rarely care about this limit).

You can also try some test from user space with a simple program like this:

 #define _GNU_SOURCE #include <errno.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/ipc.h> #include <sys/shm.h> #include <unistd.h> unsigned long int get_shmall() { FILE *f = NULL; char buf[512]; unsigned long int value = 0; if ((f = fopen("/proc/sys/kernel/shmall", "r")) != NULL) { if (fgets(buf, sizeof(buf), f) != NULL) value = strtoul(buf, NULL, 10); // no proper checks fclose(f); // no return value check } return value; } int set_shmall(unsigned long int value) { FILE *f = NULL; char buf[512]; int retval = 0; if ((f = fopen("/proc/sys/kernel/shmall", "w")) != NULL) { if (snprintf(buf, sizeof(buf), "%lu\n", value) >= sizeof(buf) || fwrite(buf, 1, strlen(buf), f) != strlen(buf)) retval = -1; fclose(f); // fingers crossed } else retval = -1; return retval; } int main() { int shm_id1 = -1, shm_id2 = -1; unsigned long int shmall = 0, shmused, newshmall; void *ptr1, *ptr2; struct shm_info shminf; if ((shmall = get_shmall()) == 0) { printf("can't get shmall\n"); goto out; } printf("original shmall: %lu pages\n", shmall); if (shmctl(0, SHM_INFO, (struct shmid_ds *)&shminf) < 0) { printf("can't get SHM_INFO\n"); goto out; } shmused = shminf.shm_tot * getpagesize(); printf("shmused: %lu pages (%lu bytes)\n", shminf.shm_tot, shmused); newshmall = shminf.shm_tot + 1; if (set_shmall(newshmall) != 0) { printf("can't set shmall\n"); goto out; } if (get_shmall() != newshmall) { printf("something went wrong with shmall setting\n"); goto out; } printf("new shmall: %lu pages (%lu bytes)\n", newshmall, newshmall * getpagesize()); printf("shmget() for %u bytes: ", (unsigned int) getpagesize()); shm_id1 = shmget(IPC_PRIVATE, (size_t)getpagesize(), IPC_CREAT | 0666); if (shm_id1 < 0) { printf("failed: %s\n", strerror(errno)); goto out; } printf("ok\nshmat 1: "); ptr1 = shmat(shm_id1, NULL, 0); if (ptr1 == 0) { printf("failed\n"); goto out; } printf("ok\nshmat 2: "); ptr2 = shmat(shm_id1, NULL, 0); if (ptr2 == 0) { printf("failed\n"); goto out; } printf("ok\n"); if (ptr1 == ptr2) { printf("ptr1 and ptr2 are the same with shm_id1\n"); goto out; } printf("shmget() for %u bytes: ", (unsigned int) getpagesize()); shm_id2 = shmget(IPC_PRIVATE, (size_t)getpagesize(), IPC_CREAT | 0666); if (shm_id2 < 0) printf("failed: %s\n", strerror(errno)); else printf("ok, although it wrong\n"); out: if (shmall != 0 && set_shmall(shmall) != 0) printf("failed to restrore shmall\n"); if (shm_id1 >= 0 && shmctl(shm_id1, IPC_RMID, NULL) < 0) printf("failed to remove shm_id1\n"); if (shm_id2 >= 0 && shmctl(shm_id2, IPC_RMID, NULL) < 0) printf("failed to remove shm_id2\n"); return 0; } 

What it does is, it sets the shmall limit one page higher than what is currently used by the system, then tries to get a new page size segment and display it twice (all successfully), and then tries to get another page segment and cannot do this ( run the program as root, because it writes to /proc/sys/kernel/shmall ):

 $ sudo ./a.out original shmall: 18446744073708503040 pages shmused: 21053 pages (86233088 bytes) new shmall: 21054 pages (86237184 bytes) shmget() for 4096 bytes: ok shmat 1: ok shmat 2: ok shmget() for 4096 bytes: failed: No space left on device 

Answer 2

I did not find any physical memory allocation in the do_shmat function (linux / ipc / shm.c)

https://github.com/torvalds/linux/blob/5469dc270cd44c451590d40c031e6a71c1f637e8/ipc/shm.c

therefore shmat consumes only vm (the address space of your process), the main shmat function is mmap