Geek Answers Handbook

Bash error code 137 vs 1 when there is no memory

Context

I run the following command in linux bash:

mono --debug --debugger-agent=transport=dt_socket,address=198.178.155.198:10000 ./Stress.exe

Stress.exe is a C # application.

What's happening

At some point, the system lost the memory it needed. The error code is returned.

Error code returned (echo $?)

Code 1: When my program creates a throw because it does not work.

Code 137: when it restarts the OS when the memory is overloaded.

Question

Why is it ever an OS that kills my application? Why is the result not always the same?

+5
source share
2 answers

Assuming that:

  • Mono launches SGC-based GC
  • Linux OOM Killer is actually enabled
  • Your Stress.exe is only distributed managed memory, i.e. no Native interop, without the use of Marshaling memory allocators, without code flags it is unsafe, etc.
  • You constantly create objects and never release these links.

Let's talk about SGEN, since you distribute objects, they are created in the nursery, since you run out of memory in the nursery, when the GC does a sweep and needs to make a children's collection as it is full, the objects are moved to it in a large pile. If the main head is full, more OS memory is required. You can adjust the amount of source memory allocated to your monoprint, and even fix the amount of memory (max.) That Sgen can use. In addition, managed objects larger than 8,000 bytes are handled by the Sgen Large feature manager, which is non-primary / main memory, but still managed objects / memory.

So usually, when mono needs more space for managed objects and asks the OS for an additional block, and the OS says “NO”, you see an OutOfMemory exception and exit code 0. Your stress test is happy.

But OOM observes that the mono process and the adjustment of its score (oom_score) are higher and higher. This can affect the mono process at any time, but I would bet that it is correct during the GC sweep, when application threads are suspended by SGEN, but before SGEN actually executes an OS memory request due to the lack of controllable memory space in nusery. Thus, you get the output from 137. 137 and 127 = 9, so the mono-process sent a signal SIGKILL (kill -9), and your stress test does not suit you.

Try this as an experiment:

  • 1) If you completely disable the OOM killer. Assuming this is not a live production box that you underline ;-) You will see "System.OutOfMemoryException" in 100% of cases.
  • or 2) Set oom_adj only the mono process to -17 and OOM will leave it alone. Just finish running mono in the shell script to capture its pid and echo -17 to this oom_adj process.
  • or 3) If you change the oom_adj of the mono process below (all the way to -16, you will see a mono-capture of its own disconnected managed memory "more time", but it will never be 100% time ....

This is not a "Mono and / or Sgen / GC" issue. Any process that consumes more and more memory can be killed by OOM. Whether it is a large thick Oracle database or just an application / daemon that has a memory leak, etc., they are all affected by kill'd.

+3
source

"This article describes the Linux killer from memory (OOM) and how to find out why it killed a particular process. It also provides OOM killer configuration methods to better meet the needs of many different environments."

http://www.oracle.com/technetwork/articles/servers-storage-dev/oom-killer-1911807.html

+1
source

More articles:


All Articles