How do you read the segfault kernel log message

Question

This may be a very simple question, I'm trying to debug an application that generates the following segfault error in kern.log

kernel: myapp[15514]: segfault at 794ef0 ip 080513b sp 794ef0 error 6 in myapp[8048000+24000]

Here are my questions:

• Is there any documentation about what the number of error errors in segfault is, in this case it is error 6, but I saw error 4, 5

• What is the meaning of the information at bf794ef0 ip 0805130b sp bf794ef0 and myapp[8048000+24000] ?

So far I have managed to compile the characters, and when I do x 0x8048000+24000 , it returns the character, is this the right way to do this? So far my assumptions are as follows:

• sp = stack pointer?
• ip = instruction pointer
• at = ????
• myapp [8048000 + 24000] = address of the character?

Answer 1

When a report points to a program, not a shared library

Run addr2line -e myapp 080513b (and repeat for other instruction pointer values) to see where the error occurs. Better, get a debug build and reproduce the problem under a debugger like gdb.

If it is a shared library

The libfoo.so[NNNNNN+YYYY] NNNNNN contains the place where the library was loaded. Subtract this from the instruction pointer ( ip ) and you will get a bias in the .so violation instruction. Then you can use objdump -DCgl libfoo.so and look for instructions with this offset. You should easily figure out which function this is from asm labels. If .so doesn't have optimizations, you can also try using addr2line -e libfoo.so <offset> .

What error means

Here's the breakdown of the fields:

• address is the location in memory that the code is trying to get to (it is likely that 10 and 11 are offsets from the pointer, which we expect to be set to a valid value, but which instead points to 0 )
• ip - instruction pointer, i.e. where the code that tries to do this lives
• sp - stack pointer
• error - architecture-specific flags; see arch/*/mm/fault.c for your platform.

Answer 2

Based on my limited knowledge, your assumptions are correct.

• sp = stack pointer
• ip = instruction pointer
• myapp[8048000+24000] = address

If I debugged the problem, I would modify the code to dump the kernel or register the backtrace stack on failure. You can also run the program under (or application) GDB.

The error code is just an architectural error code for page errors and seems to relate to architecture. They are often documented in arch/*/mm/fault.c in the kernel source. My copy of Linux/arch/i386/mm/fault.c has the following definition for error_code:

• bit 0 == 0 means that the page was not found, 1 means a security error.
• bit 1 == 0 means read, 1 means write
• bit 2 == 0 means kernel, 1 means user mode

My copy of Linux/arch/x86_64/mm/fault.c adds the following:

• bit 3 == 1 means that the error was caused by the command

Answer 3

If it is a shared library

You, unfortunately, want; it is impossible to find out where the libraries were put into memory by the dynamic linker after .

Well, there is still the ability to extract information, not from binary, but from an object. But you need the base address of the object. And this information is still inside the coredump in the link_map structure.

So first you want to import the struct link_map into GDB. So let's compile the program with it using the debug symbol and add it to GDB.

link.c

 #include <link.h> toto(){struct link_map * s = 0x400;} 

get_baseaddr_from_coredump.sh

 #!/bin/bash BINARY=$(which myapplication) IsBinPIE () { readelf -h $1|grep 'Type' |grep "EXEC">/dev/null || return 0 return 1 } Hex2Decimal () { export number="`echo "$1" | sed -e 's:^0[xX]::' | tr '[af]' '[AF]'`" export number=`echo "ibase=16; $number" | bc` } GetBinaryLength () { if [ $# != 1 ]; then echo "Error, no argument provided" fi IsBinPIE $1 || (echo "ET_EXEC file, need a base_address"; exit 0) export totalsize=0 # Get PT_LOAD size segment out of Program Header Table (ELF format) export sizes="$(readelf -l $1 |grep LOAD |awk '{print $6}'|tr '\n' ' ')" for size in $sizes do Hex2Decimal "$size"; export totalsize=$(expr $number + $totalsize); export totalsize=$(expr $number + $totalsize) done return $totalsize } if [ $# = 1 ]; then echo "Using binary $1" IsBinPIE $1 && (echo "NOT ET_EXEC, need a base_address..."; exit 0) BINARY=$1 fi gcc -g3 -fPIC -shared link.c -o link.so GOTADDR=$(readelf -S $BINARY|grep -E '\.got.plt[ \t]'|awk '{print $4}') echo "First do the following command :" echo file $BINARY echo add-symbol-file ./link.so 0x0 read echo "Now copy/paste the following into your gdb session with attached coredump" cat <<EOF set \$linkmapaddr = *(0x$GOTADDR + 4) set \$mylinkmap = (struct link_map *) \$linkmapaddr while (\$mylinkmap != 0) if (\$mylinkmap->l_addr) printf "add-symbol-file .%s %#.08x\n", \$mylinkmap->l_name, \$mylinkmap->l_addr end set \$mylinkmap = \$mylinkmap->l_next end 

it will bring you all the link_map content as part of the GDB team.

It may seem unnesseray, but with the base_addr of the shared object we are talking about, you can get additional information from the address by debugging the directly involved shared object in another GDB instance. Save the first gdb to have a character ideal.

NOTE: the script is rather incomplete, I suspect that you can add add-symbol to the second parameter of the file by printing the sum with this value:

 readelf -S $SO_PATH|grep -E '\.text[ \t]'|awk '{print $5}' 

where $ SO_PATH is the first argument of the add-symbol symbol file

Hope this helps