Linux syscalls and errno

Question

Context: I am trying to write a small C program with built-in asm that should run under Linux on x86_64 and compiled with gcc to better understand how syscalls work under Linux.

My question is: how do error numbers return from a system call (e.g. write) in this environment? I understand that when I use a library such as glibc, it takes care of saving the resulting error code in the global variable errno . But where is the error number saved when I call syscall directly through the inline assembler? Will it be stored inside a separate register or will it be encoded in %rax ?

Take the Linux write script as an example:

When calling write after the syscall message is issued, I find that it stores 0xfffffffffffffff2 inside %rax , do I need to somehow extract the error code from this?

If I have an error code, where should I look for the actual error that occurred? Suppose I got a return number of 5, in which header file I need to consult to find the corresponding symbolic name for the error.

I call syscall like this:

 asm ("mov $1,%%rax;" "mov $1,%%rdi;" "mov %1,%%rsi;" "mov %2,%%rdx;" "syscall;" "mov %%rax,%0;" : "=r" (result) : "r" (msg), "r" (len) : "%rdx", "%rsi", "%rax", "%rdi" /* EDIT: this is needed or else the registers will be overwritten */ ); 

with result , msg and len defined like this:

 long result = 0; char* msg = "Hello World\n"; long len = 12; 

Answer 1

The agreement on using the Linux system is that they encode both a possible error code and a return value for a successful call in the return value. It's just glibc or other shells of C libraries that they will set errno to the error code returned by the underscore syscall, and the shell will return -1 . Take write as an example, the kernel performs error handling similar to this:

 ssize_t write(int fd, ...) { if (fd is not valid) return -EBADF; return do_write(...); } 

So, as you can see, the error code is only in the return value, and depending on the semantics there is always a way to check whether syscall succeeded or not by comparing it with a value that is impossible for successful operation. For most system calls, for example, write , this means that it is negative.

Answer 2

Architecture Conventions

As you may have guessed, you cannot use errno because it is GLibC specific . The information you want will be in rax if it is x86_64 . The man 2 syscall contains the following explanation:

 Architecture calling conventions Every architecture has its own way of invoking and passing arguments to the kernel. The details for various architectures are listed in the two tables below. The first table lists the instruction used to transition to kernel mode (which might not be the fastest or best way to transition to the kernel, so you might have to refer to vdso(7)), the register used to indicate the system call number, the register used to return the system call result, and the register used to signal an error. arch/ABI instruction syscall # retval error Notes ──────────────────────────────────────────────────────────────────── alpha callsys v0 a0 a3 [1] arc trap0 r8 r0 - arm/OABI swi NR - a1 - [2] arm/EABI swi 0x0 r7 r0 - arm64 svc #0 x8 x0 - blackfin excpt 0x0 P0 R0 - i386 int $0x80 eax eax - ia64 break 0x100000 r15 r8 r10 [1] m68k trap #0 d0 d0 - microblaze brki r14,8 r12 r3 - mips syscall v0 v0 a3 [1] nios2 trap r2 r2 r7 parisc ble 0x100(%sr2, %r0) r20 r28 - powerpc sc r0 r3 r0 [1] s390 svc 0 r1 r2 - [3] s390x svc 0 r1 r2 - [3] superh trap #0x17 r3 r0 - [4] sparc/32 t 0x10 g1 o0 psr/csr [1] sparc/64 t 0x6d g1 o0 psr/csr [1] tile swint1 R10 R00 R01 [1] x86_64 syscall rax rax - [5] x32 syscall rax rax - [5] xtensa syscall a2 a2 - 

And pay attention to the number [5] :

 [5] The x32 ABI uses the same instruction as the x86_64 ABI and is used on the same processors. To differentiate between them, the bit mask __X32_SYSCALL_BIT is bitwise-ORed into the system call number for system calls under the x32 ABI. Both system call tables are available though, so setting the bit is not a hard requirement. 

(A table is displayed on this user page showing how to pass arguments to system calls. This is interesting to read.)

---

How are error numbers returned from Syscall (e.g. write) in this environment? :

You need to check the rax register for the return value.

Answer 3

On Linux, a failed system call using the assembly instruction syscall returns the value -errno in the rax register. So in your case, 0-0xfffffffffffffff2 == 0xE, which is 14. So your error is 14.

How do you find errno 14? You should search the Linux “Error Code Table” on Google or browse errno.h and you will find the answer.

Take a look here: http://www.virtsync.com/c-error-codes-include-errno

According to this table, 14 is EFAULT, which means "Bad Address".

Answer 4

IIRC in ABI x86-64, the error is transmitted from syscall with the carry bit set. Then eax contains the errno code.

I would suggest exploring the lower levels of the source code of some libc library, for example musl-libc