How to skip file system cache when reading a file in Golang?

Question

Suppose the contents of the Foo.txt file are as follows.

 Foo Bar Bar Foo 

Consider the following short program.

 package main import "syscall" import "fmt" func main() { fd, err := syscall.Open("Foo.txt", syscall.O_RDONLY, 0) if err != nil { fmt.Println("Failed on open: ", err) } data := make([]byte, 100) _, err = syscall.Read(fd, data) if err != nil { fmt.Println("Failed on read: ", err) } syscall.Close(fd) } 

When we run the program above, we get no errors, which is the correct behavior.

Now I am changing the syscall.Open line as follows.

 fd, err := syscall.Open("Foo.txt", syscall.O_RDONLY | syscall.O_SYNC | syscall.O_DIRECT, 0) 

When I run the program again, I get the following (unwanted) output.

 Failed on read: invalid argument 

How to pass the flags syscall.O_SYNC and syscall.O_DIRECT , as indicated in the open man page to skip the file system cache?

Please note that I use the syscall interface directly, and not through the os interface, because I could not find a way to pass these flags into the functions provided by os , but I am open to solutions that use os provided that they work correctly so that disable file system cache while reading.

Please note that I am running on Ubuntu 14.04 with ext4 as my file system.

---

Update . I tried using the @Nick Craig-Wood package in the code below.

 package main import "io" import "github.com/ncw/directio" import "os" import "fmt" func main() { in, err := directio.OpenFile("Foo.txt", os.O_RDONLY, 0666) if err != nil { fmt.Println("Error on open: ", err) } block := directio.AlignedBlock(directio.BlockSize) _, err = io.ReadFull(in, block) if err != nil { fmt.Println("Error on read: ", err) } } 

The next way out

 Error on read: unexpected EOF 

Answer 1

You can enjoy my directio package , which I made just for this purpose.

From the website

This is a Go language library that allows you to use Direct IO on all supported Go operating systems (except openbsd and plan9).

Direct IO performs input / output to and without disk without buffering data in the OS. This is useful when you read or write a lot of data that you do not want to fill the OS cache.

See packaging documentation here.

http://go.pkgdoc.org/github.com/ncw/directio

Answer 2

On the open page, under NOTES:

The O_DIRECT flag can set restrictions on the alignment of the length and address of user space buffers and the offset of input / output files. Linux alignment restrictions depend on the file system and kernel version and may be completely absent.

Thus, you may have alignment problems, be it memory or file offset, or the size of your buffer may be "wrong." What should be matching and dimensions is not obvious. The man page continues:

However, there is currently no file system independent interface for an application to detect these restrictions for a given file or file system.

And even Linus weighs, in his usual understated manner:

"The thing that has always bothered me about O_DIRECT is that the whole interface is just stupid and was probably designed by a distraught monkey over some serious mind-controlling substances." -Linus

Good luck

ps Strike in the dark: why not read 512 bytes?

Answer 3

you can use fadvice and madvice, but there is no guarantee. both will work more likely with large files / data, because:

Partial pages are intentionally stored in the expectation that it is better to save the necessary memory than to discard unnecessary memory.

see Linux source code for what will do and what not. For example, POSIX_FADV_NOREUSE does nothing.

http://lxr.free-electrons.com/source/mm/fadvise.c#L62

http://lxr.free-electrons.com/source/mm/madvise.c

 package main import "fmt" import "os" import "syscall" import "golang.org/x/sys/unix" func main() { advise := false if len(os.Args) > 1 && os.Args[1] == "-x" { fmt.Println("setting file advise") advise =true } data := make([]byte, 100) handler, err := os.Open("Foo.txt") if err != nil { fmt.Println("Failed on open: ", err) }; defer handler.Close() if advise { unix.Fadvise(int(handler.Fd()), 0, 0, 4) // 4 == POSIX_FADV_DONTNEED } read, err := handler.Read(data) if err != nil { fmt.Println("Failed on read: ", err) os.Exit(1) } if advise { syscall.Madvise(data, 4) // 4 == MADV_DONTNEED } fmt.Printf("read %v bytes\n", read) } 

/ usr / bin / time -v./direct -x

 Command being timed: "./direct -x" User time (seconds): 0.00 System time (seconds): 0.00 Percent of CPU this job got: 0% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.03 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 1832 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 2 Minor (reclaiming a frame) page faults: 149 Voluntary context switches: 2 Involuntary context switches: 2 Swaps: 0 File system inputs: 200 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0