Is RAMdisk Slower Than A Disk?

Question

The python program I created is limited to IO. Most of the time (more than 90%) is spent in one cycle, which is repeated ~ 10,000 times. In this cycle, ~ 100 KB of data is generated and written to a temporary file; it is then read by another program and statistics about this collected data. This is the only way to transfer data to the second program.

Because of this, which is the main bottleneck, I thought that moving the location of the temporary file from my main hard drive to (~ 40 MB) RAMdisk (inside more than 2 GB of free memory) would significantly increase the I / O speed for this file and thus reduce lead time. However, I got the following results (each averaged over 20 runs):

• Test data 1: Without RAMdisk - 72.7 s, with RAMdisk - 78.6s
• Test 2 data: without RAMdisk - 223.0s, with RAMdisk - 235.1s

It would seem that RAMdisk is slower than my hard drive.

What could be the reason for this?

Is there any other alternative to using RAMdisk to get a faster IO file?

Answer 1

Your operating system will almost certainly buffer / cache disk writes already. Not surprisingly, the RAM disk is so close in performance.

Not knowing what you are writing and how, we can offer only general suggestions. Some ideas:

• If you have 2 GB of RAM, you probably have a decent processor, so you can write this data to a file system with compression. This will trade I / O for CPU time, assuming your data is amenable to this.

• If you make a lot of small notes, combine them to write large pieces right away. (Can we see the source code?)

• Do you delete the 100 KB file after use? If you do not need it, delete it. Otherwise, the OS may be forced to flush it to disk.

Answer 2

Can you write data in a batch rather than one item at a time? Are you caching resources like open file descriptors, etc., or are you clearing them? Whether your disk is blocking recording, you can use background threads to saturate I / O without affecting computing performance.

I would look at optimizing write to disk first, and then look at faster disks when this is complete.

Answer 3

I know that Windows is very aggressive about caching data on disk to RAM, and 100K fits easily. Recording goes directly to the cache, and then, possibly, it is written to disk using non-blocking recording, which allows the program to continue working. The RAM disk will probably not support non-blocking operations because it expects these operations to be fast and not worth the hassle.

Reducing the amount of available memory for programs and caching, you increase the amount of input / output on the disk for paging, at least a little.

This is all speculation on my part, as I am not familiar with the kernel or drivers. I also assume that Linux will work in a similar way.

Answer 4

I had the same stunning experience, and after many attempts, I realized this. When ramdisk is formatted as FAT32, although the benchmarks show high values, the use of the real world is actually slower than an SSD with NTFS formatting. But the ramdisk format, formatted in NTFS, is faster in real life than SSD.

Answer 5

In my tests, I found that not only the batch size affects the overall performance, but also the nature of the data. I managed to get 5 times better recording time compared to SSD in only one scenario: writing a 100 MB piece of a pre-prepared random byte array to a RAM disk. Writing more “predictable” data, such as the letters “aaa” or the current day-time, gives completely opposite results: SSDs are always faster or equal. Therefore, I assume that the operating system (Win 7 in my case) does a lot of caching and optimization. It seems that the most dangerous case for a RAM drive is that you are doing a lot of small writes, not a few big ones, and the RAM drive shines when recording large amounts of data with hard compression.