Google Compute Engine disk drive is very slow

Question

We just switched to the Google Compute Engine and have serious disk speed issues. It was about 5% of Linode or worse. It never exceeded 20 M / s for writing and 10 M / s for reading. Most of the time it is 15 Mb / s for writing and 5 Mb / s for reading.

We are currently starting the n1-highmem-4 machine (4 vCPU, 26 GB of memory). The processor and memory are not a bottleneck. Just run a script that reads the rows from the PostgreSQL database, processes them, and then writes them back to PostgreSQL. It is just for the normal job of updating a database row in batch mode. Tried 20 processes to use multi-core processors, but overall progress is still slow.

We think that a disk can be a bottleneck because the traffic is abnormally low.

Finally, we decided to benchmark. We found that it is not only slow, but seems to have a big error that reproduces:

• create and connect to instance

• run the test at least three times:

   dd if=/dev/zero bs=1024 count=5000000 of=~/5Gb.file 

We found that it is becoming very slow and cannot completely complete benchmarking.

Answer 1

Constant disk performance is proportional to the size of the disk itself and the virtual machine to which it is attached. The larger the disk (or virtual machine), the higher the performance, therefore, essentially, the price you pay for the disk or VM is paid not only for the disk / CPU / RAM, but also for IOPS and bandwidth.

Indication of permanent disk documentation :

Continuous disk performance depends on the size of the volume and the type of disk you choose. Larger volumes can reach higher I / O levels than smaller volumes. There are no separate I / O costs, since the cost of I / O is included in the price of a permanent disk.

Continuous disk performance can be described as follows:

• IOPS performance limits increase linearly with the size of the volume of the permanent disk.
• The bandwidth limits also grow linearly, up to the maximum bandwidth of the virtual machine to which the permanent disk is attached.
• Larger virtual machines have higher bandwidth limits than smaller virtual machines.

There is also a more detailed pricing chart on the page that shows what you get for every GB of space you buy (the data below is as of August 2014):

  Standard disks SSD persistent disks Price (USD/GB per month) $0.04 $0.025 Maximum Sustained IOPS Read IOPS/GB 0.3 30 Write IOPS/GB 1.5 30 Read IOPS/volume per VM 3,000 10,000 Write IOPS/volume per VM 15,000 15,000 Maximum Sustained Throughput Read throughput/GB (MB/s) 0.12 0.48 Write throughput/GB (MB/s) 0.09 0.48 Read throughput/volume per VM (MB/s) 180 240 Write throughput/volume per VM (MB/s) 120 240 

and a specific example on the page of what the specific disk size will give you:

As an example of how you can use the performance chart to determine the size of the disk you want, consider the 500 GB standard persistent disk to give you:

• (0.3 x 500) = 150 small random readings
• (1.5 x 500) = 750 small random entries
• (0.12 x 500) = 60 MB / s large sequential reads
• (0.09 x 500) = 45 MB / s large sequential writes