Geek Answers Handbook

Caffe | Verification error: error == cudaSuccess (2 vs 0) not enough memory

I am trying to train the Caffe network. I have a image size of 512x640. The batch size is 1. I am trying to implement FCN-8s .

I am currently running this on an instance of Amazon EC2 (g2.2xlarge) with 4 GB of GPU memory. But when I run the solver, it immediately gives an error

Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** Aborted (core dumped)

Can someone help me get out of here?

+10
amazon-web-services deep-learning neural-network caffe cuda
source share
3 answers

The error you get is really not in memory, but it is not RAM, but rather GPU memory (note that the error comes from CUDA).
Usually, when coffee has no memory - the first thing to do is to reduce the lot size (at the cost of gradient accuracy ), but since you already have the lot size = 1 ...
Are you sure the lot size is 1 for both the TRAIN and TEST phases?

+15
source share

Caffe can use multiple GPUs. This is only supported in the C ++ interface, not in python. You can also enable cuDNN to reduce memory.

https://github.com/BVLC/caffe/blob/master/docs/multigpu.md

+2
source share

I encountered a similar problem when running Deeplab v2 on a PC with the following configuration:

 ---------- OS: Ubuntu 18.04.3 LTS (64-bit) ---------- Processor: Intel Core i7-6700k CPU @ 4.00 GHz x 8 ---------- GPU: GeForce GTX 780 (3022 MiB) ---------- RAM : 31.3 GiB ----------

Resizing the test and training parties by 1 did not help me. But, resizing the output image certainly did!

0
source share

More articles:


All Articles