Understanding Neural Network Memory Usage

Question

I am trying to understand how the calculation of memory usage in convection mode was performed here (scroll down to the VGGNet section in the detailed section).

The result of the calculation, to find out how much memory the VGGNet network uses, says:

TOTAL memory: 24M * 4 bytes ~= 93MB 

however, adding up all the memory: values memory: from each of the layers in the list gives only 15 M * 4 bytes, and I'm not sure where the rest of the memory in this sum came from.

Answer 1

You forgot to add memory for FC:

FC: [1x1x4096] memory: 4096 weight: 7 * 7 * 512 * 4096 = 102,760,448

FC: [1x1x4096] memory: 4096 weight: 4096 * 4096 = 16,777,216

FC: [1x1x1000] memory: 1000 scales: 4096 * 1000 = 4 096 000

I think the lost 9M in your calculations.