Fast implementation of MD5 in C ++

Question

First of all, to be clear, I know that in C ++ there are a huge number of MD5 implementations. The problem here is that I am wondering if there is a comparison whose implementation is faster than the others. Since I use this MD5 hash function for files larger than 10 GB, speed is indeed a serious issue here.

Answer 1

I think the avakar point is trying to do: with modern processing power, the I / O speed of your hard disk is a bottleneck, not a hash calculation. Getting a more efficient algorithm will not help you, as it is not (probably) the slowest point.

If you do something special (like 1000 rounds), it may be different, but if you just calculate the hash of the file. You need to speed up IO input, not your math.

Answer 2

I don’t think it matters a lot (on the same hardware, but in reality the GPGPUs are different and possibly faster hardware for this kind of problem). The bulk of md5 is a rather complex loop of complex arithmetic operations. The important thing is the quality of the compiler optimization.

, . Linux mmap madvise readahead . , , ( SSD, ).

, md5 ? (md4 ..). -, .

Answer 3

, CUDA/OpenCL , . → CUDA/OpenCL.

Blocking ciphers are ideal candidates for this type of implementation.

You can also get a C implementation and grab a copy of the Intel C compiler and see how good it is. Vectorization extensions in Intel processors are striking in speed acceleration.

Answer 4

The table is available here:

http://www.golubev.com/gpuest.htm

looks likely your bottleneck will be your IO hard drive