Geek Answers Handbook

Memoization: parameter concatenation or md5 hash execution?

I am adding memoization to several functions. These functions accept 2-3 string parameters (object names), an optional int parameter (record identifier), and a logical parameter (including remote records). Each combination of parameters guarantees a unique result (so it’s worth caching).

I am wondering if it will be faster to concatenate the given parameters ( $param1 . $param2 . $param3etc.) and use this as an array key, or take the same concatenated string and use the md5 hash as the key. The string length of the concatenated parameters is from 20 to 32 characters in 99% of cases (on average about 27), while the md5 hash is always 32 characters.
Edit : md5 hash is only 16 bytes, not 32. Thanks Mjh.

I am inclined to the first option, since it:

  • saves the cost of executing the md5 hash
  • it usually stores a few bytes of memory (27 medium versus 32 hashes) (Mjh pointed out that this is not true: md5 is only 16 bytes) and
  • since the md5 hash is just another line, it will usually be faster to compare a shorter line

The only reason I doubt it is because the vast majority of memoization functions seem to use hashes (md5), so I wonder if I am missing something.

Thanks in advance.

PS I forgot to mention: I separate the individual parameters with a symbol #that can never naturally occur in any of the parameters.

PPS No comments ankhzet is the best solution, because my line is almost unique to the beginning: crc32($paramString). Low memory and very fast checksum calculation function.

Crc32 () performance testing

script, 4 1 key => value. values 4 . keys , , 2 crc32() .

$test1Array = [];
$start1 = microtime(true);
for ($i = 0; $i < 1000000; $i++)
{
    $test1Array[crc32("pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1)] = "test " . $i;
}
$end1 = microtime(true);

$test2Array = [];
$start2 = microtime(true);
for ($j = 0; $j < 1000000; $j++)
{
    $test2Array[crc32("pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1)] = "test " . $j;
}
$end2 = microtime(true);

$test3Array = [];
$start3 = microtime(true);
for ($x = 0; $x < 1000000; $x++)
{
    $test3Array["pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1] = "test " . $x;
}
$end3 = microtime(true);

$test4Array = [];
$start4 = microtime(true);
for ($y = 0; $y < 1000000; $y++)
{
    $test4Array["pagemanagement" . "#" . "staticblocktype" . "#" . $i . "#" . 1] = "test " . $y;
}
$end4 = microtime(true);

3 :
1: 3.9902291297913
2: 3.6312079429626
3: 0.91605305671692
4: 0.91405177116394

1: 3.9842278957367
2: 3.6172070503235
3: 0,91405200958252
4: 0.918053150177

1: 3.9842278957367
2: 3.6282079219818
3: 0.91205215454102
4: 0.91605186462402

"Test 2" "Test 4" ( " 1", , ), 3.6255409717560 "Test 2" 0.9160522619883 "Test 4", 2,7094887097677 (2.7094887097677/1000000) = 0,0000027094887 2,72 .

, , 4 crc32() , 27 . , 1 , 23 .

md5():
1: 4.2855787277221
2: 3,8108838399251
, md5() crc32(). , crc32() - 4 md5() 16.

: , 50-200 , , ~ 135 -540 ~ 1150-4600 .

- / , .

+4
3

md5-crc32-sha1- AMD 2x2.3 PHP7:

function probe($label, $times, $callback) {
    $mem = memory_get_usage();
    $start = microtime(true);
    $array = $callback($times);
    $time = microtime(true) - $start;
    $mem = sprintf('%.3f', (memory_get_usage() - $mem) / 1024 / 1024);
    return "$label:  $time s, $mem MB";
}

$times = 1000000;

$run1 = probe('String key', $times, function ($times) {
    $a = [];
    while ($times-- > 0) {
        $a["pagemanagement" . "#" . "staticblocktype" . "#" . $times . "#" . 1] = "test " . $times;
    }
    return $a;
});

$run2 = probe('CRC32 key', $times, function ($times) {
    $a = [];
    while ($times-- > 0) {
        $a[crc32("pagemanagement" . "#" . "staticblocktype" . "#" . $times . "#" . 1)] = "test " . $times;
    }
    return $a;
});

$run3 = probe('MD5 key', $times, function ($times) {
    $a = [];
    while ($times-- > 0) {
        $a[md5("pagemanagement" . "#" . "staticblocktype" . "#" . $times . "#" . 1)] = "test " . $times;
    }
    return $a;
});

$run4 = probe('SHA1 key', $times, function ($times) {
    $a = [];
    while ($times-- > 0) {
        $a[sha1("pagemanagement" . "#" . "staticblocktype" . "#" . $times . "#" . 1)] = "test " . $times;
    }
    return $a;
});

echo join("<br/>\n", [
    $run1,
    $run2,
    $run3,
    $run4,
    ]);

: 1.2421879768372 s, 111.923 MB
CRC32 : 1.3447260856628 s, 58.517 MB
MD5 : 2.1748039722443 s, 111.923 MB
SHA1: 2.2480459213257 , 119.552

, MD5 , crc32, crc32 .

( x10 , 64 ) PHP5.5 + - PHP7 hhvm.

: ( ​​ ). , crc32 1,5-2 .

: sha1. havier, md5.

: , / .

+2

:

$cache[$paramString] = $value;  // or
$cache[crc32($paramString)] = $value;

PHP , unsigned long. $paramString . , , - crc32() md5(), , $paramString .

: https://nikic.imtqy.com/2011/12/12/How-big-are-PHP-arrays-really-Hint-BIG.html

+1

, toString , , , - . , , .

.

You can use sha1 (), which, although it is more complex than md5 (), works much faster (at least the last time I checked).

But it smells like premature optimization.

-1
source

More articles:


All Articles