SHA1Managed.ComputeHash sometimes differs on different servers

Question

Background (you can skip this section)

I have a large amount of data (about 3 mb) that needs to be updated on several hundred machines. Some of the machines run C #, and some run Java. Data may change at any time and should be transmitted to customers within minutes. Data is delivered in Json format from 4 load-balanced servers. These 4 servers run ASP.NET 4.0 with Mvc 3 and C # 4.0.

The code that runs on 4 servers has a hash algorithm that hashes the Json response and then converts the hash to a string. This hash is provided to the client. Then, every few minutes, clients ping the server with a hash, and if the hash is out of date, a new Json object is returned. If the hash remains current, 304 is returned with the body of emptry.

Sometimes the hashes generated by the four boxes are inconsistent across all fields, which means that clients constantly download data (each request can go to another server).

Code sniper

Here is the code that is used to generate the hash.

internal static HashAlgorithm Hasher { get; set; } ... Hasher = new SHA1Managed(); ... Convert.ToBase64String(Hasher.ComputeHash(Encoding.ASCII.GetBytes(jsonString))); 

To try and debug the problem, I split it like this:

 Prehash = PreHashBuilder.ToString(); ASCIIBytes = Encoding.ASCII.GetBytes(Prehash); HashedBytes = Hasher.ComputeHash(ASCIIBytes); Hash = Convert.ToBase64String(HashedBytes); 

Then I added a route that spits out the above values ​​and uses Beyond Compare to compare the differences.

Byte arrays are converted to string format for use by BeyondCompare with:

 private static string GetString(byte[] bytes) { StringBuilder sb = new StringBuilder(); foreach (byte b in bytes) { sb.Append(b); } return sb.ToString(); } 

As you can see, the byte array is displayed as a sequence of bytes. He is not "transformed."

Problem

I found that the Prehash and ASCIIBytes values ​​were the same, but the HashedBytes values ​​were different - this meant that the hash was also different.

I restarted IIS websites on four server boxes several times and, when they had different hashes, compared the values ​​in BeyondCompare. In any case, the value of "HashedBytes" was different (SHA1Managed.ComputeHash (...) results)

Question

What am I doing wrong? The input to the ComputeHash function is identical. Is SHA1Managed machine dependent? This does not happen because half the time when 4 machines have the same hash.

I searched for StackOverFlow and Bing but could not find anyone else with this problem. The closest I could find were people with problems with their encoding, but I think I proved that encoding is not a problem.

Exit

I was hoping not to dump everything here because of how long this takes, but here's a sniper dump that I compare:

Hash: o1ZxBaVuU6OhE6De96wJXUvmz3M =
HashedBytes: 163861135165110831631611916022224717299375230207115
ASCIIBytes: .... Prehash: ...

When I compare two pages on different servers, the ASCII bytes are identical, but HashedBytes are not. The dump method that I use for bytes does not do any conversion, it just unloads each byte in the sequence. I could limit the bytes to the character '.' I guess.

Follow Up I changed the value of b.ToString (CultureInfo.InvariantCulture) and made the HashAlgorithm a local variable instead of a static property. I am waiting for the code to be deployed to the servers.

Answer 1

I am trying to duplicate the problem, but could not do it as soon as I made the SHA1Managed property of a local variable instead of global static.

The problem was multithreading. My code was thread safe, with the exception of the SHA1Managed class, which was marked as static. I assumed that SHA1Managed.ComputeHash would be thread safe from below, but apparently this does not mean that it is marked as internal.

To repeat, SHA1Managed.ComputeHash is not thread safe if internal static is checked.

MSDN Status:

 Any public static (Shared in Visual Basic) members of this type are thread safe. Any instance members are not guaranteed to be thread safe. 

I do not know why internal static behaves differently than public static.

I would put @pst as an answer and add a comment to clarify the problem, but @pst made a comment, so I cannot mark it as an answer.

Thanks for all your details.

Answer 2

Your GetString method could potentially produce different results on machines of different cultures, because StringBuilder.Append (byte) calls byte.ToString (CultureInfo.CurrentCulture). Try

 private static string GetString(byte[] bytes) { StringBuilder sb = new StringBuilder(); foreach (byte b in bytes) { sb.Append(b.ToString(CultureInfo.InvariantCulture)); } return sb.ToString(); } 

But using a method that does not use decimal string representations of byte values ​​would be better.

Answer 3

The problem is that your code is most likely messing around with leading 0s, use the following as your array to compare string code. it will give reliable results and is specifically designed to convert byte arrays to strings so that they can be transmitted between machines.

 using System.Runtime.Remoting.Metadata.W3cXsd2001; public byte[] StringToBytes(string value) { SoapHexBinary soapHexBinary = SoapHexBinary.Parse(value); return soapHexBinary.Value; } public string BytesToString(byte[] value) { SoapHexBinary soapHexBinary = new SoapHexBinary(value); return soapHexBinary.ToString(); } 

In addition, I would recommend that you check that JSON is not subtle, as this would create a completely different hash. For example, some cultures present the number "One thousand six hundred and seven" as 1,600.7 , 1 000.7 or even 1 600,7 (see this Wikipedia ).