Signed message using OpenSSL; cannot verify using android java

Question

I use SHA256 and RSA to sign a message on my Ubuntu machine using OpenSSL. My goal is to test this post on Android using Android Java.

The following commands were used in Ubuntu:

openssl genrsa -out private.pem 1024 openssl rsa -in private.pem -out public.pem -outform PEM -pubout echo 'foobar' > data.txt openssl dgst -sha256 < data.txt > hash openssl rsautl -sign -inkey private.pem -keyform PEM -in hash > signature openssl rsa -in private_key.pem -pubout -outform DER -out public_key.der openssl enc -base64 -in signature -out base64_signature 

Now I created the keys, signed this message, created a .der file for the public key, which should be available in Java, and encoded the message with Base64. Then I place the .der public key on my device and successfully upload the key to the PublicKey class.

This method is used to check the message:

 public static boolean verify(PublicKey publicKey,String data,String verification){ java.security.Signature sig; try { sig = java.security.Signature.getInstance("SHA256WithRSA"); sig.initVerify(publicKey); try { sig.update(verification.getBytes()); } catch (Exception e) { ... } if (!sig.verify(Base64.decode(data, Base64.DEFAULT))) { return false; } return true; } catch .... return false; } 

Parameters when calling the method:

 verify(PublicKey, Base64 encoded data in a String that is to be verified, "foobar"); 

Obviously, the check failed, but I can’t understand why. I assume that he should do something with the encoding (?).

---

Update! Therefore, I managed to write the results of Base64.decode(data, Base64.DEFAULT)) to a file and compare it with the original signature file using hexeditor. Completely different!

Answer 1

Java produces and expects to receive signatures in a slightly different form. The message hash must be encoded in DER, then supplemented by PKCS # 1 and only then signed with the private key. And Openssl has a command for this (because it is actually a standard procedure). Instead

 openssl dgst -sha256 < data.txt > hash openssl rsautl -sign -inkey private.pem -keyform PEM -in hash > signature 

you do

 openssl dgst -sha256 -binary -sign private.pem data.txt > signature 

Also note:

• your data.txt contains a new line, do not forget about it in the variable String verification
• sig.update(verification.getBytes()) must explicitly indicate the encoding - the same encoding used to populate the data.txt file, for example: sig.update(verification.getBytes("UTF-8"))

The rest of your commands / code looks fine.

---

UPD - reply to @GilCol about the differences:

The indentation is the same for both signed messages (PKCS # 1). But the messages are different.

When you use openssl dgst -sha256 < data.txt > hash , hash will contain (depending on the version of openssl):

 (stdin)= aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f 

or

 aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f 

This is just text, and you will sign this message with openssl rsautl -sign ... We can see that with openssl rsautl -verify ... :

 # raw message as-is - we can see the padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -raw -hexdump 0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff 00 61 ...............a # 00c0 - 65 63 30 37 30 36 34 35-66 65 35 33 65 65 33 62 ec070645fe53ee3b # 00d0 - 33 37 36 33 30 35 39 33-37 36 31 33 34 66 30 35 3763059376134f05 # your plain-text message 00e0 - 38 63 63 33 33 37 32 34-37 63 39 37 38 61 64 64 8cc337247c978add # 00f0 - 31 37 38 62 36 63 63 64-66 62 30 30 31 39 66 0a 178b6ccdfb0019f. # we can even see newline char (0a) at the end # strip the padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -pkcs -hexdump 0000 - 61 65 63 30 37 30 36 34-35 66 65 35 33 65 65 33 aec070645fe53ee3 0010 - 62 33 37 36 33 30 35 39-33 37 36 31 33 34 66 30 b3763059376134f0 0020 - 35 38 63 63 33 33 37 32-34 37 63 39 37 38 61 64 58cc337247c978ad 0030 - 64 31 37 38 62 36 63 63-64 66 62 30 30 31 39 66 d178b6ccdfb0019f 0040 - 0a . 

If you use openssl dgst -sha256 -binary < data.txt > hash to get the hash in binary (clean) form, and then sign it, the result will be better, but still not right:

 # raw message as-is - we can see the same padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -raw -hexdump 0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00d0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff 00 ................ 00e0 - ae c0 70 64 5f e5 3e e3-b3 76 30 59 37 61 34 f0 ..pd_.>..v0Y7a4. # the hash - now in binary form 00f0 - 58 cc 33 72 47 c9 78 ad-d1 78 b6 cc df b0 01 9f X.3rG.x..x...... # # strip the padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -pkcs -hexdump 0000 - ae c0 70 64 5f e5 3e e3-b3 76 30 59 37 61 34 f0 ..pd_.>..v0Y7a4. # just the hash, nothing else 0010 - 58 cc 33 72 47 c9 78 ad-d1 78 b6 cc df b0 01 9f X.3rG.x..x...... # 

But when you use openssl dgst -sha256 -sign ... , the message is different - it is now the standard ASN.1 structure for message digests (hashes). Let's get a look:

 # raw message as-is - we can see the same padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -raw -hexdump 0000 - 00 01 ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0010 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0020 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0030 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0040 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0050 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0060 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0070 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0080 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 0090 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00a0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00b0 - ff ff ff ff ff ff ff ff-ff ff ff ff ff ff ff ff ................ 00c0 - ff ff ff ff ff ff ff ff-ff ff ff ff 00 30 31 30 .............010 # 00d0 - 0d 06 09 60 86 48 01 65-03 04 02 01 05 00 04 20 ...`.He...... # the message - it different 00e0 - ae c0 70 64 5f e5 3e e3-b3 76 30 59 37 61 34 f0 ..pd_.>..v0Y7a4. # <- we can see the hash (in binary form) starting at this line 00f0 - 58 cc 33 72 47 c9 78 ad-d1 78 b6 cc df b0 01 9f X.3rG.x..x...... # # strip the padding $ openssl rsautl -in signature -pubin -inkey public.pem -verify -pkcs -hexdump 0000 - 30 31 30 0d 06 09 60 86-48 01 65 03 04 02 01 05 010...`.He.... 0010 - 00 04 20 ae c0 70 64 5f-e5 3e e3 b3 76 30 59 37 .. ..pd_.>..v0Y7 0020 - 61 34 f0 58 cc 33 72 47-c9 78 ad d1 78 b6 cc df a4.X.3rG.x..x... 0030 - b0 01 9f ... # parse the message and show the underlying ASN.1 structure $ openssl rsautl -in signature -pubin -inkey public.pem -verify -pkcs -asn1parse 0:d=0 hl=2 l= 49 cons: SEQUENCE 2:d=1 hl=2 l= 13 cons: SEQUENCE 4:d=2 hl=2 l= 9 prim: OBJECT :sha256 # type of hash 15:d=2 hl=2 l= 0 prim: NULL 17:d=1 hl=2 l= 32 prim: OCTET STRING 0000 - ae c0 70 64 5f e5 3e e3-b3 76 30 59 37 61 34 f0 ..pd_.>..v0Y7a4. # the hash in binary form 0010 - 58 cc 33 72 47 c9 78 ad-d1 78 b6 cc df b0 01 9f X.3rG.x..x...... # and no extra newline chars 

As you can see, only the last signature file had the correct ASN.1 structure, the previous two were just “arbitrary” messages signed with the RSA private key.