Update IV from Cipher after Byte Encryption

Question

I am working on a project that requires adding to the encrypted AES / CTR file. Now, since this is counter mode, I know that I can move the counter to any place and start reading in a place in the file. However, I am wondering if I have a way to get the current IV, which Cipher will have access to after using it.

Cipher c = Cipher.getInstance("AES/CTR/NoPadding"); SecretKeySpec keySpec = new SecretKeySpec(aeskey, "AES"); IvParameterSpec ivSpec = new IvParameterSpec(iv); c.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec); CipherOutputStream cipher_out = new CipherOutputStream(output, c); try { while (true) { cipher_out.write(input.readByte()); } } catch (EOFException e) { } byte curIV[] = c.getIV(); 

Instead, I find that curIV, instead of having the updated IV, has the same IV with which I switched to ivSpec for a start. Is there no way to get the current IV?

The idea is to store:

 <AES key><begin IV><current IV> 

in an asymmetrically encrypted file that can be decrypted, read, and we can start reading the encrypted AES file from the very beginning or add new data to our output file using the <current IV> that we saved.

Any other suggestions on how to implement this?

---

Java according to the documents I found uses the following (next to RFC3686 ):

 <NONCE><COUNTER> 

Like entering it in the CTR, and for updating the counter it is considered a large number of endian.

This is indicated as IvParameterSpec shown above.

---

Also, what I'm trying to return is a counter, whether we want to call it IV, or if we want to call it a counter, or a nonce + iv + counter.

---

Further information on the Sun CTR implementation: http://javamex.ning.com/forum/topics/questions-on-aes-ctr-mode

Answer 1

In an experiment with the "SunJCE" AES provider in CTR mode, a sentence published by NIST follows, where the initial value of the counter simply increases by one with each subsequent block. This is consistent with the general guidelines given in NIST SP 800-38A, Appendix B.1. when the number of additional bits, m , is the number of bits in the block, b .

This is contrary to RFC 3686. That is, the entire counter is incremented, not just a limited portion, as specified in RFC No. 3686.

You can find out the block index by counting blocks (starting from zero) or by measuring the length of the encryption text and performing integer division by block size. If these parameters seem too easy, you can also XOR use the last block of ciphertext with the corresponding plain text, decrypt this result and subtract IV to get the block index.

To add, simply set IV to the original IV plus the block index. If you write threads that can end with a partial block, you will have additional work to make the thread in the correct state.

 int BLOCK_SIZE = 16; BigInteger MODULUS = BigInteger.ONE.shiftLeft(BLOCK_SIZE * 8); ... /* Retrieve original IV. */ byte[] iv = ... ; /* Compute the index of the block to which data will be appended. */ BigInteger block = BigInteger.valueOf(file.length() / BLOCK_SIZE); /* Add the block to the nonce to find the current counter. */ BigInteger nonce = new BigInteger(1, iv); byte[] tmp = nonce.add(block).mod(MODULUS).toByteArray(); /* Right-justify the counter value in a block-sized array. */ byte[] ctr = new byte[BLOCK_SIZE]; System.arraycopy(tmp, 0, ctr, BLOCK_SIZE - tmp.length, tmp.length); /* Use this to initialize the appending cipher. */ IvParameterSpec param = new IvParameterSpec(ctr); 

Answer 2

(Bad form, I know, but I answer my question to document it for other people)

Counter update ...

 public static byte[] update_iv(byte iv[], long blocks) { ByteBuffer buf = ByteBuffer.wrap(iv); buf.order(ByteOrder.BIG_ENDIAN); long tblocks = buf.getLong(8); tblocks += blocks; buf.putLong(8, tblocks); return buf.array(); } 

Explanation AES / CTR-BE

This is the main idea. If you read erickson's answer to my question, you will see that IV is basically:

 <8 bytes nonce><8 bytes counter> 

The counter is stored in BIG_ENDIAN format, so if you have to pull out the counter in state 1, you will get the following:

 0x0 0x0 0x0 0x1 

Then, when it falls into the second block, it updates it to

 0x0 0x0 0x0 0x2 

etc., it can technically overflow in nonce, but it is not recommended to encrypt this data in the first place.

Now I personally create nonce / counter randomly. So it becomes even harder to guess, this is not a requirement.

What you are doing is letting you update counter how many blocks in the counter you want to go, no matter if you start with 0x1 or any other counter value (like myself).

Now, if we finish half the block or less, we need to make sure that we move forward in the AES-CTR by a couple of bytes, so we can just do:

 c.update(new byte[count]) 

where count is the number of characters, which is the distance to the block.

Explanation of my implementation

I have my keyfile stored on disk (in clear text, PLEASE DO NOT DO THIS!) Looks like this:

 <16 bytes AES key> <8 bytes nonce> <8 bytes counter> <8 bytes (long) block count> <4 byte partial block count> 

This gives us all the information we need to add something to the end of an already encrypted file without having to decrypt any content first. This is absolutely fantastic for log files that need to be encrypted, as well as any other content that can be transferred.

Testing

The way I tested that this really works is as follows:

 echo "1234567890ABCDEF" > file1 echo "0987654321ABCDEFGHIJKLMNOPQRSTUVWXYZ" > file2 cat file1 file2 > file3 

Now, if we encrypt file1 and then add file2 , we should get the same result as encrypting file3 , as long as we use the same / IV switch for both.

 javac AESTest.java # Compile the java file java AESTest key file1 append.aes java AESTest key file2 append.aes append 

Adding append tells the program to go into add mode and move the number of blocks forward and partially go to the next CTR cycle using the aforementioned c.update() method. From there, it starts encryption, as at any other time, and simply adds data to the output file.

 java AESTest key file3 noappend.aes 

Since my program will simply ignore the block counter / partial block, if you do not pass the append argument, it will just start encrypting the file using the same / IV switch as before.

Now, if we look at both files using HexEditor or vbindiff , we can verify that the two files are the same, but one of them has added content after the fact.

Full source ...

(Please note that this is the first time I have been programming in Java from a school that was a few years ago, please excuse the awful code)

Full source code for my program, where all this is implemented.

 import java.util.Random; import java.security.*; import javax.crypto.*; import javax.crypto.spec.*; import java.lang.String; import java.io.File; import java.io.*; import java.nio.ByteBuffer; import java.nio.ByteOrder; public class AESTest { public static byte[] update_iv(byte iv[], long blocks) { ByteBuffer buf = ByteBuffer.wrap(iv); buf.order(ByteOrder.BIG_ENDIAN); long tblocks = buf.getLong(8); tblocks += blocks; buf.putLong(8, tblocks); return buf.array(); } public static void main(String args[]) throws Exception { if (args.length < 3) { System.out.println("Not enough parameters:"); System.out.println("keyfile input output [append]"); return; } File keyfile = new File(args[0]); DataInputStream key_in; DataOutputStream key_out; DataInputStream input = new DataInputStream(new FileInputStream(args[1])); DataOutputStream output = null; byte key[] = new byte[16 + 16]; byte aeskey[] = new byte[16]; byte iv[] = new byte[16]; byte ivOrig[] = new byte[16]; long blocks = 0; int count = 0; if (!keyfile.isFile()) { System.out.println("Creating new key"); Random ranGen = new SecureRandom(); ranGen.nextBytes(aeskey); ranGen.nextBytes(iv); iv = update_iv(iv, 0); System.arraycopy(iv, 0, ivOrig, 0, 16); } else { System.out.println("Using existing key..."); key_in = new DataInputStream(new FileInputStream(keyfile)); try { for (int i = 0; i < key.length; i++) key[i] = key_in.readByte(); } catch (EOFException e) { } System.arraycopy(key, 0, aeskey, 0, 16); System.arraycopy(key, 16, iv, 0, 16); System.arraycopy(key, 16, ivOrig, 0, 16); if (args.length == 4) { if (args[3].compareTo("append") == 0) { blocks = key_in.readLong(); count = key_in.readInt(); System.out.println("Moving IV " + blocks + " forward"); iv = update_iv(iv, blocks); output = new DataOutputStream(new FileOutputStream(args[2], true)); // Open file in append mode } } } if (output == null) output = new DataOutputStream(new FileOutputStream(args[2])); // Open file at the beginnging key_out = new DataOutputStream(new FileOutputStream(keyfile)); Cipher c = Cipher.getInstance("AES/CTR/NoPadding"); SecretKeySpec keySpec = new SecretKeySpec(aeskey, "AES"); IvParameterSpec ivSpec = new IvParameterSpec(iv); c.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec); if (count != 0) { c.update(new byte[count]); } byte cc[] = new byte[1]; try { while (true) { cc[0] = input.readByte(); cc = c.update(cc); output.writeByte(cc[0]); if (count == 15) { blocks++; count = 0; } else { count++; } } } catch (EOFException e) { } cc = c.doFinal(); if (cc.length != 0) output.writeByte(cc[0]); // Before we quit, lets write our AES key, start IV, and current IV to disk for (int i = 0; i < aeskey.length; i++) key_out.writeByte(aeskey[i]); for (int i = 0; i < ivOrig.length; i++) key_out.writeByte(ivOrig[i]); System.out.println("Blocks: " + blocks); System.out.println("Extra: " + count); key_out.writeLong(blocks); key_out.writeInt(count); } } 

Answer 3

You are right that getIV returns the original IV, not the current one, after some encryption / decryption has occurred.

In Java, 16 bytes transferred to the AES block cipher in CTR mode are IV plus the number of the current block (added as if both were 16-byte bonuses in big-endian format, see code below).

Be sure to read fooobar.com/questions/750835 / ... , it has a lot of good tips to avoid CTR mode security errors (summary: NEVER encrypt twice with the same IV).

For your use case, you just need to keep the beginning of IV plus the block number (and even the block number if you can get the file size in another way). You can calculate the current IV from this to further encrypt or decrypt an arbitrary search.

Code to calculate the correct IV for use, taking into account the block number (the first block is 0):

 int block = ...; byte[] iv = ...; byte[] blockbytes = new byte[16]; for (int i = 0; i < 4; i++) blockbytes[15 - i] = (byte)(block >> 8*i); int carry = 0; for (int i = 15; i >= 0; i--) { int sum = (iv[i] & 255) + (blockbytes[i] & 255) + carry; iv[i] = (byte)sum; carry = sum >> 8; } 

Caution: I got this from figuring out what the code does - I did not see it in the specification, so it is possible that the algorithm is vendor-specific.

Here you can try a more complete test program:

 import java.math.*; import javax.crypto.*; import javax.crypto.spec.*; public class TestCTR { static SecretKeySpec keySpec = new SecretKeySpec(new BigInteger("112233445566778899aabbccddeeff00", 16).toByteArray(), "AES"); static IvParameterSpec ivSpec = new IvParameterSpec(new BigInteger("66778899aaffffffffffffffffffffff", 16).toByteArray()); public static void main(String[] args) throws Exception { byte[] plaintext = new byte[256]; for (int i = 0; i < 256; i++) plaintext[i] = (byte)i; // encrypt with CTR mode byte[] ciphertext = new byte[256]; Cipher c = Cipher.getInstance("AES/CTR/NoPadding"); c.init(Cipher.ENCRYPT_MODE, keySpec, ivSpec); c.doFinal(plaintext, 0, 256, ciphertext, 0); // decrypt, implementing CTR mode ourselves Cipher b = Cipher.getInstance("AES/ECB/NoPadding"); b.init(Cipher.ENCRYPT_MODE, keySpec); for (int block = 0; block < 16; block++) { byte[] iv = ivSpec.getIV(); int carry = 0; byte[] blockbytes = new byte[16]; for (int i = 0; i < 4; i++) blockbytes[15 - i] = (byte)(block >> 8*i); for (int i = 15; i >= 0; i--) { int sum = (iv[i] & 255) + (blockbytes[i] & 255) + carry; iv[i] = (byte)sum; carry = sum >> 8; } b.doFinal(iv, 0, 16, iv, 0); for (int i = 0; i < 16; i++) plaintext[block*16+i] = (byte)(ciphertext[block*16+i] ^ iv[i]); } // check it for(int i = 0; i < 256; i++) assert plaintext[i] == (byte)i; } }