How to encode long in Base64 in Python?

Question

In Java, I can encode BigInteger as:

 java.math.BigInteger bi = new java.math.BigInteger("65537L"); String encoded = Base64.encodeBytes(bi.toByteArray(), Base64.ENCODE|Base64.DONT_GUNZIP); // result: 65537L encodes as "AQAB" in Base64 byte[] decoded = Base64.decode(encoded, Base64.DECODE|Base64.DONT_GUNZIP); java.math.BigInteger back = new java.math.BigInteger(decoded); 

In C #:

 System.Numerics.BigInteger bi = new System.Numerics.BigInteger("65537L"); string encoded = Convert.ToBase64(bi); byte[] decoded = Convert.FromBase64String(encoded); System.Numerics.BigInteger back = new System.Numerics.BigInteger(decoded); 

How can I encode long integers in Python as Base64 encoded strings? What I tried so far gives results different from implementations in other languages ​​(so far I tried in Java and C #), in particular, it creates Base64-encoded strings in length encoded.

 import struct encoded = struct.pack('I', (1<<16)+1).encode('base64')[:-1] # produces a longer string, 'AQABAA==' instead of the expected 'AQAB' 

When using this Python code to create a Base64 encoded string, the resulting decoded integer in Java (for example) instead of 65537 creates instead of 16777472 . First of all, what am I missing?

Secondly, I need to find out manually which length format to use in struct.pack ; and if I try to encode a long number (greater than (1<<64)-1 ), the 'Q' format specification is too short to preserve the view. Does this mean that I have to do the presentation manually, or is there a documentless format specifier for the struct.pack function? (I am not forced to use a struct , but at first glance it seemed to do what I needed.)

Answer 1

Struct module:

... performs conversions between Python values ​​and C strings, represented as Python strings.

Since C does not have integers of infinite length, there are no functions for packing them.

But it is very easy to write. For example:

 def pack_bigint(i): b = bytearray() while i: b.append(i & 0xFF) i >>= 8 return b 

Or:

 def pack_bigint(i): bl = (i.bit_length() + 7) // 8 fmt = '<{}B'.format(bl) # ... 

And so on.

And of course, you will need the unpack function, for example jbatista, from the comments:

 def unpack_bigint(b): b = bytearray(b) # in case you're passing in a bytes/str return sum((1 << (bi*8)) * bb for (bi, bb) in enumerate(b)) 

Answer 2

Check out this page on converting integer to base64 .

 import base64 import struct def encode(n): data = struct.pack('<Q', n).rstrip('\x00') if len(data)==0: data = '\x00' s = base64.urlsafe_b64encode(data).rstrip('=') return s def decode(s): data = base64.urlsafe_b64decode(s + '==') n = struct.unpack('<Q', data + '\x00'* (8-len(data)) ) return n[0] 

Answer 3

It's a little late, but I decided that I would throw my hat in the ring:

 def inttob64(n): """ Given an integer returns the base64 encoded version of it (no trailing ==) """ parts = [] while n: parts.insert(0,n & limit) n >>= 32 data = struct.pack('>' + 'L'*len(parts),*parts) s = base64.urlsafe_b64encode(data).rstrip('=') return s def b64toint(s): """ Given a string with a base64 encoded value, return the integer representation of it """ data = base64.urlsafe_b64decode(s + '==') n = 0 while data: n <<= 32 (toor,) = struct.unpack('>L',data[:4]) n |= toor & 0xffffffff data = data[4:] return n 

These functions turn a long number of arbitrary size into / from base64 representation in base-end format.

Answer 4

Here is what can help. Instead of using struct.pack() I create a string of bytes to encode, and then cause the encoding to be BASE64. I did not write decoding, but obviously the decoder can restore an identical string of bytes, and the loop can restore the original value. I don’t know if fixed-size integers are needed (for example, always 128-bit), and I don’t know if you need Big Endian, so I left you a decoder.

In addition, encode64() and decode64() are @msc answers, but modified to work.

 import base64 import struct def encode64(n): data = struct.pack('<Q', n).rstrip('\x00') if len(data)==0: data = '\x00' s = base64.urlsafe_b64encode(data).rstrip('=') return s def decode64(s): data = base64.urlsafe_b64decode(s + '==') n = struct.unpack('<Q', data + '\x00'* (8-len(data)) ) return n[0] def encode(n, big_endian=False): lst = [] while True: n, lsb = divmod(n, 0x100) lst.append(chr(lsb)) if not n: break if big_endian: # I have not tested Big Endian mode, and it may need to have # some initial zero bytes prepended; like, if the integer is # supposed to be a 128-bit integer, and you encode a 1, you # would need this to have 15 leading zero bytes. initial_zero_bytes = '\x00' * 2 data = initial_zero_bytes + ''.join(reversed(lst)) else: data = ''.join(lst) s = base64.urlsafe_b64encode(data).rstrip('=') return s print encode(1234567890098765432112345678900987654321)