AudioTrack - convert short arrays to byte arrays using jlayer (java mp3 decoder)

Question

I am using jLayer to decode MP3 data using this call:

SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream); 

This call, which returns the decoded data, returns an array of short []. output.getBuffer();

When I call AudioTrack write () with this method, it works fine when I scroll the file:

 at.write(output.getBuffer(), 0, output.getBuffer().length); 

However, when I convert the short [] array to the byte [] array using any of the methods in this answer: https://stackoverflow.com/a/167449/2326/2326326

 at.write(output.getBuffer(), 0, output.getBuffer().length); 

becomes:

 byte[] array = ShortToByte_Twiddle_Method(output.getBuffer()); at.write(array, 0, array.length); 

Am I doing something wrong and what can I do to fix it? Unfortunately, I need the pcm data to be in a byte array for another third-party library that I'm using. The file has a value of 22 kHz, if that matters, and this is how it is created:

 at = new AudioTrack(AudioManager.STREAM_MUSIC, 22050, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT, 10000 /* 10 second buffer */, AudioTrack.MODE_STREAM); 

Thank you so much in advance.

Edit: This is how I now create the AudioTrack variable. So for files with a frequency of 44 kHz, the send value is 44100, and for files with a frequency of 22 kHz - 22050.

 at = new AudioTrack(AudioManager.STREAM_MUSIC, decoder.getOutputFrequency(), decoder.getOutputChannels() > 1 ? AudioFormat.CHANNEL_OUT_STEREO : AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, 10000 /* 10 second buffer */, AudioTrack.MODE_STREAM); 

This is the decoding method:

 public byte[] decode(InputStream inputStream, int startMs, int maxMs) throws IOException { ByteArrayOutputStream outStream = new ByteArrayOutputStream(1024); float totalMs = 0; boolean seeking = true; try { Bitstream bitstream = new Bitstream(inputStream); Decoder decoder = new Decoder(); boolean done = false; while (!done) { Header frameHeader = bitstream.readFrame(); if (frameHeader == null) { done = true; } else { totalMs += frameHeader.ms_per_frame(); if (totalMs >= startMs) { seeking = false; } if (!seeking) { // logger.debug("Handling header: " + frameHeader.layer_string()); SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream); short[] pcm = output.getBuffer(); for (short s : pcm) { outStream.write(s & 0xff); outStream.write((s >> 8) & 0xff); } } if (totalMs >= (startMs + maxMs)) { done = true; } } bitstream.closeFrame(); } return outStream.toByteArray(); } catch (BitstreamException e) { throw new IOException("Bitstream error: " + e); } catch (DecoderException e) { throw new IOException("Decoder error: " + e); } } 

Here's how it sounds (wait a few seconds): https://vimeo.com/60951237 (and this is the actual file: http://www.tonycuffe.com/mp3/tail%20toddle.mp3 )

Edit: I would love to split the generosity, but instead I gave the generosity to Bill and the accepted answer to Neil. Both of them were of great help. For those who wondered, I ended up rewriting my own Sonic code, which helped me move through this process.

Answer 1

As @Bill Pringlemeir says, the problem is that your conversion method is not actually converted. Short is a 16-bit number; the byte is an 8-bit number. The selected method does not convert the contents of the shorts (i.e., from 16 bits to 8 bits for the content), it changes the way that the same set of bits is stored. As you say, you need something like this:

 SampleBuffer output = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream); byte[] array = MyShortToByte(output.getBuffer()); at.write(array, 0, array.length); 

An approach

@Bill Pringlemeir is equivalent to dividing all short circuits by 256 to ensure they match in the byte range:

 byte[] MyShortToByte(short[] buffer) { int N = buffer.length; ByteBuffer byteBuf = ByteBuffer.allocate(N); while (N >= i) { byte b = (byte)(buffer[i]/256); /*convert to byte. */ byteBuf.put(b); i++; } return byteBuf.array(); } 

This will work, but will probably give you very quiet, sharp tones. If you can afford the processing time, a two-pass approach is likely to give better results:

 byte[] MyShortToByte(short[] buffer) { int N = buffer.length; short min = 0; short max = 0; for (int i=0; i<N; i++) { if (buffer[i] > max) max = buffer[i]; if (buffer[i] < min) min = buffer[i]; } short scaling = 1+(max-min)/256; // 1+ ensures we stay within range and guarantee no divide by zero if sequence is pure silence ... ByteBuffer byteBuf = ByteBuffer.allocate(N); for (int i=0; i<N; i++) { byte b = (byte)(buffer[i]/scaling); /*convert to byte. */ byteBuf.put(b); } return byteBuf.array(); } 

Again beware of signature / unsigned issues. The above works, signed-> signed and unsigned-> unsigned; but not between them. You may be reading signed shorts (-32768-32767), but you need to output unsigned bytes (0-255), ...

If you can afford the processing time, a more accurate (smoother) approach would be to go through the float (this also circumvents the signature / unsigned issue):

 byte[] MyShortToByte(short[] buffer) { int N = buffer.length; float f[] = new float[N]; float min = 0.0f; float max = 0.0f; for (int i=0; i<N; i++) { f[i] = (float)(buffer[i]); if (f[i] > max) max = f[i]; if (f[i] < min) min = f[i]; } float scaling = 1.0f+(max-min)/256.0f; // +1 ensures we stay within range and guarantee no divide by zero if sequence is pure silence ... ByteBuffer byteBuf = ByteBuffer.allocate(N); for (int i=0; i<N; i++) { byte b = (byte)(f[i]/scaling); /*convert to byte. */ byteBuf.put(b); } return byteBuf.array(); } 

Answer 2

The problem is converting short to byte . The byte conversion link saves all information, including the upper and lower parts of the byte . When you convert from 16 bits to 8 bits PCM , you must discard the low byte. My Java skills are weak, so the following may not work verbatim. See also: short conversion to bytes.

 ByteBuffer byteBuf = ByteBuffer.allocate(N); while (N >= i) { /* byte b = (byte)((buffer[i]>>8)&0xff); convert to byte. native endian */ byte b = (byte)(buffer[i]&0xff); /*convert to byte; swapped endian. */ byteBuf.put(b); i++; } 

This is the next conversion,

  AAAA AAAA SBBB BBBB -> AAAA AAAA, +1 if S==1 and positive else -1 if S==1 

A is the bit that is saved. B is the discarded bit, and S is the bit that you can use for rounding. Rounding is not required, but may seem a little better. Basically, 16 bit PCM has a higher resolution than 8 bit PCM. You lose these bits when the conversion is done. The short to byte routine tries to save all the information.

Of course, you must specify the sound library that you use 8-bit PCM . My suggestion,

 at = new AudioTrack(AudioManager.STREAM_MUSIC, 22050, AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_8BIT, 10000 /* 10 second buffer */, AudioTrack.MODE_STREAM); 

If you can use 16bit PCM to play sound, you need to invert and convert 8bit PCM from the library to 16bit PCM for playback. Also note that usually 8bit samples are often NOT direct PCMs, but u-law or a-law is encoded. If a third-party 3 ^rd library uses these formats, the conversion is different, but you should be able to encode it from links in wikipedia.

NOTE. I did not include the rounding code as overflow and sign handling will complicate the response. You should check overflow (Ie, 0x8f + 1 gives 0xff or 255 + 1, giving -1). However, I suspect the library is not a direct 8bit PCM .

See also: Alsa PCM Review, PCM Multimedia Recording Wiki - Ultimately, Android uses ALSA for sound.

Other factors that must be correct for the raw PCM buffer are the sampling rate, the number of channels (stereo / mono), the PCM format, including bits, companding, small / large endian, and pattern rotation. Strike>

EDIT: After some research, the JLayer decoder usually returns a big endian 16 bits. The sound filter accepts byte , but threatens them like a 16bit little endian from below. Finally, the AudioTrack class expects 16 bits of little endian below it. I believe that for some reason, the JLayer mp3 decoder will return 16 bits of little endian . The decode() method in the question performs a byte replacement of 16-bit values. In addition, the placed sound sounds as if the bytes are reversed.

 public byte[] decode(InputStream inputStream, int startMs, int maxMs, bool swap) throws IOException { ... short[] pcm = output.getBuffer(); for (short s : pcm) { if(swap) { outStream.write(s & 0xff); outStream.write((s >> 8) & 0xff); } else { outStream.write((s >> 8) & 0xff); outStream.write(s & 0xff); } } ... 

For 44k mp3, you call the routine with swap = true; . For 22k mp3 swap = false . This explains all reported phenomena. I don’t know why the JLayer mp3 decoder sometimes outputs big endian and other little endian times. I think it depends on the original mp3, not the sample rate.