How to play Opus encoded sound in Java?

Question

When playing back the decoded sound, I was able to create many sounds from gurgling to screeching demonic chants. The closest one sounds the same as fast-forward, and playback lasts about 15 seconds. I tried with a large combination of parameters for decoding methods and AudioSystem API, nothing works.

So what causes this sound distortion?

Opusinfo for this file shows the following:

Processing file "test.opus"... New logical stream (#1, serial: 00002c88): type opus Encoded with libopus 1.1 User comments section follows... ENCODER=opusenc from opus-tools 0.1.9 Opus stream 1: Pre-skip: 356 Playback gain: 0 dB Channels: 1 Original sample rate: 44100Hz Packet duration: 20.0ms (max), 20.0ms (avg), 20.0ms (min) Page duration: 1000.0ms (max), 996.8ms (avg), 200.0ms (min) Total data length: 1930655 bytes (overhead: 1.04%) Playback length: 4m:09.173s Average bitrate: 61.99 kb/s, w/o overhead: 61.34 kb/s Logical stream 1 ended 

This file plays correctly using VLC.

To decode a file, I am trying to use the following libraries:

• VorbisJava ( https://github.com/Gagravarr/VorbisJava/ ) - To pull frames from an OGG container

• LibJitsi ( https://jitsi.org/Projects/LibJitsi ) - which has a JNI shell for Opus, used to decode opus frames

SSCCE below

 package me.justinb.mediapad.audio; import org.gagravarr.ogg.OggFile; import org.gagravarr.ogg.OggPacket; import org.jitsi.impl.neomedia.codec.audio.opus.Opus; import javax.sound.sampled.*; import java.io.*; import java.nio.ByteBuffer; public class OpusAudioPlayer { private static int BUFFER_SIZE = 1024 * 1024; private static int INPUT_BITRATE = 48000; private static int OUTPUT_BITRATE = 44100; private OggFile oggFile; private long opusState; private ByteBuffer decodeBuffer = ByteBuffer.allocate(BUFFER_SIZE); private AudioFormat audioFormat = new AudioFormat(OUTPUT_BITRATE, 16, 1, true, false); public static void main(String[] args) { try { OpusAudioPlayer opusAudioPlayer = new OpusAudioPlayer(new File("test.opus")); opusAudioPlayer.play(); } catch (IOException e) { e.printStackTrace(); } } public OpusAudioPlayer(File audioFile) throws IOException { oggFile = new OggFile(new FileInputStream(audioFile)); opusState = Opus.decoder_create(INPUT_BITRATE, 1); System.out.println("Audio format: " + audioFormat); } private byte[] decode(byte[] packetData) { int frameSize = Opus.decoder_get_nb_samples(opusState, packetData, 0, packetData.length); int decodedSamples = Opus.decode(opusState, packetData, 0, packetData.length, decodeBuffer.array(), 0, frameSize, 0); if (decodedSamples < 0) { System.out.println("Decode error: " + decodedSamples); decodeBuffer.clear(); return null; } decodeBuffer.position(decodedSamples * 2); // 2 bytes per sample decodeBuffer.flip(); byte[] decodedData = new byte[decodeBuffer.remaining()]; decodeBuffer.get(decodedData); decodeBuffer.flip(); System.out.println(String.format("Encoded frame size: %d bytes", packetData.length)); System.out.println(String.format("Decoded frame size: %d bytes", decodedData.length)); System.out.println(String.format("Decoded %d samples", decodedSamples)); return decodedData; } public void play() { int totalDecodedBytes = 0; try { SourceDataLine speaker = AudioSystem.getSourceDataLine(audioFormat); OggPacket nextPacket = oggFile.getPacketReader().getNextPacket(); // Move to beginning of stream while ( !nextPacket.isBeginningOfStream()) { nextPacket = oggFile.getPacketReader().getNextPacket(); } speaker.open(); speaker.start(); while(nextPacket != null) { // Decode each packet byte[] decodedData = decode(nextPacket.getData()); if(decodedData != null) { // Write packet to SourceDataLine speaker.write(decodedData, 0, decodedData.length); totalDecodedBytes += decodedData.length; } nextPacket = oggFile.getPacketReader().getNextPacket(); } speaker.drain(); speaker.close(); System.out.println(String.format("Decoded to %d bytes", totalDecodedBytes)); } catch (Exception e) { e.printStackTrace(); } } } 

Answer 1

My particular problem seems to have been caused by an error in VorbisJava. Now I use J-Ogg, which handles container parsing without any problems. I'm sure someone will find this helpful.

This is the last code that shows how to play Opus encoded sound in Java:

 package me.justinb.mediapad.audio; import de.jarnbjo.ogg.FileStream; import de.jarnbjo.ogg.LogicalOggStream; import org.jitsi.impl.neomedia.codec.audio.opus.Opus; import javax.sound.sampled.AudioFormat; import javax.sound.sampled.AudioSystem; import javax.sound.sampled.SourceDataLine; import java.io.File; import java.io.IOException; import java.io.RandomAccessFile; import java.nio.ByteBuffer; import java.util.Collection; public class OpusAudioPlayer { private static int BUFFER_SIZE = 1024 * 1024; private static int INPUT_BITRATE = 48000; private static int OUTPUT_BITRATE = 48000; private FileStream oggFile; private long opusState; private ByteBuffer decodeBuffer = ByteBuffer.allocate(BUFFER_SIZE); private AudioFormat audioFormat = new AudioFormat(OUTPUT_BITRATE, 16, 1, true, false); public static void main(String[] args) { try { OpusAudioPlayer opusAudioPlayer = new OpusAudioPlayer(new File("test.opus")); opusAudioPlayer.play(); } catch (IOException e) { e.printStackTrace(); } } public OpusAudioPlayer(File audioFile) throws IOException { oggFile = new FileStream(new RandomAccessFile(audioFile, "r")); opusState = Opus.decoder_create(INPUT_BITRATE, 1); } private byte[] decode(byte[] packetData) { int frameSize = Opus.decoder_get_nb_samples(opusState, packetData, 0, packetData.length); int decodedSamples = Opus.decode(opusState, packetData, 0, packetData.length, decodeBuffer.array(), 0, frameSize, 0); if (decodedSamples < 0) { System.out.println("Decode error: " + decodedSamples); decodeBuffer.clear(); return null; } decodeBuffer.position(decodedSamples * 2); // 2 bytes per sample decodeBuffer.flip(); byte[] decodedData = new byte[decodeBuffer.remaining()]; decodeBuffer.get(decodedData); decodeBuffer.flip(); return decodedData; } public void play() { int totalDecodedBytes = 0; try { SourceDataLine speaker = AudioSystem.getSourceDataLine(audioFormat); speaker.open(); speaker.start(); for (LogicalOggStream stream : (Collection<LogicalOggStream>) oggFile.getLogicalStreams()) { byte[] nextPacket = stream.getNextOggPacket(); while (nextPacket != null) { byte[] decodedData = decode(nextPacket); if(decodedData != null) { // Write packet to SourceDataLine speaker.write(decodedData, 0, decodedData.length); totalDecodedBytes += decodedData.length; } nextPacket = stream.getNextOggPacket(); } } speaker.drain(); speaker.close(); System.out.println(String.format("Decoded to %d bytes", totalDecodedBytes)); } catch (Exception e) { e.printStackTrace(); } } } 

Answer 2

Looking at my code, I assume that you missed the meaning of "frame length". You accept the number of bytes, but the frame length directly depends on how the file was encoded.

An audio file recorded at 48,000 Hz has 48,000 samples per second. This sample audio is usually a 16-bit integer (2 bytes), which means you will have 48,000 * 2 bytes per second in unencrypted form (PCM-WAV).

An audio encoder such as opus will receive several sound samples at once and encode them in a packet. This is the frame. At 48 kHz these values ​​can be for opus 120, 240, 480, 960, 1920 and 2880.