I have been struggling with this problem for more than a week, and most likely this is a mistake in the Qualcomm GPU encoder / hardware video encoder . Since we are forced to release the application, and their developer forums did not provide any feedback, I also post it here, hoping that someone can provide some tips or, better yet, a workaround, so the error in the encoder does not work.
The application is encoded from the surface. When certain images are rendered on the surface, the encoder fails (100%) in the same place (when the frame that is currently being encoded is the key frame). Logcat exits the encoder at the point of failure (with the last lines repeating at a very high speed):
11-26 11:41:33.312: E/OMX-VENC-720p(25949): ERROR: async_msg_process() - Error statuscode = 1 11-26 11:41:33.312: E/ACodec(29317): [OMX.qcom.video.encoder.avc] ERROR(0x80001009) 11-26 11:41:33.312: E/MediaCodec(29317): Codec reported an error. (omx error 0x80001009, internalError -2147483648) 11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: venc_use_buf:set input buffer failed 11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: in dev_use_buf 11-26 11:41:33.362: E/OMX-VENC-720p(25949): ERROR: empty_buffer_done() failed! 11-26 11:41:33.372: E/OMX-VENC-720p(25949): m_fbd_count at o/p flush = 306 11-26 11:41:33.372: E/OMX-VENC-720p(25949): m_etb_count at i/p flush = 313 11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: ioctl VEN_IOCTL_CMD_FILL_OUTPUT_BUFFER failed 11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: dev_fill_buf() Failed 11-26 11:41:33.372: E/OMX-VENC-720p(25949): ERROR: FTBProxy() failed!
I can also replicate it to Grafika by simply replacing the drawing of two geometric shapes with a drawing in the Record GL application, one of the problematic images, full screen, in landscape mode and changing the bitrate to a higher value (7 Mbit / s). Encoder crashes faster to increase bitrate.
Here is one image that seems to easily break it, and here is RecordFBOActivity.java with the necessary changes.
With Grafika, I tested, and the encoder crashes both on the Samsung S4, and on the international version, as well as on the original Nexus 4. With our software, which is a little more complicated than a simple image, it still crashes on both. Not tested on other Adreno devices. On a Samsung S3 with a Mali 400 GPU, it works great.
At a speed of 4 Mbit / s, the encoder in our application still falls on both S4 and N4, but later. Graphics fall on N4 in the same place, but not on S4.
EDIT: according to the comments below, it can also be played back when encoding the same image from the buffer. Various tests seem to narrow down the conditions for their playback on: h264 hw encoder on Qualcomm devices, encoding a still image for many frames (this determines very low bitrates in the encoder due to similar frames), if an error occurs when encoding a key frame, an error appears only when encoding certain images, which seem to have more detailed information, i.e. many bits are required for internal coding).