Android ImageReader.acquireLatestImage returns invalid jpg

Question

I use the Android ImageReader class to get bitmaps from the MediaProjection.createVirtualDisplay method.

My code looks like this:

mProjection.createVirtualDisplay("test", width, height, density, flags, mImageReader.getSurface(), new VirtualDisplayCallback(), mHandler); mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() { @Override public void onImageAvailable(ImageReader reader) { Image image = null; try { image = mImageReader.acquireLatestImage(); final Image.Plane[] planes = image.getPlanes(); final ByteBuffer buffer = planes[0].getBuffer(); final byte[] data = new byte[buffer.capacity()]; buffer.get(data); final Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length); if (bitmap==null) Log.e(TAG, "bitmap is null"); } catch (Exception e) { if (image!=null) image.close(); } } }, mHandler); 

The problem is that BitmapFactory cannot decode data [] back to Bitmap, that is, BitmapFactory always returns null. The only messages I see from logcat relate to android_media_ImageReader.cpp and go as follows:

 D/ImageReader_JNI(1432): ImageReader_imageSetup: Receiving JPEG in HAL_PIXEL_FORMAT_RGBA_8888 buffer. W/ImageReader_JNI(1432): Image_getJpegSize: No JPEG header detected, defaulting to size=width=3891200 

The image object returned by the LatestImage retrieval method is not null, but is not a valid JPEG, I tried to check with the following test, which fails:

 if((buf [0] & 0xFF) == 0xFF && (buf[1] & 0xFF) == 0xD8 && (buf[2] & 0xFF) == 0xFF && (buf[3] & 0xFF) == 0xE0) Log.e(TAG, "is JPG"); else Log.e(TAG, "not a valid JPG"); 

The only thing I suspect at the moment is that the Android 5.0 emulator I'm testing with cannot call API calls.

Any ideas?

Answer 1

I checked the code of the first answer, but unfortunately it does not work on a real device. I am doing some investigations, and the following code solved my problem:

  mImgReader = ImageReader.newInstance(mWidth, mHeight, PixelFormat.RGBA_8888, 5); mSurface = mImgReader.getSurface();// mSurfaceView.getHolder().getSurface(); mImgReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() { @Override public void onImageAvailable(ImageReader reader) { Log.i(TAG, "in OnImageAvailable"); FileOutputStream fos = null; Bitmap bitmap = null; Image img = null; try { img = reader.acquireLatestImage(); if (img != null) { Image.Plane[] planes = img.getPlanes(); if (planes[0].getBuffer() == null) { return; } int width = img.getWidth(); int height = img.getHeight(); int pixelStride = planes[0].getPixelStride(); int rowStride = planes[0].getRowStride(); int rowPadding = rowStride - pixelStride * width; byte[] newData = new byte[width * height * 4]; int offset = 0; bitmap = Bitmap.createBitmap(metrics,width, height, Bitmap.Config.ARGB_8888); ByteBuffer buffer = planes[0].getBuffer(); for (int i = 0; i < height; ++i) { for (int j = 0; j < width; ++j) { int pixel = 0; pixel |= (buffer.get(offset) & 0xff) << 16; // R pixel |= (buffer.get(offset + 1) & 0xff) << 8; // G pixel |= (buffer.get(offset + 2) & 0xff); // B pixel |= (buffer.get(offset + 3) & 0xff) << 24; // A bitmap.setPixel(j, i, pixel); offset += pixelStride; } offset += rowPadding; } String name = "/myscreen" + count + ".png"; count++; File file = new File(Environment.getExternalStorageDirectory(), name); fos = new FileOutputStream(file); bitmap.compress(Bitmap.CompressFormat.JPEG, 100, fos); Log.i(TAG, "image saved in" + Environment.getExternalStorageDirectory() + name); img.close(); } } catch (Exception e) { e.printStackTrace(); } finally { if (null != fos) { try { fos.close(); } catch (IOException e) { e.printStackTrace(); } } if (null != bitmap) { bitmap.recycle(); } if (null != img) { img.close(); } } } }, mHandler); 

Answer 2

The code in the answer from @charlesjean works, but I would prefer not to generate each pixel myself. The best way to get an image from ImageReader is to simply create a bitmap image with the correct size and use the copyPixelsFromBuffer () method. Create an ImageReader as follows:

 mImageReader = ImageReader.newInstance(mWidth, mHeight, ImageFormat.RGB_565, 2); 

Then you can get the image from mImageReader using the following code.

 final Image.Plane[] planes = image.getPlanes(); final ByteBuffer buffer = planes[0].getBuffer(); int offset = 0; int pixelStride = planes[0].getPixelStride(); int rowStride = planes[0].getRowStride(); int rowPadding = rowStride - pixelStride * mWidth; // create bitmap bitmap = Bitmap.createBitmap(mWidth+rowPadding/pixelStride, mHeight, Bitmap.Config.RGB_565); bitmap.copyPixelsFromBuffer(buffer); image.close(); 

I described the screen capture process using the MediaProjection API, along with the mistakes that most people made when they received images from ImageReader in a blog post that you can read if you're interested.

Answer 3

I faced exactly your problem. My ImageReader is created like this:

 ImageReader.newInstance(mCaptureSize.getWidth(), mCaptureSize.getHeight(), ImageFormat.JPEG, 1); 

ImageReader above should only return compressed images, and they need to be unzipped. I acquire LatestImage () and then pass it through the following:

 ByteBuffer bBuffer = planes[0].getBuffer; bBuffer.rewind(); byte[] buffer = new byte[bBuffer.remaining()]; planes[0].getBuffer().get(buffer); Bitmap bitmap = BitmapFactory.decodeByteArray(buffer, 0, buffer.length); 

The key for me was rewinding ByteBuffer. Your code should work like this:

 mProjection.createVirtualDisplay("test", width, height, density, flags, mImageReader.getSurface(), new VirtualDisplayCallback(), mHandler); mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() { @Override public void onImageAvailable(ImageReader reader) { Image image = null; try { image = mImageReader.acquireLatestImage(); final Image.Plane[] planes = image.getPlanes(); final ByteBuffer buffer = planes[0].getBuffer(); buffer.rewind() final byte[] data = new byte[buffer.capacity()]; buffer.get(data); final Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length); if (bitmap==null) Log.e(TAG, "bitmap is null"); } catch (Exception e) { if (image!=null) image.close(); } } }, mHandler); 

I do not like to copy ByteBuffer through intermediate byte [], but the internal array is protected.

Tested working on 5.0.1 on HTC

Answer 4

If someone else stumbles upon this, the working code is as follows:

  mImageReader = ImageReader.newInstance(width, height, ImageFormat.JPEG, 5); mProjection.createVirtualDisplay("test", width, height, density, flags, mImageReader.getSurface(), new VirtualDisplayCallback(), mHandler); mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() { @Override public void onImageAvailable(ImageReader reader) { Image image = null; FileOutputStream fos = null; Bitmap bitmap = null; try { image = mImageReader.acquireLatestImage(); fos = new FileOutputStream(getFilesDir() + "/myscreen.jpg"); final Image.Plane[] planes = image.getPlanes(); final Buffer buffer = planes[0].getBuffer().rewind(); bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888); bitmap.copyPixelsFromBuffer(buffer); bitmap.compress(CompressFormat.JPEG, 100, fos); } catch (Exception e) { e.printStackTrace(); if (image!=null) image.close(); } finally { if (fos!=null) { try { fos.close(); } catch (IOException ioe) { ioe.printStackTrace(); } } if (bitmap!=null) bitmap.recycle(); } } }, mHandler); 

As you can see, I save the bitmap captured from ImageReader to the file stream stream, and this creates a valid jpeg file.

The messages I returned from android_media_ImageReader.cpp did not indicate any wrong behavior.

Hope this helps someone in the future!