Is it possible to use lockCanvas () in an onPreviewFrame callback?

Question

I am fighting the IllegalArgumentException when using lockCanvas () in onPreviewFrame.

Basically, I want to get to capture the frame, process it and draw it directly on the SurfacePreview.

here is my onPreviewFrame code

@Override public void onPreviewFrame(byte[] data, Camera camera) { Canvas canvas = null; if (mHolder == null) { return; } int mImgFormat = mCamera.getParameters().getPreviewFormat(); try { canvas = mHolder.lockCanvas(); canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR); mHolder.unlockCanvasAndPost(canvas); } catch (Exception e) { e.printStackTrace(); } finally { if (canvas != null) { mHolder.unlockCanvasAndPost(canvas); } } } 

I read a lot of documentation and topics about cameras in android, and I believe that locking the surface onto which frames are drawn is not possible, since it goes beyond application management. It's true?

One possible solution is to draw on top of the surface view with a different view, but I want to save my processed frames (some red detection and color corrections that may take a lot of time will be detected), performed in synchronization with the preview. When the camera continues to spit frames with good fps, my calculations will complicate the time, and in the worst case draw an overlay with dozens of frames.

I have studied the source of openCV, and for me it looks like they managed to do this in CameraBridgeViewBase.java:

 protected void deliverAndDrawFrame(CvCameraViewFrame frame) { Mat modified; if (mListener != null) { modified = mListener.onCameraFrame(frame); } else { modified = frame.rgba(); } boolean bmpValid = true; if (modified != null) { try { Utils.matToBitmap(modified, mCacheBitmap); } catch(Exception e) { Log.e(TAG, "Mat type: " + modified); Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight()); Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage()); bmpValid = false; } } if (bmpValid && mCacheBitmap != null) { Canvas canvas = getHolder().lockCanvas(); if (canvas != null) { canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR); canvas.drawBitmap(mCacheBitmap, (canvas.getWidth() - mCacheBitmap.getWidth()) / 2, (canvas.getHeight() - mCacheBitmap.getHeight()) / 2, null); if (mFpsMeter != null) { mFpsMeter.measure(); mFpsMeter.draw(canvas, 20, 30); } getHolder().unlockCanvasAndPost(canvas); } } } 

I either missed something, or I'm too old for this :)

Also, this is my first post here, so hello everyone :)

Answer 1

In fact, the openCV camera plays a little trick. In the CameraBridgeViewBase.java and JavaCameraView.java file. I found that the SurfaceTexture class (from api 11) is used to receive frames from the camera. The advantage of SurfaceTexture is that you do not always need to draw it on the screen (it may be kept in memory). The SurfaceView class is then responsible for drawing the processed frame from the onPreviewFrame function. Please note that SurfaceView is not attached to the camera. Hope this helps.