Using a SurfaceTexture populated with a camera preview as a Renderscript tab Distribution in a jelly bean

In Bean Jelly, is it possible to create a Renderscript distribution from a SurfaceTexture filled with a camera preview? I am building my application from the Android source tree, so I use the @hide APIs such as Allocation.setSurfaceTexture (). However, I would like to avoid using the legacy Graphics Graphics APIs. A similar question here did not receive a complete answer and was not specific to JB.

When trying the code below, the following problems occur:

  • Renderscript data is always null
  • To call the onFrameAvailable callback for multiple calls, I need to update TexImage (), because when I call Allocation.ioReceive () it doesn’t get the callback anymore after the first time, and there is an “invalid EGLDisplay” in logcat, However, I thought ioReceive () is the way to go - it is internally also updateTexImage ().
  • Supported distribution types include RGBA8888, but not NV21 (which is the camera preview format), how can the RS code handle the data this way?

(I know that the device I'm working with supports the requested VGA resolution).

public class SampleRSCPCActivity extends Activity implements SurfaceTexture.OnFrameAvailableListener { final static int DO_KERNEL = 0; private static final String TAG="SAMPLERSCP"; private static Camera mCamera; private Camera.Parameters mParams; private int mFrameWidth, mFrameHeight; private static SurfaceTexture mST; private RenderScript mRS; private Allocation mInAllocation; private Allocation mOutAllocation; private ScriptC_mono mScript; public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); Log.i(TAG, "onCreate()"); createGUI(); createCamera(); createRSEnvironment(); } public void onPause() { Log.i(TAG, "onPause"); mCamera.stopPreview(); mCamera.release(); mCamera = null; super.onPause(); } private void createCamera() { mCamera = Camera.open(); mParams = mCamera.getParameters(); mFrameWidth = 640; mFrameHeight = 480; mParams.setPreviewSize(mFrameWidth, mFrameHeight); mParams.setPreviewFormat(ImageFormat.NV21); mCamera.setParameters(mParams); } private void createRSEnvironment () { mRS = RenderScript.create(this); mScript = new ScriptC_mono(mRS, getResources(), R.raw.mono); Type.Builder b = new Type.Builder(mRS, Element.U8(mRS)); int usage = Allocation.USAGE_SCRIPT | Allocation.USAGE_IO_INPUT; mInAllocation = Allocation.createTyped(mRS, b.setX(mFrameWidth).setY(mFrameHeight).create(), usage); mOutAllocation = Allocation.createTyped(mRS, b.setX(mFrameWidth).setY(mFrameHeight).create()); Log.i(TAG, "Getting SurfaceTexture from input Allocation"); mST = mInAllocation.getSurfaceTexture(); mST.setOnFrameAvailableListener(this); try { Log.i(TAG, "Setting SurfaceTexture for camera preview"); mCamera.setPreviewTexture(mST); Log.i(TAG, "Starting preview"); mCamera.startPreview(); } catch (IOException e) { Log.e(TAG, "Oops, something got wrong with setting the camera preview texture"); } } private void createGUI() { requestWindowFeature(Window.FEATURE_NO_TITLE); getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN); setContentView(R.layout.main); } private Handler handler = new Handler() { @Override public void handleMessage(Message msg) { if (msg.what == DO_KERNEL) Log.i(TAG, "Calling RS kernel"); mST.updateTexImage(); // mInAllocation.ioReceive(); mScript.forEach_root(mInAllocation, mOutAllocation); Log.i(TAG, "Finishing RS"); mRS.finish(); Log.i(TAG, "Ok"); } }; public void onFrameAvailable(SurfaceTexture st) { Log.i(TAG, "onFrameAvailable callback"); handler.sendEmptyMessage(DO_KERNEL); } 

}

The RS code is pretty simple, just trying to detect non-zero data:

 void root(const uchar *v_in, uchar *v_out, uint32_t x, uint32_t y) { if (*v_in != 0) rsDebug("input data non null !", *v_in); *v_out = (x / 640) * 255; 

}

+7
source share
1 answer

Following my own question:

It turns out that reading NV21 buffers from a SurfaceTexture filled with a camera is not possible. I had to change the Android source code to experiment with this (for experts: get the current SurfaceTexture buffer and then lock it to get the real buffer pointer). I did this in the RS driver rsdAllocationIoReceive ()). It would be great to avoid running a backup copy from the camera to RS.

The latest version of JB (version MR1) has a test application called LivePreview that performs RS processing on the camera preview. However, it uses the application-specific preview callback buffers, which are then copied to the input distribution. This interestingly uses the new ScriptIntrinsicRgbToYuv class to convert colors. The bulk of the conversion is the manual-encoded Neon assembly, so it’s probably pretty fast.

Perhaps even the Nexus 10 (which has a Mali-enabled RS driver) does this on the GPU, I would love to play this device. (Donors are welcome ;-)!)

+6
source

All Articles