Video Capture on AR (Vuforia) with Unity on Android

Question

I am trying to add a video capture function to an AR application. Basically record what is happening on the screen and save it as a video (allowing the user to share it). AR APP is written with the Vuforia-Unity SDK. We have successfully achieved this on the iOS platform.

However, we have great difficulties with the same thing on the Android platform. (we hope to achieve this without rooting the device)

Following is our progress:

• The camera works in the Vuforia program, I can’t access the video stream.

• I tried to capture a screeshot image of each frame, and then combine them with some video output; but the frame rate is very low (less than 1 frame per second). It takes 700 ms to capture a screenshot.

Am I thinking of the wrong direction? Any help would be greatly appreciated! Many thanks! Isaac

Below is my test code:

public void acquireScreenshot() { DisplayMetrics metrics = new DisplayMetrics(); WindowManager WM = (WindowManager) MainActivity.this.getSystemService(Context.WINDOW_SERVICE); Display display = WM.getDefaultDisplay(); display.getMetrics(metrics); int height = metrics.heightPixels; // screen height int width = metrics.widthPixels; // screen width int pixelformat = display.getPixelFormat(); PixelFormat localPixelFormat1 = new PixelFormat(); PixelFormat.getPixelFormatInfo(pixelformat, localPixelFormat1); int deepth = localPixelFormat1.bytesPerPixel; byte[] arrayOfByte = new byte[height* width* deepth]; long tmp = System.currentTimeMillis(); try { for(int i = 0 ; i < 10 ; i++){ InputStream localInputStream = readAsRoot(); DataInputStream localDataInputStream = new DataInputStream( localInputStream); android.util.Log.e("mytest", "-----read start-------"); localDataInputStream.readFully(arrayOfByte); android.util.Log.e("mytest", "-----read end-------time = " + (System.currentTimeMillis() -tmp )); localInputStream.close(); File mid = new File("/mnt/sdcard/AAA"); if(!mid.exists()){ mid.mkdir(); } FileOutputStream out = new FileOutputStream(new File( "/mnt/sdcard/AAA/"+System.currentTimeMillis()+".png")); int[] tmpColor = new int[width * height]; int r, g, b; tmp = System.currentTimeMillis(); android.util.Log.e("mytest", "-----bitmap start-------"); for (int j = 0; j < width * height * deepth; j+=deepth) { b = arrayOfByte[j]&0xff; g = arrayOfByte[j+1]&0xff; r = arrayOfByte[j+2]&0xff; tmpColor[j/deepth] = (r << 16) | (g << 8) | b |(0xff000000); } Bitmap tmpMap = Bitmap.createBitmap(tmpColor, width, height, Bitmap.Config.ARGB_8888); android.util.Log.e("mytest", "-----bitmap end-------time = " + (System.currentTimeMillis() -tmp )); tmp = System.currentTimeMillis(); android.util.Log.e("mytest", "-----compress start-------"); tmpMap.compress(Bitmap.CompressFormat.PNG, 100, out); android.util.Log.e("mytest", "-----compress end-------time = " + (System.currentTimeMillis() -tmp )); out.close(); Thread.sleep(40); } } catch (Exception e) { android.util.Log.e("mytest", "Exception"); e.printStackTrace(); } } 

Answer 1

We used some Intel Plugin to achieve camera recording.

Video Capture for Unity3d * Android Application *

This is pretty easy to implement. I hope you find them useful.