How to use the experimental mesh API with Project Tango

Question

I apologize in advance for my long post.

My goal is to create a mesh application for the Project Tango Yellowstone device to create a 3D map of building interiors. I intend to use the experimental API added in the latest versions of tango example-c code.

I am using point-cloud-jni-example (turing) as a starting point and so far have done the following:

• Set the configuration parameter config_experimental_enable_scene_reconstruction tango in point_cloud_app.cc (see docs )

  // Enable scene reconstruction ret = TangoConfig_setBool(tango_config_, config_experimental_enable_scene_reconstruction", true); if (ret != TANGO_SUCCESS) { LOGE("PointCloudApp: config_experimental_enable_scene_reconstruction() failed" "with error code: %d", ret); return ret; } 

• Added native extractMesh method in TangoJNINative.java

   // Extracts the full mesh from the scene reconstruction. public static native float extractMesh(); 

• Added appropriate extractMesh function for jni_interface.cc

   JNIEXPORT void JNICALL Java_com_projecttango_experiments_nativepointcloud_TangoJNINative_extractMesh( JNIEnv*, jobject) { app.ExtractMesh(); } 

• Added ExtractMesh method in point_cloud_app.cc

   void PointCloudApp::ExtractMesh() { // see line 1245 of tango_client_api.h mesh_ptr = new TangoMesh_Experimental(); TangoService_Experimental_extractMesh(mesh_ptr); mesh = *mesh_ptr; LOGE("PointCloudApp: num_vertices: %d", mesh.num_vertices); float float1, float2, float3; float1 = mesh.vertices[1][0]; float2 = mesh.vertices[1][1]; float3 = float1 + float2; // these lines show I can use the vertex data LOGE("PointCloudApp: First vertex, x: %f", mesh.vertices[1][0]); // this line causes app to crash; printing the vertex data seems to be the problem } 

• Added TangoMesh_Experimental ad in point_cloud_app.h

   // see line 1131 of tango_client_api.h TangoMesh_Experimental* mesh_ptr; TangoMesh_Experimental mesh; 

• An additional button has been added to call the native extractMesh method. (not showing this as it is quite simple)

For reference, here is the TangoMesh_Experimental Struct from the API :

  // A mesh, described by vertices and face indices, with optional per-vertex // normals and colors. typedef struct TangoMesh_Experimental { // Index into a three-dimensional fixed grid. int32_t index[3]; // Array of vertices. Each vertex is an {x, y, z} coordinate triplet, in // meters. float (*vertices)[3]; // Array of faces. Each face is an index triplet into the vertices array. uint32_t (*faces)[3]; // Array of per-vertex normals. Each normal is a normalized {x, y, z} vector. float (*normals)[3]; // Array of per-vertex colors. Each color is a 4-tuple of 8-bit {R, G, B, A} // values. uint8_t (*colors)[4]; // Number of vertices, describing the size of the vertices array. uint32_t num_vertices; // Number of faces, describing the size of the faces array. uint32_t num_faces; // If true, each vertex will have an associated normal. In that case, the // size of the normals array will be equal to num_vertices. Otherwise, the // size of the normals array will be 0. bool has_normals; // If true, each vertex will have an associated color. In that case, the size // of the colors array will be equal to num_vertices. Otherwise, the size of // the colors array will be 0. bool has_colors; } TangoMesh_Experimental; 

My current understanding of this structure is:

• Three pointers in float (*vertices)[3]; point to the addresses at the beginning of three pieces of memory for the x, y and z coordinates for the vertices for the grid (the same is true for normal colors and colors). A particular vertex consists of the x, y, and z components found at a particular index in three arrays.

• Similarly, the uint32_t (*faces)[3] array has three pointers to the beginning of three pieces of memory, but instead there is only one set of three elements containing index numbers that indicate which three vertices (from an array of vertices (each of which has three coordinates)) make up this facet.

Current state - I can extract the grid and print part of it on the console, and then crash without errors

 PointCloudApp: PointCloudApp: num_vertices: 8044 

If I omit the last line added to point_cloud_app.cc (# 4, above), the application will not work. I can access the vertex data and do something with it, but printing it with LOGE will crash 9 times out of 10. Sometimes it correctly prints the value without crashing. Can vertex data have holes or invalid values?

I tried returning test_float from the JNI back to java, but it will work again when I try to do this.

Suggestions?

Answer 1

vertices is a dynamic array of points, where each point is a float[3] . Try this example:

 for (int i = 0; i < mesh.num_vertices; ++i) { printf("%d: x=%fy=%fz=%f\n", i, mesh.vertices[i][0], mesh.vertices[i][1], mesh.vertices[i][2]); } 

If you look at the memory layout, it will be x0 y0 z0 x1 y1 z1, etc., each of which is a float.