Geek Answers Handbook

Can ARKit detect specific surfaces as planes?

Using iOS 11, iOS 12, and ARKit, we can currently detect planes on horizontal surfaces, and we can also visualize this plane on a surface.

I am wondering if we can declare through some kind of image file specific surfaces on which we want to detect planes? (possibly ignoring all other planes that ARKit detects on other surfaces)

If this is not possible, can we then capture the detected plane (through the image), which we could then process from a CoreMLmodel CoreMLthat identifies this particular surface?

+7
source share
2 answers

ARKit does not currently support such a thing. You can really capture a plane detected as an image, and if you are able to match this with the main ML in real time, I’m sure many people will be interested!

You must:

  • get the three-dimensional position of the corners of the plane
  • find your 2D position in the frame using sceneView.projectPoint
  • extract frame from currentFrame.capturedImage
  • make an affine transformation on the image that you want to leave with your plane, redraw it on a rectangle
  • do some ML / image processing to find a match

Keep in mind that finding an ARKit rectangle is often not well aligned and can only have part of the full plane.

, , , ARKit, , - , , Apple , .

+2

. # Swift5 Github

0

More articles:


All Articles