iOS ARKit Face Tracking Vertices   2 comments

I’m currently working on a project that involves face tracking, and as a first prototype am using the built-in features in the ARKit library, Apple’s augmented reality API for iOS.

Using a device with a front-facing TrueDepth camera, an augmented reality session can be started to return quite a bit of information about faces it is tracking in an ARFaceAnchor object. One of these details is the face geometry itself, contained in the aptly named ARFaceGeometry object. For those who care about face shape landmarks, the 1220 individual vertices (points) can provide a wealth of information. However, there is little to no published information on which vertex corresponds to what point on the face.

While keeping in mind that this means these vertices could easily change in future versions of ARKit, I’ve taken the liberty of labeling points in case it is helpful for a future developer.

Update: A few people have asked how I generated this map. Details below.

Update: Since posting these pictures, a few people have asked how I generated this map and/or to share some code. I recommend starting with Yono Mittlefehldt’s AR Face Tracking Tutorial. That page goes through the basics of setting up your ARSession, ARFaceTrackingConfiguration, etc etc. It even goes so far as to attach SCNNode and child nodes that move with the face.

This is where Mittlefehldt stops short: he gives a few “magic numbers” corresponding to key points (eyes, nose, mouth, and hat) to attach emojis. To generate the maps above, I took this one short step further and generated a child node that puts the number itself on each node. If you’re reading through the tutorial, the difference is to add the following to your extension ViewController: ARSCNViewDelegate extension:

You can replace the list of vertex numbers with anything you’d like. The images above include all vertices, followed by every 4th vertex to clear out particularly “busy” areas.

Posted 18 Dec 2019 by John McManigle in Technical

2 responses to iOS ARKit Face Tracking Vertices

Subscribe to comments with RSS.

  1. hi could you explain the code to find out each vertex point?

Leave a Reply

Your email address will not be published. Required fields are marked *