Video feed Face Tracking in FaceRig Studio is done via the face markers provided by the ULSee face tracking library.
If you have a head-mounted camera used just for face expressions, and you get the head and torso motions from other sensors, not the head-mounted webcam, then toggling on “Front Facing Mode” (FFM for short) will give you greater accuracy for face expressions.
Why would we need FFM?
By default there are a number of inaccuracies when evaluating the facial expressions, that stem mostly from the actor’s face being rotated away from the camera (these happen in set-ups with a webcam that rests on a stand or on the monitor, and is not head-mounted).
These inaccuracies are dealt with in the FaceRig Studio interpretation layer with a set of assumptions about which marker information is reliable face expression data, and which information is likely faulty, caused by head rotation or lighting changes. These assumptions however are no longer needed in a head mounted camera set-up ( especially if the camera helmet includes even face lighting) , so we can assume most of the data is accurate, so we want to do away with those assumptions.
How to switch on FFM ?
Advanced UI-> Advanced Tracking Configuration->Trackers Tab-> Additional options Section.
Keep in mind that once you toggle this on, the “head movement” and “upper body movement” toggles in the ULSee tracker section will no longer do anything (it is best if they are toggled off if you use the Perception Neuron Suit anyway, as that motion will no longer have to be inferred from the webcam feed, that head and body data will come in a much more reliable fashion from the Axis Neuron). Remember to switch them on the Perception Neuron section.
A neutral expression calibration is recommended after turning on front facing mode.
Head mounted camera placement ( its height) is also super important in front facing mode. Place it too high above and looking down on the face, and the avatar will have problems frowning. Place it too low, you will get similar problems.
If you notice “odd” things happening such as the avatar always looking at the render camera, even if the actor’s eyes are looking somewhere to the left , or right, please also take a look at the section called Avatar Behaviours Tips and tricks. There are a number of face features (gaze direction being one of them) that can be overridden by pre-set behaviors. In the case above “Force avatar look at camera” behavior might be dialed up too high.