Nvidia introduces deepfake eye contact effect

Updated on: 13 January 2023

Justinas Vainilavičius
Senior Journalist

Eye contact signifies confidence and plays a key role in establishing social connections, the company said.

Now you can read your notes or stare out of the window during video calls or webcasts – while appearing to look directly into the camera thanks to Nvidia’s new Eye Contact effect.

“Continuously making eye contact is not feasible in video conferencing scenarios,” Nvidia said in a blog announcement.

“This can be difficult if one is reading off a script or reviewing data on a computer screen,” Nvidia said, adding that maintaining direct eye contact can also be challenging for some people due to psychological reasons.

The Eye Contact effect is based on an AI algorithm that estimates the user’s gaze in real-time and redirects it toward the camera by applying a filter to a webcam.

Nvidia said that eyes retain their natural color and blinks. The feature detects a person’s hands or other objects blocking the eyes and adjusts accordingly. It can also disconnect if you look too far away and is supposed to transition smoothly between simulated and real eyes.

Nvidia has released a video demonstration of how the effect works. It said it was still in a beta version and called for users to report any issues they might encounter to improve the algorithm further.

The effect is built to take into account environmental factors such as lighting, shadows, blurriness, white balance, and hue, as well as the subject’s head pose, skin color, eyeglasses, eye gaze, and face and eye shape.

Preservation of eye color was one of the key challenges for any gaze redirection algorithm, Nvidia said. It said it based its own Eye Contact on a dataset of 4 million images, with about a quarter of those synthetically generated “to add diversity in eye color and shape.”

The effect comes as part of an update of Nvidia’s Broadcast package for live streaming and video conferencing, which is free for Nvidia and GeForce RTX GPU owners.