Microsoft patents AI tool to "clean" live broadcasts based on your profile
Live translation tool or deepfake nightmare?
Image by Cybernews
Follow us
- Microsoft patented software that could replace words in live videos based on a viewer’s profile.
- The tool could change audio and mouth movements so edits appear natural to viewers.
- The patent describes fast translation for live videos, with delays of only a few seconds.
- Users worry the technology could help create deceptive AI-edited content without clear disclosure.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
Microsoft patented software that could rewrite speech in real time. The document has sparked concerns among users who worry about deceptive AI practices.
While a user watches a live video, AI would detect certain words that need to be replaced based on the user’s profile (age, language, or location).
It would then replace these words with better or more appropriate alternatives and alter the speaker’s mouth movements to match any changes.
The software would also translate videos from foreign languages with a delay of just a few seconds.
AI as a speech editor
The AI agent would listen to audio and identify words that need to be replaced. These could be filler words like "um," "uh," "er," mispronunciations, mumbling, or offensive language.
Substituted words should be taken from an “internet search,” which would then be fact-checked, according to Patentylze.
An original recording would be kept as “ground truth” in cases where such a tool may cause confusion or misuse.
In near-real-time, AI would replace the chosen words with alternatives and edit audio and video, so users wouldn’t notice the discrepancy.
The patent also details a translation feature that can be used if the person in the video is speaking a language the viewer does not understand.
The AI-powered software would run on various devices, including phones, tablets, and AR glasses.
Stay updated with our latest stories and follow us on social media
Be the first to discover new stories, ideas, and updates from our team.
Another tool to misuse?
While the tool could be useful, some users have already raised concerns.
Many users believe that this type of AI-powered editing tool could be used to fabricate content.
“The only positive side to this I can see is that you can have dubbed movies where the actor's mouth moves in sync with what's being said, but realistically, this is going to add to the endless pile of deepfakes,” noted one Redditor.
Microsoft Patents AI That Rewrites Speakers' Words in Live and Recorded Videos While Adjusting Lips to Match, Raising Concerns Over Fabricated Histories and Personalized Gaslighting
by u/CollapsingTheWave in ObscurePatentDangers
“We need strong regulatory laws against deceptive AI practices. No one should be allowed to publish AI-modified content without disclosure,” shared another user.
The patent doesn’t indicate whether Microsoft will release such software in the future.
While the document stated that it would include a copy of the original recording, it didn’t state whether or how it would indicate that the content is edited by AI.
People were quick to point out that this patent isn’t new and reminded other users that deepfakes “have been a thing for a while.”
