Meta makes clips, teases new generative AI video model

Mark Zuckerberg’s Meta is now making 16 second clips with what it has dubbed “the most advanced media foundation model to date” – Meta Movie Gen.

Meta is known for its dominant position as a social media giant, with apps like Facebook, WhatsApp, and Instagram falling under its umbrella.

Like many tech giants, Meta is also developing artificial intelligence (AI) models to compete in the AI race. Meta already has its large language model, Llama, but it has expanded its repertoire to rival OpenAI and PikaLabs.

Now, Meta is premiering its text-to-video model, which allows users to generate “high-quality and high-definition images and videos from a single text prompt,” AI at Meta said in a post on X.

🎥 Today we’re premiering Meta Movie Gen: the most advanced media foundation models to-date.

Developed by AI research teams at Meta, Movie Gen delivers state-of-the-art results across a range of capabilities. We’re excited for the potential of this line of research to usher in… pic.twitter.com/NDOnyKOOyq

undefined AI at Meta (@AIatMeta) October 4, 2024

A team of artificial intelligence (AI) researchers at Meta have developed the generative text-to-video model, a transformer model that can create videos and audio from a single sentence.

Movie Gen can also transform video input and text to create high-fidelity audio. “It can generate ambient sound, instrumental background music, and foley sound – delivering state-of-the-art results in audio quality, video-to-audio alignment, and text-to-audio alignment,” Meta said.

Along with video and audio, Movie Gen also offers video editing, which can be done using generated or existing video and additional text prompts that allow you to add, remove, or manipulate elements like style or background changes.

What’s perhaps novel about this model is that Movie Gen allows you to upload images of your friends or family members to make personalized videos for the people you know. This might spark some concerns about safety and privacy, as it almost functions as a create-your-own deepfake.

More examples of what Meta Movie Gen can do across video generation, precise video editing, personalized video generation and audio generation. pic.twitter.com/Dz65QGNgaJ

undefined AI at Meta (@AIatMeta) October 4, 2024

Meta is obviously trying to rival OpenAI’s Sora and PikaLabs Pika 1.5 with this model. The AI-powered video suite claims to democratize creativity by giving everyone access to Movie Gen.

“We believe everyone should have access to tools that help enhance their creativity,” Meta said.

Much like OpenAI’s Sora, which isn’t widely available yet, we could see a big shift in the industry as we rely more on AI to create films, advertisements, and music videos. Sora, Pika 1.5, and now Movie Gen have the ability to shake up Hollywood and the advertising industry.

Although the model isn’t available to the public yet, Meta has said that it will continue to improve the model and move toward a potential future release.

“We’ll work closely with filmmakers and creators to integrate their feedback. By taking a collaborative approach, we want to ensure we’re creating tools that help people enhance their inherent creativity in new ways they may have never dreamed would be possible,” Meta concludes.