Movie Gen also can generate background music and sound effects synced to the content of the videos
Facebook-owner Meta announced on Friday that it had created a new AI model called Movie Gen that can create realistic-looking video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups such as OpenAI and ElevenLabs.
Samples of Movie Gen’s creation provided by Meta showed videos of animals swimming and surfing, as well as using real photos of people to show them performing actions such as painting on a canvas.
Movie Gen can also create background music and sound effects synced with the content of a video, and use tools to edit existing videos, Meta said in a blog post.
In one such video, Meta inserted pom-poms into the hands of a man running alone in the desert, while in another it transformed a parking lot where a man was skateboarding from dry land into a field covered with splashing puddles.
Meta said videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long. It shared data showing blind tests indicating that the model performs better than offerings from startups including Runway, OpenAI, ElevenLabs and Kling.
The announcement comes as Hollywood has been looking at how to use generative AI video technology this year after Microsoft-backed OpenAI first showed in February how its product Sora could create feature film-like videos in response to text prompts.
Entertainment industry technologists are eager to use such tools to enhance and speed up filmmaking, while others are concerned about the adoption of systems that appear to be trained on copyrighted works without permission.
Lawmakers have also raised concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including the US, Pakistan, India and Indonesia.
Meta spokespeople said the company is unlikely to release Movie Gen for open use by developers, as it has done with its Llama series of large-language models, saying it considers the risks for each model individually. They declined to comment specifically on Meta’s assessment for Movie Gen.
Instead, they said, Meta was working directly with the entertainment community and other content creators on Movie Gen’s use and would incorporate it into Meta’s own products sometime next year.
The company used a mix of licensed and publicly available datasets to create Movie Gen, according to a blog post about the tool and a research paper released by Meta.
OpenAI has been meeting with Hollywood executives and agents this year to discuss potential partnerships involving Sora, though no deals have emerged from those talks yet. Concerns about the company’s approach were fueled in May when actress Scarlett Johansson accused the ChatGPT maker of mimicking her voice without permission for its chatbot. Lions Gate Entertainment, the company behind “The Hunger Games” and “Twilight,” announced in September that it was giving AI startup Runway access to its film and television library to train AI models. In return, it said, studios and its filmmakers could use the models to enhance their own work.
Continuing the achievement of the journey of effectiveness and credibility of more than 10 years in the career of journalism, as a woman journalist, I am Serving as the founder, promoter and editor of DiaryTimes with the trust and support of all. My credible coverage may not have given a big shape to the numbers, but my journey presents articles that make you aware of the exact and meaningful situations of Himachal’s politics, ground issues related to the public, business, tourism and the difficult geographical conditions of the state and financial awareness. DiaryTimes, full of the experience of my precise editorial expertise, is awakening the flame of credible journalism among all of you, so that the eternal flame of meaningful change can be lit in the life of the people of the state and the atrocities being committed against the people can be brought to the fore, I am motivated for that. If even a small change comes with the power of my journalism and the whole world becomes a witness to that issues, then I will consider myself fortunate.
