Meta has unveiled its latest artificial intelligence innovation, Movie Gen, designed to generate videos and audio from text prompts. Positioned as a competitor to OpenAI’s Sora, this AI model is capable of creating videos based on user descriptions and generating matching audio. In addition, it can personalize videos by using real photos of individuals, placing them in various scenarios. While this feature is groundbreaking, Meta is unlikely to release Movie Gen for open developer use, unlike its Llama AI series, according to a Reuters report.
In their research, Meta explains that Movie Gen is trained in both text-to-image and text-to-video tasks. It generates a series of colorful images serving as frames to form a video. The model can produce high-definition videos of up to 16 seconds at 16 frames per second, with adjustable resolution and duration according to user needs. This capability is enhanced by its understanding of real-world visuals, which allows the AI to interpret object motion, camera angles, and interactions effectively.
For audio, Movie Gen uses advanced video-to-audio and text-to-audio techniques to produce high-quality, 48kHz audio with cinema-like sound effects and music. Although its video duration is capped at a few seconds, the AI can provide “long-form coherent audio” for videos lasting several minutes.
A notable feature of Movie Gen is its ability to condition on both text and images, enabling it to incorporate a selected person from a real image into generated videos, preserving identity while following user prompts. Furthermore, the model offers advanced video editing capabilities. It can make precise edits to both generated and real videos based on user instructions. For example, Meta demonstrated how Movie Gen edited a video’s background and added elements to the main subject, showcasing its versatility and innovation in video technology.
Original Story https://www.business-standard.com/technology/tech-news/meta-announces-movie-gen-ai-model-what-is-it-how-it-works-and-more-124100700350_1.html
Category :
Tags:
