Luma AI launched its artificial intelligence (AI) text-to-video generation model, Dream Machine, globally on Wednesday. The platform can generate up to five-second-long videos from simple or descriptive text prompts. The AI model can generate videos in various styles such as cinematic, animation, realistic, and more. The AI firm claims that Dream Machine was trained entirely on videos, and is capable of generating “physically accurate, consistent and eventful shots.” The platform is currently free to access and use, but it is likely to have a daily generation limit.
Luma AI debuts Dream Machine
According to the website, the Dream Machine AI model is built on a transformer model and was trained directly on videos. Usually, large language models (LLMs) are trained on text and images and then are moved to videos, as they require a deeper spatial and motion understanding. “Dream Machine is our first step towards building a universal imagination engine,” the company added.
Dream Machine joins video generation platforms such as Runway AI and Pika 1.0 which are also available in the public domain and offer video generation between three to five seconds. Gadgets 360 tried the platform and found that the prompt adherence of the platform is sub-par. It struggles with multiple characters or when the prompt is too complex. However, compared to the other two, it is capable of generating higher-quality cinematic videos.
The AI platform takes 120 seconds to generate a video which the company claims will have 120 different frames. Dream Machine is also said to understand how people, animals, and objects interact with the physical world and create videos with accurate physics and character consistency.
However, Luma AI also highlighted several limitations in the current mode such as movement, text, morphing, and the famous Janus problem, where instead of showing a consistent 3D output, the AI model shows multiple canonical views of an object in different directions.
Luma AI’s Dream Machine creates a video about Stormtroopers
Photo Credit: Luma AI/Dream Machine
Luma AI has not shared any technical details about the AI model, so information about parameter size, benchmarks, architecture, and training methods is not known. The company also did not share any details about the procurement of the training data. Notably, Gadgets 360 was able to generate several videos with copyrighted characters in them.
To try out the platform, enthusiasts can go to the website and click on the ‘Try Now’ button. Users will have to sign up for the platform before they can generate videos.
