2

Apple is developing an AI tool for animating images using text prompts - The Ver...

 7 months ago
source link: https://www.theverge.com/2024/2/14/24072917/apple-keyframer-ai-animation-tool-2d-images-text-descriptions
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Apple’s latest prototype AI tool can animate images using text descriptions

/

It’s early days for Keyframer, which joins Apple’s expanding list of generative AI projects.

By Jess Weatherbed, a news writer focused on creative industries, computing, and internet culture. Jess started her career at TechRadar, covering news and hardware reviews.

Feb 14, 2024, 4:58 PM UTC

Share this story

A screenshot of Apple’s Keyframer, a prototype animation tool that shows a text prompt box and a code window.
SVG images and text descriptions fed into Keyframer are automatically converted into animation code.Image: Apple

Researchers at Apple have unveiled Keyframer, a prototype generative AI animation tool that enables users to add motion to 2D images by describing how they should be animated.

In a research paper published on February 8th, Apple said that large language models (LLMs) are “underexplored” in animation despite the potential they’ve shown across other creative mediums like writing and image generation. The LLM-powered Keyframer tool is being pitched as one example of how the technology could be applied.

Utilizing OpenAI’s GPT4 as its base model, Keyframer can take Scalable Vector Graphic (SVG) files — an illustration format that can be resized without interfering with quality — and generate CSS code to animate the image based on a text prompt. You just upload the image, type something like “make the stars twinkle,” in the prompt box, and hit generate. Examples provided in the research paper show how a Saturn illustration can transition between background colors, or show stars fading in and out of the foreground.

There’s no video available, but these frame-by-frame comparisons are an example of Keyframer’s capabilities.Image: Apple

Users can produce multiple animation designs in a single batch, and adjust properties like color codes and animation durations in a separate window. No coding experience is necessary as Keyframer automatically converts these changes into CSS, though the code itself is also fully editable. This description-based approach is much simpler than other forms of AI-generated animation, which typically requires several different applications and some coding experience.

Keyframer’s editing tools are fairly limited, but at least you don’t need to understand code to use it.Image: Apple

One professional motion designer who took part in Apple’s research said, “Part of me is kind of worried about these tools replacing jobs, because the potential is so high. But I think learning about them and using them as an animator — it’s just another tool in our toolbox. It’s only going to improve our skills. It’s really exciting stuff.”

Still, it has a long way to go. Keyframer isn’t publicly available yet, and the user study within Apple’s research paper comprised just 13 people, who could only use two simple, pre-selected SVG images when experimenting with the tool.

Apple was also careful to mention its limitations within the paper, specifying that Keyframer focuses on web-based animations like loading sequences, data visualization, and animated transitions. By contrast, the kind of animation you see in movies and video games is far too complex to produce using descriptions alone — for now, at least.

Keyframer is one of several generative AI innovations that Apple has announced in recent months. In December, the company introduced Human Gaussian Splats (HUGS), which can create animation-ready human avatars from video clips. Last week, Apple also released MGIE, a new AI model that can edit images using text-based descriptions.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK