Animate Anybody: All About The AI Animation Software That Has Been Producing Controversy

Isn’t it astonishing to see AI creation tools generating artificial images with a single click? And, it has only got more mind-boggling as generative AI goes through further development. Animate Anyone, a new AI project that breathes lives into inanimate images is a recent example of this revolution.

Animating images with AI isn’t exactly new, considering that there have been several image-to-video synthesis research projects that intend to animate subjects from still images. While most of the existing attempts have produced barely acceptable output, Animate Anyone is probably the first one that leaves users fascinated with its AI-animated image creations. Based on the image diffusion models, it can successfully animate human subjects, anime, cartoons and humanoid figures and all it needs is a clean photo and some reference dance moves.

Being a research project, Animate Anyone isn’t available in a similar fashion as OpenAI’s ChatGPT or Google’s Bard yet. The only way to play with it requires you to be a developer proficient with coding. However, you needn’t be a coding genius to understand how the technology works and what all it can do. Join us as we try to explain it in the simplest way possible.

How does this AI tool animate images?

AI has been able to create images and videos from scratch based on all the data available on the internet. But Animate Anyone takes it to the next level. It uses advanced techniques to keep details consistent and movements smooth by combining different models for details like ReferenceNet, a post guider for control and a temporal overlay for smoother transitions.

In layman’s terms, it simply analyses the image and adapts several tinkered versions of the image to a present movement pattern. The result is an image of a person or an anime that appears to dance within the frame with little to no disruptions or noise. Unless alerted, it will be easy to mislead people into believing that a real video of the subject was shot.

The creators of the project, namely Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang and Liefeng Bo hailing from the Institute for Intelligent Computing, Alibaba Group, have published their project on GitHub and their paper is available on arXiv. The team has also made available the code as well as the demo videos on GitHub, thereby allowing app developers to work on the technology and even use it in their apps or services.

Some of the demo videos shared on the page show astonishing results achieved with simple portrait images of people, anime and humanoid figures. So far, the model has even been successful in reading a blurry image of a woman and creating a dancing video out of it, all while preserving the facial details as well as the rest of the clothing. What’s surprising is that this AI tool also renders the shadows and other elements with believable laws of physics.

Animate Anyone can generate videos of anyone with a single image. 🤯pic.twitter.com/479PSFmLZN

— Max Layn Stein Cohen (@maxlayn) December 3, 2023

Just recently dove into the ‘Animate Anyone’ paper and I’m blown away! Imagine taking your favorite character, or a picture and bringing it to life in videos with smooth, realistic motion.

Animate Anyone can transforms any static images into dynamic videos with unmatched… pic.twitter.com/PnQ0ksW5rI

— Richardson Dackam (@RichardsonDx) November 30, 2023

Is it perfect? No. There’s a lot of work left to fine-tune the algorithms and make them work on subjects against complex backgrounds.

Who will find Animate Anyone useful?

Other than the AI research industry, the technology behind Animate Anyone can be of immense use to video game developers, fashion designers and movie creators. We expect to see it help in driving down the development costs as well as time.

We can only imagine the possibilities that Animate Anyone will bring to the table. What if the next Grand Theft Auto game relies on this technology to improve its character animations, or Marvel Studios use it to train its next generation of superheroes?

Users raise concerns about its abilities

As always with a new AI technology, the internet is divided over the pros and cons of such technology. While some are eager to see what this AI tool can do for the entertainment industry, the other lot is concerned about digital privacy and malicious intent. Especially after the latest deepfake controversy surrounding Indian actress Rashmika Mandanna, after which people have expressed fear over the rapid dissemination of misinformation and the role of AI in this troubling phenomenon.

AI-generated human videos from pose detection is coming with Alibaba researcher’s new ‘Animate Anyone’.

Absolutely wild to think anyone will be able to quite literally be *anyone* in the future.https://t.co/Iq3R5xy41b

— Rowan Cheung (@rowancheung) December 4, 2023

How are things moving THIS FAST!?

See Also

Ladies of Energy: The Shortest Historical past

There’s been a major breakthrough in AI animation, announced as part of the Animate Anyone project.

They seem to have solved the major image-to-video challenges with a new framework I’ll share more on in the comments. #aianimation #aivideo… pic.twitter.com/Z05VfoBjJZ

— The Reel Robot (@TheReelRobot) December 1, 2023

As the accessibility of the AI tool increases in the future, it will be up to the makers to come up with rules and regulations to keep the internet safe for users.

How to animate with this AI tool

Currently, the Animate Anyone project is only available on GitHub as an open-source tool and is yet to be implemented by any developer in an app or website. Hence, you may have to wait until someone figures out a way to provide this in some editing app or a new filter on TikTok.

But remember, with great power comes great responsibility!

(Hero and Featured Image: Courtesy Alibaba Group via GitHub)

This story first appeared on Augustman India.

Frequently Asked Questions (FAQs)

– How does the Animate Anyone AI tool work?
The Animate Anyone AI tool uses advanced techniques to keep details consistent and movements smooth by combining different models for details like ReferenceNet, a post guider for control and a temporal overlay for smoother transitions.