The word “deepfake” no longer surprises active Internet users. And what until recently seemed like a joke, today seems to be an interesting and promising technology. We consider a deepfake a certain alien character artificially introduced into a video clip using neural networks and machine learning. But the meaning of this phenomenon is much broader.
So, artificial aging of photos using FaceApp is also considered a deepfake. This technology is constantly finding new areas of application that seemed unthinkable a year ago. If earlier we were surprised at how the old photograph came to life, today you can make the character in the picture sing the chosen song.
You probably remember how the pictures on the walls came to life in the Harry Potter films. It seemed like magic until services with similar capabilities appeared on the Internet. The pictures were run through neural networks, and a short video appeared at the output. But a moving person could live in those days when they did not know how to shoot a movie.
How to animate a photo and teach it to sing?
The Wombo application develops the concept of deepfake, allowing not only to animate the photo, but also to make the character’s movements more active by teaching him to sing the selected song.
It is worth emphasizing right away that if your goal is simple animation of a photo, then it is better to choose a specialized application for this purpose, for example, Deep Nostalgia… If you need voice acting in addition to photography, then Wombo will help. The service will do the bulk of the work itself. The user only needs to upload a photo and select a composition for it from the list.
2. After starting the program, give it access to the photos on your device.
3. Select a finished photo from the gallery or create a new photo for processing.
Selfies are best, where the person is looking directly into the frame and, if possible, teeth are visible. It is worth choosing a photograph that best meets this criterion.
4. Tracks for performance can be selected in the application itself, even if there are few songs, but they fit perfectly on the performer’s face.
5. Just wait until the photo is processed and reformatted into a video.
– Dima Mikhnevich (@nestanda) March 21, 2021
– Socialist Distancer 🌹 (@VideoMilitia) March 10, 2021
The true use of Wombo is posthumously expanding someone’s taste in music. pic.twitter.com/Ur4SlUOV2d
— Sam Moore (@IAMSMOORES) March 11, 2021
This WOMBO AI is crazy lol pic.twitter.com/A7aVT4ISBN
– heyben10 (@ HeyBen10_) March 10, 2021
— Kasai Everquill (MYST1F0X) (@MYST1FOX) March 12, 2021
— Bebop (@Beb0pu) March 10, 2021
– (trsh) birschbox (@birschbox) March 11, 2021
– Zoya Belova (@AliraSirin) March 12, 2021
Wombo is a good app pic.twitter.com/5vMsouLY90
— 🌈Wings & Strings (@TomColetti) March 11, 2021
The uniqueness of the Wombo service is obvious. It’s not enough just to bring a photo to life – such services are no longer new. But in this case, the characters are animated differently, depending on the song chosen for them. And the style of music directly affects the manner of performance, as a result, it can vary significantly.
How does this deepfake work?
Speaking about the merits of Wombo, one cannot fail to mention its key feature – support for Lipsink. That is, it is not just the animation of the photo to the music that takes place, but the person’s mouth opens clearly to the beat, as if the song is actually being performed. It is enough to familiarize yourself with some examples of processing even old photographs by a neural network, as you will see various lip movements and a high-quality hit in words. On some songs, the effect is especially pronounced.
While it is difficult to talk about a high-quality result for all photos and songs – it’s still a funny, but experiment. However, by choosing the appropriate picture, you can achieve a better result. And you can imagine as a performer not only yourself or your friends, but also public figures, politicians, and even animals, cartoon characters, sculptures and other inanimate objects.
In an interview, the developer said that his neural network uses pre-created choreography. A team of enthusiasts, not even in the studio, but in the apartment, shot a basic video for each song. After that, on the basis of this, an animation was created, superimposed by artificial intelligence on the selected image. Only about 20 songs are available in the application so far. But as soon as the creators resolve the copyright issues, the list will be expanded. But uploading and overlaying your song on a photo will not work. This is quite understandable by the Wombo algorithm.
The developers also paid attention to security issues. Users are assured that the photos they upload are not transferred anywhere and are deleted after processing. But the resulting karaoke video can be shared with others, the standard sharing function will help with this. Best of all, the Wombo app is completely free! Its creators earn money by displaying advertising banners. And users of both iOS and Android can try this unique deepfake tool.