We all know that AI will inevitably take over the world, bringing humanity to its knees by the power of our own creation–but for now, it’s settled for making inroads in the wonderful world of Japanese anime.
Artists have long since been partaking in nijigenka, or “2D-ification,” taking pictures of real life people and transforming them into anime characters. Just as we might all like comparing our fave anime characters to the actors who might play them in real life, it can be just as fun to reimagine what flesh-and-blood folks would look like as shoujo heroes. But can a robot do it better?
At recent workshop in Tokyo, Fudan University undergrad Yanghua Jin demonstrated how, using Generative Adversarial Networks (GAN) and Deep Learning (yeah, idk what any of that means either), he can create gorgeous anime portraits without wielding any artistic skill himself.
In his presentation, he explains a lot of complex math-y things that I am really too dumb to explain. But here’s what I have gathered from SoraNews24’s explanation: basically, the tech takes a two-pronged (or rather, two-networked) approach.
The first network, the generator, takes the faces of real people and translates them into anime using features like hair color and style, facial expression, etc, as well as considerations like facial proportion and body angle. The first network, the generator, takes the faces of real people and translates them into anime using features like hair color and style, facial expression, etc, as well as considerations like facial proportion and body angle. Then, after creating the new image, the second network, the discriminator, scans its library of anime images and tries to pick out the new, “fake” anime image produced by the generator from the original artworks.
One of the biggest difficulties is, while GAN is generally good at transferring textures, anime characters have proportions (huge-ass eyes, tiny-ass mouths) one doesn’t generally see IRL.
Since the GAN learns from its mistakes in past models (cool but terrifying), the results should only get better and better.
At first, the AI anime turned out…pretty wonky.
An otaku’s acid trip nightmare.
But thanks to ~Deep Learning~, they’ve vastly improved since.
Watch Jin’s entire presentation: