Artificial Intelligence Brings to Life Figures from 7 Famous Paintings: The Mona Lisa, Birth of Venus & More

Denis Shiryaev is an AI wiz­ard who has lib­er­al­ly applied his mag­ic to old film—upscaling, col­oriz­ing, and oth­er­wise mod­ern­iz­ing scenes from Vic­to­ri­an Eng­land, late Tsarist Rus­sia, and Belle Époque Paris. He trained machines to restore the ear­li­est known motion pic­ture, 1888’s Round­hay Gar­den Scene and one of the most mythol­o­gized works of ear­ly cin­e­ma, the Lumière Broth­ers 50-sec­ond Arrival of a Train at La Cio­tat Sta­tion.

Shiryaev’s casu­al dis­tri­b­u­tion of these efforts on YouTube can make us take for grant­ed just how extra­or­di­nary they are. Such recre­ations would have been impos­si­ble just a decade or so ago. But we should not see these as his­toric restora­tions. The soft­ware Shiryaev uses fills in gaps between the frames, allow­ing him to upscale the frame rate and make more natur­is­tic-look­ing images. This often comes at a cost. As Ted Mills wrote in an ear­li­er Open Cul­ture post on Shiryaev’s meth­ods, “there are a lot of arti­facts, squooshy, mor­ph­ing moments where the neur­al net­work can’t fig­ure things out.”

But it’s an evolv­ing tech­nol­o­gy. Unlike wiz­ards of old, Shiryaev hap­pi­ly reveals his trade secrets so enter­pris­ing coders can give it a try them­selves, if they’ve got the bud­get. In his lat­est video, above, he plugs the NVIDIA Quadro RTX 6000, a $4,000 graph­ics card (and does some grip­ing about rights issues), before get­ting to the fun stuff. Rather than make old film look new, he’s “applied a bunch of dif­fer­ent neur­al net­works in an attempt to gen­er­ate real­is­tic faces of peo­ple from famous paint­ings.”

These are, Shiryaev empha­sizes, “esti­ma­tions,” not his­tor­i­cal recre­ations of the faces behind Leonardo’s Mona Lisa and Lady with an Ermine, Botticelli’s mod­el for The Birth of Venus, Vermeer’s for Girl with a Pearl Ear­ring, or Rembrandt’s The Night Watch. In the case of Amer­i­can Goth­ic, we have a pho­to of the mod­el, artist Grant Wood’s sis­ter, to com­pare to the AI’s ver­sion. Fri­da Kahlo’s Self-Por­trait with Thorn Neck­lace and Hum­ming­bird gets the treat­ment. She left per­haps a few hun­dred pho­tographs and some films that prob­a­bly look more like her than the AI ver­sion.

The GIF-like “trans­for­ma­tions,” as they might be called, may remind us of a less fun use of such tech­nol­o­gy: AI’s abil­i­ty to cre­ate real­is­tic faces of peo­ple who don’t exist for devi­ous pur­pos­es and to make “deep fake” videos of those who do. But that needn’t take away from the fact that it’s pret­ty cool to see Botticelli’s Venus, or a sim­u­la­tion of her any­way, smile and blink at us from a dis­tance of over 500 years.

Relat­ed Con­tent:

Watch Scenes from Czarist Moscow Vivid­ly Restored with Arti­fi­cial Intel­li­gence (May 1896)

Watch AI-Restored Film of Labor­ers Going Through Life in Vic­to­ri­an Eng­land (1901)

Icon­ic Film from 1896 Restored with Arti­fi­cial Intel­li­gence: Watch an AI-Upscaled Ver­sion of the Lumière Broth­ers’ The Arrival of a Train at La Cio­tat Sta­tion

The Ear­li­est Known Motion Pic­ture, 1888’s Round­hay Gar­den Scene, Restored with Arti­fi­cial Intel­li­gence

Josh Jones is a writer and musi­cian based in Durham, NC. Fol­low him at @jdmagness