AI tool automatically colorize black and white photos

Swedish machine learning researcher Emil Wallner has released a free web tool called Palette.fm that automatically colorizes black and white photos using artificial intelligence. Once the photo is uploaded, users can select a color filter or refine the colors with a written text description.

Palette.fm uses a deep learning model for image classification that determines baseline guesses about the colors of objects in a photo or illustration. “I created my own AI model that uses an image and text to create a coloring page,” Wallner wrote in a post to Ars. “One model creates the text, while the other takes the image and text to color in.”

Once you upload an image, the site’s elegant interface will provide a rough caption (description) of what it thinks it sees in the image. If you don’t like any of the preset color filters, you can click the pencil icon to edit the caption yourself, which guides the colorization model with a text hint.

To test this, we took a photo of a small pumpkin and removed the color with Photoshop. We then downloaded the black and white version and experimented with a selection of premade filters that Palette.fm provides. Once we found a good filter, we edited the title to refine the colors by describing the objects in the scene. For example, Palette.fm initially thought the pumpkin was a “claw”and didn’t recognize the pavement. But once we put those terms into the written prompt, the colors became more understandable. We later enhanced the image (not shown in the example below) by adding “green leaves”in the background.

For now, Palette.fm is available as a free service, but Wallner plans to add a paid option. The site processes images online, in the cloud. Regarding the privacy of uploaded photos, the Palette.fm website states: “We do not store your images.”But, as with any cloud service, take this with a grain of salt when it comes to personal photos. Notably, Palette.fm does not currently require any kind of user account registration.

So far, Palette.fm has fascinated people on Hacker News who have used the tool to colorize photos of loved ones, historical photos, and more. A variation of Wallner’s colorization technology has also been available as a bot on Twitter since late last year. Have fun coloring.