You are here: Home » NewsFeeds » Automatic Image Colorization of Greyscale Images Using CNN

Automatic Image Colorization of Greyscale Images Using CNN

Abstract:

We present a novel technique to automatically colorize
grayscale images that combines both global priors and local
image features. Based on Convolutional Neural Networks, our
deep network features a fusion layer that allows us to
elegantly merge local information dependent on small image
patches with global priors computed using the entire image. The
entire framework, including the global and local priors as well
as the colorization model, is trained in an end-to-end fashion.
Furthermore, our architecture can process images of any
resolution, unlike most existing approaches based on CNN. We
leverage an existing large-scale scene classification database
to train our model, exploiting the class labels of the dataset
to more efficiently and discriminatively learn the global
priors. We validate our approach with a user study and compare
against the state of the art, where we show significant
improvements. Furthermore, we demonstrate our method
extensively on many different types of images, including
black-and-white photography from over a hundred years ago, and
show realistic colorizations.

Paper (15.3MB) Code (GitHub) BibTex

Colorization Architecture:

Our model consists of four main components: a low-level
features network, a mid-level features network, a global
features network, and a colorization network. The components
are all tightly coupled and trained in an end-to-end fashion.
The output of our model is the chrominance of the image which
is fused with the luminance to form the output image.

Publications:

Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa.
“Let there be Color!: Joint End-to-end Learning of Global and
Local Image Priors for Automatic Image Colorization with
Simultaneous Classification”.
ACM Transaction on Graphics (Proc. of SIGGRAPH 2016), to
appear.
@Article{IizukaSIGGRAPH2016,
  author =
{Satoshi Iizuka and Edgar Simo-Serra and Hiroshi Ishikawa},
  title = {{Let there be Color!: Joint End-to-end
Learning of Global and Local Image Priors for Automatic Image
Colorization with Simultaneous Classification}},
  journal = "ACM Transactions on Graphics
(Proc. of SIGGRAPH 2016)",
  year = 2016,
  volume = 35,
  number = 4,
}
This work was partially supported by JST CREST.


 

Original article