I was a research intern (Summer 2012) at Tachyon Technologies between graduating and starting full-time at Yahoo!. I worked on an applied computer vision problem to convert photographs of comic books into individual panels suitable for mobile viewing. My mentor was the CEO and an MIT TR35 awardee.

This was my first exposure to machine learning and image processing. I performed a bunch of experiments and eventually settled on a research prototype that worked well in most cases. I also built an Android application wrapping this research up into a usable tool.

Solving the problem involved a series of steps. better understood in pictures (taken off the Tachyon Demos site):

Comicreel Overview

Figure 1: Overview

  • Dewarping: Book pages in photographs usually appear warped and skewed, depending on the angle at which the photograph was taken. We need flat panels.

Comicreel Dewarping

Figure 2: Dewarping

  • Splitting Panels: A comic book page consists of a series of panels. We needed to cut these out and rearrange them in reading order.

Comicreel Splitting

Figure 3: Splitting the comic into panels

  • Restoring Colour: The colours in photographs are usually noisy, have gradients, and appear faded. But we know that the comic artist simply filled in outlines with solid colours. We need to restore these colours.

Comicreel Example 1

Figure 4: Restoring colour

And the end result:

Comicreel Example 2

Figure 5: Result