I wanted to start learning about computer vision and I grew up on Harry Potter, so as my first experiment, I tried to make Harry's coveted invisibility cloak.

This is an introduction into colour detection, colour spaces, and opencv.

For this demo, I am using GoCV, as I also want to further experiment with Go + computer vision.

For anyone not familiar, this is what an invisibility cloak is

It makes the wearer invisible!

Theory

 Color spaces

The first aspect we need to understand is colour spaces. Most people will be familiar with the RGB colour space. If you're not here's an article explaining RGB.

The problem with this colour space is that, it attempts to describe the level of red, green and blue, all in one value. This works for display purposes, but not for detecting colour in video.

Colour in the natural world is very much dependant on lighting, and any object has a range of shades, even for the same colour.

This is where the HSV color space comes into play. HSV describes color with one channel, hue (H), which means that color is very much only dependant on the hue, and different shades, and lighting are taken into account by saturation (S) and value (V).
So in our example, a cloth might all green, but because of the folds, the lighting in the room etc, different parts of the cloth will have slightly different colors. Using HSV, all we would need to do is adjust the V, to take into account light changes, and our algorithm is still able to recognise different greens still being green.

Masks and morphing in CV

A mask is essentially just another image, that you can apply on top of another image of the same size.

Morhping is the term we use to describe applying certain transformations to an image. OpenCV offers a few different transformations, all of which are based on a 'Kernal'.

In the world of image processing, a kernal is a matrix, which is used to actually apply the different effects we want, such as blurring or dilation.

Check out transformations here

Show me the code

Code here

Let's look more closely at the code.

In this block, we are doing all the set up, creating the video capture object (0 refers to webcam), creating the window, setting up the various mask objects we will use, and setting up the HSV values for a green cloak.

Here we grab the static background, with a sleep, which gives the webcam time to start up and video to load (if we capture the background as soon as the webcam starts, we end up with some odd lighting in the background).

We then start continuously reading frames from the webcam, and doing some transformations.

First, we convert our frame to the HSV colour space.
Then we filter that HSV image for the pixels that are within our HSV green space, and create a mask. This allows us to create a mask specifically for our green cloak.

Once we've identified our cloak, the magic can begin.

Here we can see we create our kernal, we then do a dilation. Up to this step we have a black dilated mask on our cloak.

A dilation is a type of morphological transformation that increases the object's area. I'm using this to ensure all the coloured areas is covered by our mask, while still keeping the shape of our cloak.

Since we want it to be invisible, we then invert the mask.

Now, we have an 'invisible' mask, in the shape of our cloak.

The final step is to combine the masks, with the frame, and the background and display the finished product on screen.

And now you have your very own invisibility cloak!

Notes

I suggest editing the HSV values to suit your needs, play with lighting, and colours to get the best effect for you. In my case, the lighting in my apartment was quite nice to work with, dimmer in different areas, so I was able to play around with values and locations to come up with a an example such as this

Video here