Hi guys, this is a new entry of my blog and the first one in which I will upload the project to Github, where you can download it. As you remember, first you need to configure your Visual Studio Environment. You can find those steps in my previous entries.
One of the most important things we want to achieve with Computer vision is to recognize an object and determine the position of the object in the space. In this project, I converted the captured image (in the RGB color space) to an HSV color model. In HSV, we can easily segment the desired object. At last, the centroid of the segmented object was calculated using the Spatial Moments of the image.
Why do we need HSV?
The most commonly used color model is the RGB. This means that the captured image is represented by three matrix; the first one represents the value of Red per pixel, the second one the value of Green and the last one the value of Blue. If we are using an 8-bits resolution, the maximum value a pixel may have will be 255 meanwhile the minimum value will be 0. The resultant color of each pixel will be determined by the combination of the three values.
However, it is really hard to segment the image by color using this model. Therefore, a conversion to the HSV model is needed. In HSV, we have also three matrix, however the first one represents the Hue (color), the second one the Saturation of color and the third one the Value of lightness. As you can imagine, the expected color is determined by only one value (Hue) and not by three as in the RGB model; therefore the segmentation by color is easier.
But what does Saturation and Value mean?
Saturation indicates the “amount” of the color in one pixel, while the Value indicates the “lightness” of the pixel. If you want to learn more about color models and all the maths involved on it, you can look at any Computer Vision book. The important thing is that the HSV allow us to segment an image by color easily, something that wouldn’t be as easy if we use RGB.
Based on what has been described before, we can implement it to segment the captured frames. The general procedure consists on:
- Obtain the image captured by the webcam of video device.
- As on EmguCv 3.1, you need to convert the image from Mat to Image<Bgr, Byte>
- Convert the Image<Bgr,Byte> to Image<HSV,Byte>.
- From the UI, obtain the values for Low and High limit. For this purpose, the user can modify the value for each limit using a trackbar.
- Apply the inRange function to the HSV frame; all values that are between the low and the high limit will be displayed as white. This frame will be displayed in the UI.
- A closure operation will be applied 3 times in order to reduce noise.
- On the binarized frame, obtained after the inRange function was applied, the moments should be calculated. Afterwards, the zero and one order spatial moments will be used to calculate the centroid.
- Using the m00, m10 and m01 spatial moments, the centroid is calculated as following:
Centroid_x = m10 / m00; Centroid_y = m01/m00;
- Using the calculated centroids, a red circle is drawn on the original frame.
Testing the application
When we execute the application, the low limit values are all set to 0 and the high limit values are set to 360. In the binarized frame, we can see that all the frame is displayed in white. This happens as all the pixels are between the given range. In the third frame, we can see that the centroid of the “segmented object” is in the middle.
Now we proceed to play with the scrollbars. For example, I have a blue glass that was segmented correctly and the centroid was displayed correctly.
Another example, a banana!
As you can see, this is a simple object recognition application. We have identified the object and its centroid. We can identify the position of the segmented object, the trayectory if it is moving and once it has been binarized, the object’s shape.
Obviously, this approach has some downsides: we need an controled environment, in order to control the lightness and brightness of the space so we don’t need to calibrate the system every time we execute the application. Another downside is that if two objects have the same color, they will be segmented as one.
In future post, we will apply other techniques to identify projects by shape.
The project can be found in here: GitHub
As always, if you have any doubt feel free to contact me or comment in this post. I will reply ASAP.