The last section of this chapter is dedicated to creating another effect, called cartoonize; the purpose of this effect is to create an image that looks like a cartoon. To do this, we divide the algorithm into two steps: edge detection and color filtering.
The cartoonCallback function defines this effect, which has the following code:
void cartoonCallback(int state, void* userData)
{
/** EDGES **/
// Apply median filter to remove possible noise
Mat imgMedian;
medianBlur(img, imgMedian, 7);
// Detect edges with canny
Mat imgCanny;
Canny(imgMedian, imgCanny, 50, 150);
// Dilate the edges
Mat kernel= getStructuringElement(MORPH_RECT, Size(2,2));
dilate(imgCanny, imgCanny, kernel);
// Scale edges values to 1 and invert values
imgCanny= imgCanny/255;
imgCanny= 1-imgCanny;
// Use float values to allow multiply between 0 and 1
Mat imgCannyf;
imgCanny.convertTo(imgCannyf, CV_32FC3);
// Blur the edgest to do smooth effect
blur(imgCannyf, imgCannyf, Size(5,5));
/** COLOR **/
// Apply bilateral filter to homogenizes color
Mat imgBF;
bilateralFilter(img, imgBF, 9, 150.0, 150.0);
// truncate colors
Mat result= imgBF/25;
result= result*25;
/** MERGES COLOR + EDGES **/
// Create a 3 channles for edges
Mat imgCanny3c;
Mat cannyChannels[]={ imgCannyf, imgCannyf, imgCannyf};
merge(cannyChannels, 3, imgCanny3c);
// Convert color result to float
Mat resultf;
result.convertTo(resultf, CV_32FC3);
// Multiply color and edges matrices
multiply(resultf, imgCanny3c, resultf);
// convert to 8 bits color
resultf.convertTo(result, CV_8UC3);
// Show image
imshow("Result", result);
}
The first step is to detect the most important edges of the image. We need to remove noise from the input image before detecting the edges. There are several ways to do it. We are going to use a median filter to remove all possible small noise, but we can use other methods, such as Gaussian blur. The OpenCV function is medianBlur, which accepts three parameters: input image, output image, and the kernel size (a kernel is a small matrix used to apply some mathematical operation, such as convolutional means, to an image):
Mat imgMedian;
medianBlur(img, imgMedian, 7);
After removing any possible noise, we detect the strong edges with the Canny filter:
// Detect edges with canny
Mat imgCanny;
Canny(imgMedian, imgCanny, 50, 150);
The Canny filter accepts the following parameters:
Input image
Output image
First threshold
Second threshold
Sobel size aperture
Boolean value to indicate whether we need to use a more accurate image gradient magnitude
The smallest value between the first threshold and the second threshold is used for edge linking. The largest value is used to find initial segments of strong edges. The sobel size aperture is the kernel size for the sobel filter that will be used in the algorithm. After detecting edges, we are going to apply a small dilation to join broken edges:
// Dilate the edges
Mat kernel= getStructuringElement(MORPH_RECT, Size(2,2));
dilate(imgCanny, imgCanny, kernel);
Similar to what we did in the lomography effect, if we need to multiply our edges' result image with the color image, then we require the pixel values to be in the 0 and 1 range. For this, we will divide the canny result by 256 and invert the edges to black:
// Scale edges values to 1 and invert values
imgCanny= imgCanny/255;
imgCanny= 1-imgCanny;
We will also transform the canny 8 unsigned bit pixel format to a float matrix:
// Use float values to allow multiply between 0 and 1
Mat imgCannyf;
imgCanny.convertTo(imgCannyf, CV_32FC3);
To give a cool result, we can blur the edges, and to give smooth result lines, we can apply a blur filter:
// Blur the edgest to do smooth effect
blur(imgCannyf, imgCannyf, Size(5,5));
The first step of the algorithm is finished, and now we are going to work with the color. To get a cartoon look, we are going to use the bilateral filter:
// Apply bilateral filter to homogenizes color
Mat imgBF;
bilateralFilter(img, imgBF, 9, 150.0, 150.0);
The bilateral filter is a filter that reduces the noise of an image while keeping the edges. With appropriate parameters, which we will explore later, we can get a cartoonish effect.
The bilateral filter's parameters are as follows:
Input image
Output image
Diameter of pixel neighborhood; if it's set to negative, it is computed from a sigma space value
Sigma color value
Sigma coordinate space
With a diameter greater than five, the bilateral filter starts to become slow. With sigma values greater than 150, a cartoonish effect appears.
To create a stronger cartoonish effect, we truncate the possible color values to 10 by multiplying and dividing the pixels values:
// truncate colors
Mat result= imgBF/25;
result= result*25;
Finally, we have to merge the color and edges results. Then, we have to create a three-channel image as follows:
// Create a 3 channles for edges
Mat imgCanny3c;
Mat cannyChannels[]={ imgCannyf, imgCannyf, imgCannyf};
merge(cannyChannels, 3, imgCanny3c);
We can convert our color result image to a 32-bit float image and then multiply both images per element:
// Convert color result to float
Mat resultf;
result.convertTo(resultf, CV_32FC3);
// Multiply color and edges matrices
multiply(resultf, imgCanny3c, resultf);
Finally, we only need to convert our image to 8 bits and then show the resulting image to the user:
// convert to 8 bits color
resultf.convertTo(result, CV_8UC3);
// Show image
imshow("Result", result);
In the next screenshot, we can see the input image (left image) and the result of applying the cartoonize effect (right image):