OpenCV教程（22）-- 图像平滑 Image Smoothing

Table of Contents

Goals

2D Convolution ( Image Filtering )

Image Blurring (Image Smoothing)

1. Averaging

2. Gaussian Blurring

3. Median Blurring

4. Bilateral Filtering

Additional Resources

Goals

Learn to:

• Blur the images with various low pass filters
• Apply custom-made filters to images (2D convolution)

2D Convolution ( Image Filtering )

As in one-dimensional signals, images also can be filtered with various low-pass filters(LPF), high-pass filters(HPF) etc. LPF helps in removing noises, blurring the images etc. HPF filters helps in finding edges in the images.

OpenCV provides a function cv.filter2D() to convolve a kernel with an image. As an example, we will try an averaging filter on an image. A 5x5 averaging filter kernel will look like below:

Operation is like this: keep this kernel above a pixel, add all the 25 pixels below this kernel, take its average and replace the central pixel with the new average value. It continues this operation for all the pixels in the image. Try this code and check the result:

# !/usr/bin/env python
# -*- coding: utf-8 -*-
"""
# @Time    : 2019/10/6 17:18
# @Author  : HaoWang
# @Site    : HongKong, China
# @project : $[PROJECT_NAME]
# @File    : 0801.image filter2d.py
# @Software: PyCharm
# @license: haowanghk@gmail.com 
"""

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt

# read noisy image
img = cv.imread('../pictures/noisy_color.jpg')

# define filter kernel
kernel = np.ones((7,7),np.float32)/49

dst = cv.filter2D(img,-1,kernel)

# change color space fromBGR to RGB
img = cv.cvtColor(img,cv.COLOR_BGR2RGB)
dst = cv.cvtColor(dst,cv.COLOR_BGR2RGB)

plt.subplot(121),plt.imshow(img),plt.title('Original')

plt.xticks([]), plt.yticks([])

plt.subplot(122),plt.imshow(dst),plt.title('Averaging : filter kernel '+ str(np.shape(kernel)))

plt.xticks([]), plt.yticks([])

plt.show()

Result:

Image Blurring (Image Smoothing)

Image blurring is achieved by convolving the image with a low-pass filter kernel. It is useful for removing noises. It actually removes high frequency content (eg: noise, edges) from the image. So edges are blurred a little bit in this operation. (Well, there are blurring techniques which doesn't blur the edges too). OpenCV provides mainly four types of blurring techniques.

1. Averaging

This is done by convolving image with a normalized box filter. It simply takes the average of all the pixels under kernel area and replace the central element. This is done by the function cv.blur() or cv.boxFilter(). Check the docs for more details about the kernel. We should specify the width and height of kernel. A 3x3 normalized box filter would look like below:

If you don't want to use normalized box filter, use cv.boxFilter(). Pass an argument normalize=False to the function.

Check a sample demo below with a kernel of 5x5 size:

import cv2 as cv

import numpy as np

from matplotlib import pyplot as plt

img = cv.imread('opencv-logo-white.png')

blur = cv.blur(img,(5,5))

plt.subplot(121),plt.imshow(img),plt.title('Original')

plt.xticks([]), plt.yticks([])

plt.subplot(122),plt.imshow(blur),plt.title('Blurred')

plt.xticks([]), plt.yticks([])

plt.show()

Result image

2. Gaussian Blurring

In this, instead of box filter, gaussian kernel is used. It is done with the function, cv.GaussianBlur(). We should specify the width and height of kernel which should be positive and odd. We also should specify the standard deviation in X and Y direction, sigmaX and sigmaY respectively. If only sigmaX is specified, sigmaY is taken as same as sigmaX. If both are given as zeros, they are calculated from kernel size. Gaussian blurring is highly effective in removing gaussian noise from the image.

If you want, you can create a Gaussian kernel with the function, cv.getGaussianKernel().

The above code can be modified for Gaussian blurring:

blur = cv.GaussianBlur(img,(5,5),0)

Result:

3. Median Blurring

Here, the function cv.medianBlur() takes median of all the pixels under kernel area and central element is replaced with this median value. This is highly effective against salt-and-pepper noise in the images. Interesting thing is that, in the above filters, central element is a newly calculated value which may be a pixel value in the image or a new value. But in median blurring, central element is always replaced by some pixel value in the image. It reduces the noise effectively. Its kernel size should be a positive odd integer.

In this demo, I added a 50% noise to our original image and applied median blur. Check the result:

median = cv.medianBlur(img,5)

Result:

4. Bilateral Filtering

cv.bilateralFilter() is highly effective in noise removal while keeping edges sharp. But the operation is slower compared to other filters. We already saw that gaussian filter takes the a neighbourhood around the pixel and find its gaussian weighted average. This gaussian filter is a function of space alone, that is, nearby pixels are considered while filtering. It doesn't consider whether pixels have almost same intensity. It doesn't consider whether pixel is an edge pixel or not. So it blurs the edges also, which we don't want to do.

Bilateral filter also takes a gaussian filter in space, but one more gaussian filter which is a function of pixel difference. Gaussian function of space make sure only nearby pixels are considered for blurring while gaussian function of intensity difference make sure only those pixels with similar intensity to central pixel is considered for blurring. So it preserves the edges since pixels at edges will have large intensity variation.

Below samples shows use bilateral filter (For details on arguments, visit docs).

blur = cv.bilateralFilter(img,9,75,75)

Result:

See, the texture on the surface is gone, but edges are still preserved. 

and the PSNR values

Code:

# !/usr/bin/env python
# -*- coding: utf-8 -*-
"""
# @Time    : 2019/10/6 17:18
# @Author  : HaoWang
# @Site    : HongKong, China
# @project : $[PROJECT_NAME]
# @File    : 0801.image filter2d.py
# @Software: PyCharm
# @license: haowanghk@gmail.com 
"""

import numpy as np
import cv2 as cv
import tensorflow as tf
from PSNR import psnr
from matplotlib import pyplot as plt




# read noisy image
img = cv.imread('../pictures/room.jpg')

# define filter kernel
kernel = np.ones((5,5),np.float32)/25

dst_2d = cv.filter2D(img,-1,kernel)

dst_g = cv.GaussianBlur(img,(5,5),0)

# .   cv.boxFilter(src, ddepth, ksize)
#         @param src input image.
#     .   @param dst output image of the same size and type as src.
#     .   @param ddepth the output image depth (-1 to use src.depth()).
#     .   @param ksize blurring kernel size.
dst_box = cv.boxFilter(img, -1, (5,5))

# bilateral filter
dst_bilateral = cv.bilateralFilter(img,16,75,75)

# change color space fromBGR to RGB
img = cv.cvtColor(img,cv.COLOR_BGR2RGB)
dst_g = cv.cvtColor(dst_g,cv.COLOR_BGR2RGB)
dst_2d = cv.cvtColor(dst_2d,cv.COLOR_BGR2RGB)
dst_box = cv.cvtColor(dst_box,cv.COLOR_BGR2RGB)
dst_bilateral = cv.cvtColor(dst_bilateral, cv.COLOR_BGR2RGB)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    PSNR_2d = sess.run(psnr(img, dst_2d))
    PSNR_bil = sess.run(psnr(img, dst_bilateral))
    PSNR_Gua = sess.run(psnr(img, dst_g))

# plt.subplot(111),plt.imshow(img),plt.title('Original')
#
# plt.xticks([]), plt.yticks([])

# plt.subplot(111),plt.imshow(dst_bilateral),plt.title('Bilateral Filter & PSNR:'+str(PSNR_bil))
#
# plt.xticks([]), plt.yticks([])

# plt.subplot(111),plt.imshow(dst_g),plt.title("Gaussian " + str(np.shape(kernel))+ "PSNR : " +str(PSNR_Gua))
#
# plt.xticks([]), plt.yticks([])

plt.subplot(111),plt.imshow(dst_2d),plt.title("2dFilter " + str(np.shape(kernel))+ "PSNR : " +str(PSNR_2d))

plt.xticks([]), plt.yticks([])

plt.show()

Additional Resources

1. Details about the bilateral filtering