Glare removal With Inpainting[OpenCV Python]
This article is originally posted here
Glare is a visual sensation caused by excessive and uncontrolled brightness. It can be disabling or simply uncomfortable. It is subjective, and sensitivity to glare can vary widely. Older people are usually more sensitive to glare due to the ageing characteristics of the eye.
First, We need to detect the place where the glare exists. We can easily identify them using global binarization since when a glare exists commonly, that pixel value is higher than 180. Using that, we can detect the glare. So we need to get the pixels which are greater than 180, so then we do the removing part.
The below function is used to get the mask of an image in which pixels are white when the pixels are greater than 180 and below its black. We have actual glare places of the image as white and other places as black.
def create_mask(image):
gray = cv2.cvtColor( image, cv2.COLOR_BGR2GRAY )
blurred = cv2.GaussianBlur( gray, (9,9), 0 )
_,thresh_img = cv2.threshold( blurred, 180, 255, cv2.THRESH_BINARY)
thresh_img = cv2.erode( thresh_img, None, iterations=2 )
thresh_img = cv2.dilate( thresh_img, None, iterations=4 )
# perform a connected component analysis on the thresholded image,
# then initialize a mask to store only the "large" components
labels = measure.label( thresh_img, neighbors=8, background=0 )
mask = np.zeros( thresh_img.shape, dtype="uint8" )
# loop over the unique components
for label in np.unique( labels ):
# if this is the background label, ignore it
if label == 0:
continue
# otherwise, construct the label mask and count the
# number of pixels
labelMask = np.zeros( thresh_img.shape, dtype="uint8" )
labelMask[labels == label] = 255
numPixels = cv2.countNonZero( labelMask )
# if the number of pixels in the component is sufficiently
# large, then add it to our mask of "large blobs"
if numPixels > 300:
mask = cv2.add( mask, labelMask )
return maskWhat we have done in this function is, We first convert the image to grayscale, using a Gaussian matrix (9x9) to blur the image to reduce the noise. Setting a threshold value to 180 in the global threshold method to convert our blurred image into a binary image where the pixel value is above 180 is the white other is black. We might have small blobs of noise; for that, we have done a series of erosion and dilations to our binary image.
After this dilation, erosion of our image, we might have small noises. For that, we performed a connected component analysis on the threshold image. measure.labels method of the scikit-image library is used for the connected component analysis. A new black image is created using the np.zeros method with the exact shape of the binary image. It is named as a mask.
We start looping over each of the unique labels. If the label is zero, then we know we are examining the background region and can safely ignore it. Otherwise, we construct a mask for just the current. Then counts the number of non-zero pixels in the labelMask. If numPixels exceeds a pre-defined threshold (in this case, a total of 300 pixels), then we consider the blob “large enough” and add it to our mask. This detection method was inspired from here. The author explained this method well there.
So our mask will be like below
We spotted our glare/bright places of the images. We can remove these spots using various methods.
- Inpainting method
- CLAHE method
Inpainting method of OpenCV
Filling some region of the image in different ways is called as inpainting in image preprocessing. Basically, inpainting is just filling gaps.
So which are the ways we can fill it in python OpenCV. You can fill it with Naiver-Stokes method or Fast — Marching method.
Naiver-Stokes method
Image intensities of the region can be updated using the partial differential equation, and the smoothness of the image can be calculated by the image Laplacian (The Laplacian is a 2-D isotropic measure of the 2nd spatial derivative of an image. The Laplacian of an image highlights regions of rapid intensity change and is therefore often used for edge detection (see zero-crossing edge detectors). The Laplacian is often applied to an image that has first been smoothed with something approximating a Gaussian smoothing filter in order to reduce its sensitivity to noise, and hence the two variants will be described together here. The operator normally takes a single gray-level image as input and produces another gray-level image as output)
Laplacian and partial differential equations can be used to preserve the edges and continue to propagate colour information in smooth regions. This is one of the methods to do image inpainting.
paper → https://www.math.ucla.edu/~bertozzi/papers/cvpr01.pdf
Fast Marching method
a weighted average over a known image neighbourhood of the pixel is used for image smoothness to inpaint. The known neighbourhood pixels and gradients are used to estimate the colour of the pixel to be inpainted.
We can use any of the above algorithms to inpaint.
How to use in OpenCV python
dst = cv2.inpaint( src, inpaintMask,inpaintRadius,flags)Here
src → The input glared image
inpaintMask →A binary mask indicating pixels to be inpainted.
dst → Output image
inpaintRadius →Neighborhood around a pixel to inpaint.
flags → INPAINT_NS,(Navier-Stokes based method) or INPAINT_TELEA (Fast marching based method)
When we select the inpaintRadius, if the regions to be inpainted are thin, smaller values produce better results (less blurry).
Let’s apply this to our image
References
https://dsp.stackexchange.com/questions/1215/how-to-remove-a-glare-clipped-brightness-from-an-image
Spatial Filters — Laplacian/Laplacian of Gaussian. http://homepages.inf.ed.ac.uk/rbf/HIPR2/log.html
