Cutting out the noise with non-local means image restoration

• High image quality is essential in modern applications such as medicine, planetary imaging, and digital photography.
• Standard denoising techniques can be computationally expensive.
• Chartese Darnel Jones, professor of mathematics at the University of Missouri, Columbia, USA, proposes an efficient Quarter-Match Non-Local Means (QM-NLM) algorithm.
• The robust and effective method produces better images than standard denoising approaches on a range of image types, in half the computational time!

We live in an increasingly visual and digital world, using images to record life moments, diagnose illnesses, and even analyse planets. High image quality is essential in all these applications, but the digital and mechanical limitations of imaging devices mean that we need effective software to restore damaged or distorted images. Developing efficient image processing algorithms is crucial to maximising the effectiveness of imaging applications – but what exactly is a digital image?

A digital image is a visual input such as a digital photo, divided into several square picture elements, known as pixels. Finite, discrete pixel quantities like intensity and colour are stored and can be manipulated digitally, thus adjusting the image. Modern image restoration methods employ computational algorithms to analyse digital images.

Smoothing out image noise

Problems and imperfections in image gathering techniques and equipment can affect the sharpness or structure of a digital image, making it difficult to interpret. Denoising is a common technique that addresses grainy textures and discolouration in images, and the key is to remove this noise without losing important visual features.

Smoothing filters are an effective computational tool for rectifying image noise. The basic idea behind ‘local-mean smoothing filters’ is to replace a particular pixel value with the mean value for the surrounding pixels. The downside of local-mean filtering is that it can lead to image blurring.

Non-local means filters work a little differently by taking all of the pixels in an image into account. These filters calculate the mean values for all pixels, which are weighted according to their similarities to a particular target pixel. This more complex technique accurately provides sharp images without losing too many fine details. However, using every pixel in an image to restore individual pixels inevitably requires a lot of computing time. Researchers must find a way to lower the computational burden of non-local means denoising.

A new type of filter

Chartese Darnel Jones, professor of mathematics at the University of Missouri, Columbia, USA, tackles the heavy computational load of non-local means denoising using a selection technique that makes robust comparisons between similar windows in greyscale images. This new selective feature is more efficient than standard approaches as it reduces the computational burden of denoising images.

The selective approach involves using the ratio of average grey values in the neighbourhoods of compared pixels to calculate pixel weights and better select similar neighbourhoods. This restricts the mean calculations needed for each pixel within a particular search window instead of the whole image, thus lowering the number of calculations made.

Carving up neighbourhoods

An issue with selecting similar zones in an image is that these could have similar values (greyscale, intensity) but be visually different. Jones’ economical denoising algorithm uses the idea of quarter matching to allow for more specific comparisons than standard methods, effectively and efficiently enhancing image quality. This Quarter Match Non-Local Means (QM-NLM) method involves dividing neighbourhoods in an image into quarters. Quarter areas are weighted depending on their similarity to a quarter area of interest.

Denoising is a common technique that addresses grainy textures and discolouration in images, and the key is to remove this noise without losing important visual features.

First, a chosen image is reflected across its edges to allow the algorithm to denoise at the image boundaries; then, the neighbourhoods are created. These are divided into quarters, and the algorithm checks each quarter against comparable quarters in other neighbourhoods.

If a quarter in another neighbourhood is too dissimilar to a particular quarter, the whole neighbourhood is considered to be dissimilar, and the algorithm goes on to calculate values for the next neighbourhood of interest. The code then performs pixel-by-pixel restoration depending on the weights given to each neighbourhood.

Quarter-match filtering reduces the number of non-similar neighbourhoods in an image more effectively than standard non-local methods, which strengthens the filtering process while minimising its computational burden. This leads to sharper images with better computational efficiency than standard non-local means methods.

Sharpening photos

Jones and his team used several photos with different features, including a clock and a woman’s face, to compare denoising techniques and test the effectiveness and robustness of QM-NLM. When comparing QM-NLM with standard noise removal methods and an original image, Jones found that the selective process was more accurate and needed less computer time than the other methods. Amazingly, QM-NLM used only half the computational time of standard non-local means!

Studying the planet

Synthetic aperture radar (SAR) images are created using radar equipment usually mounted on a spacecraft or orbiting platform above the Earth. These devices produce energy and measure the amount of energy bounced back towards them from the Earth. SAR signals can be used to collect subtle detailed information about a planet’s surface, such as morphology and moisture levels.

Jones’ image restoration method works well on photos, SAR images, and MRIs, providing high image quality at only half the computational expense of standard denoising approaches.

Intrigued by the question of how the method would compare to the denoising capacity of other methods used in SAR applications, Jones restored a noisy SAR image using both QM-NLM and standard algorithms. As in the case of photos, QM-NLM successfully addressed image noise while maintaining original image features. The QM-NLM-processed image edges were much sharper than those produced using other approaches, clearly showing the effectiveness of the novel method.

Imaging the body

The next step was to test the effectiveness of QM-NLM on yet another type of application: magnetic resonance images (MRIs). MRIs are routinely used in medical applications, and their precision is key to detecting illnesses early. This makes MRIs a key target for high-quality image restoration. QM-NLM clearly detected edges in MRIs of a liver and gallstones, effectively and efficiently restoring the images.

Robust image restoration

Jones’ robust and broadly applicable image restoration method works well on photos, SAR images, and MRIs, providing high image quality at only half the computational expense of standard denoising approaches.

This work paves the way for fast, high-quality image restoration in almost every aspect of the modern world, from space applications
to medicine.