Fusing medical images is a topic of interest in processing medical images. This is achieved to through fusing information from multimodality images for the purpose of increasing the clinical diagnosis accuracy. This fusion aims to improve the image quality and preserve the specific features. The methods of medical image fusion generally use knowledge in many different fields such as clinical medicine, computer vision, digital imaging, machine learning, pattern recognition to fuse different medical images. There are two main approaches in fusing image, including spatial domain approach and transform domain approachs. This paper proposes a new algorithm to fusion multimodal images. This algorithm is based on Entropy optimization and the Sobel operator. Wavelet transform is used to split the input images into components over the low and high frequency domains. Then, two fusion rules are used for obtaining the fusing images. The first rule, based on the Sobel operator, is used for high frequency components. The second rule, based on Entropy optimization by using Particle Swarm Optimization (PSO) algorithm, is used for low frequency components. Proposed algorithm is implemented on the images related to central nervous system diseases. The experimental results of the paper show that the proposed algorithm is better than some recent methods in term of brightness level, the contrast, the entropy, the gradient and visual information fidelity for fusion (VIFF), Feature Mutual Information (FMI) indices.

Fusing medical images is combining the information of multimodality images to acquire accurate information [

There are two main approaches in fusing image, including spatial domain approach and transform domain approachs [

Recently, there are many new techniques in fusing images. Mishra et al. [

The new techniques which are based deeplearning, are proposed recently. In [

The medical image fusion approach, uses wavelet transform, usually applies the average selection rule on low frequency components and max selection rule on high frequency components. This causes the resulting image to be greatly grayed out compared to the original image because the grayscale values of the frequency components of the input images differ greatly. In addition, some recent methods focus mainly on the fusion so that they can reduce the contrast and brightness of the fused image. This makes it difficult to diagnose and analyze based on the fused image. To overcome the limitations, this paper proposes a novel algorithm for fusing multimodal images by combining of Entropy optimization and the Sobel operator.

The main contributions of this article include:

Propose a new algorithm based on the Sobel operator for combining high frequency components.

Propose a novel algorithm that is used for fusing multimodal images based on wavelet transform.

Propose a new algorithm based on the Sobel operator for combining low frequency components. This algorithm is combined by Entropy based on parameter optimization using PSO algorithm. The fusion image preserves colors and textures similarly to input image.

The remaining of this article is structured as follows. In Section 2, some related works are presented. The proposed algorithm about image fusion is presented in Section 3. Section 4 presents some experiments of our algorithm and other related algorithms on selected images. Conclusions and the future researches are given in Section 5.

Wavelet Transformation (WT) is a mathematical tool [

When DWT performed, the size of image LL is four times smaller than the image LL of the previous stage. Therefore, if the input image is disaggregated into 3 levels, size of the final approximate image is 64 times smaller than the input image. Wavelet transformation of image is illustrated as in

PSO is an algorithm about finding solutions to optimization problems [

where:

Reference [

With _{F}

Average method:

Select Maximum:

Select Minimum:

The algorithm of combining high frequency components based on Sobel operator (CHCSO) is stated as follows:

The main steps of CHCSO include:

In this section, a new algorithm for fusing medical images named as the Entropy optimization and Sobel operator based Image Fusion (ESIF) is proposed. The general framework of the algorithm ESIF is shown in

Where, _{1} is PET or SPEC image (color images), _{2} is CT or MRI image (grey images).

According to

_{1} in Red, Blue and Green (RGB) color space to Hue, Saturation, Intensity (HIS) color space to get

_{1}, LH_{1}, HH_{1}) and (HL_{2}, LH_{2}, HH_{2}) to get HL, LH, HH using the rule which is based on the algorithm CHCSO as follows:

_{1}) and (LL_{2}) to get LL using the rule as follows:

_{fusion} using IDWT transformation.
_{fusion} and _{2}.

_{fusion}, _{Img1}, _{Img1} in HIS color space to RGB color space to obtain the output fused image.

The proposed algorithm has some advantages, including:

Combining the high frequency components is adaptive using the algorithm CHCSO with the Sobel operator instead of the rule Select Maximum [

Combining the low frequency components using weighted parameters which are found by using an algorithm PSO with the optimization of objective function in formula

Overcome the limitations of the approach that is based on wavelet transform as mentioned in section I.

Input data is downloaded from Atlas [

To assess image quality, we use the measures such as the brightness level (

Herein, we illustrate the experiment with 5 slices 070, 080 and 090, 004, 007 as below. Input and output images of the fused methods are presented in

Slice | Input images | Output images of | Output images of ESIF (Proposed) | |||
---|---|---|---|---|---|---|

Img_{1} |
Img_{2} |
WIF | PCAIF | CSMCA | ||

070 | ||||||

080 | ||||||

090 | ||||||

004 | ||||||

007 |

From the output images of four methods in

The WIF and PCAIF methods do not highlight the boundary of the areas in the resulting images.

The CSMCA method even generates very dark fused image compared to WIF and PCAIF methods. This makes it difficult to distinguish areas in the image.

The fused images generated by the proposed method has better contrast and bright and clearly distinguishing the areas than fused images using the compared methods.

For the quantity evaluation, the values of criteria ^{2}, E, G, VIFF and FMI indexes of the output images that generated by the fusion methods are calculated and given in

Slice | Index | Other methods | ESIF (Proposed) | ||
---|---|---|---|---|---|

WIF | WPCAIF | CSMCA | |||

070 | 0.2327 | 0.2447 | 0.1622 | ||

0.0697 | 0.0739 | 0.0444 | |||

E | 5.1737 | 5.1090 | 4.5970 | ||

G | 0.0465 | 0.0420 | 0.0567 | ||

VIFF | 0.4046 | 0.4476 | 0.7047 | ||

FMI | 0.8549 | 0.8865 | 0.8672 | ||

080 | 0.2294 | 0.2390 | 0.1578 | ||

0.0691 | 0.0720 | 0.0415 | |||

E | 5.0248 | 4.9701 | 4.5108 | ||

G | 0.0458 | 0.0406 | 0.0558 | ||

VIFF | 0.3995 | 0.4186 | 0.6979 | ||

FMI | 0.8518 | 0.8798 | 0.8681 | ||

090 | 0.2172 | 0.2296 | 0.1521 | ||

0.0689 | 0.0740 | 0.0409 | |||

E | 4.7520 | 4.6961 | 4.2308 | ||

G | 0.0399 | 0.0365 | 0.0484 | ||

VIFF | 0.3911 | 0.4129 | 0.6934 | ||

FMI | 0.8535 | 0.8830 | 0.8700 | ||

004 | 0.1362 | 0.1384 | 0.0865 | ||

0.0275 | 0.0276 | 0.0144 | |||

E | 6.0988 | 5.8967 | 5.0934 | ||

G | 0.0325 | 0.0269 | 0.0372 | ||

VIFF | 0.4243 | 0.4442 | 0.6953 | ||

FMI | 0.8411 | 0.8649 | 0.8526 | ||

007 | 0.1710 | 0.1730 | 0.1031 | ||

0.0380 | 0.0382 | 0.0179 | |||

E | 6.0416 | 5.8606 | 5.2180 | ||

G | 0.0334 | 0.0276 | 0.0396 | ||

VIFF | 0.4386 | 0.4595 | 0.6798 | ||

FMI | 0.8452 | 0.8669 | 0.8471 |

From the results in ^{2}, E, G, VIFF and FMI obtained are the best values on all slices. To compare the results on each criterion, the average values of ^{2}, E, G, VIFF and FMI indexes obtained by applying four methods on five slices are visually presented as in

^{2}, E obtained by CSMCA are the worst values comparing with those of other methods. However, the average values of G, VIFF and FMI obtained by this method are higher than those of VIF. Comparing with PCAIF, CSMCA is better in two criteria (G and VIFF). This means that the quality of the fused images of the CSMCA method is not always good and unstable.

Moreover, from the results in

This paper introduces the new algorithm of fusing multimodal images based on Entropy optimization and the Sobel operator (ESIF). This algorithm aims to get the fused images without reducing the brightness and contrast. The proposed method has advantages as the adaptability of combining the high frequency components by using the algorithm CHCSO with the Sobel operator; the high performance in combining the low frequency components based on the weighted parameter obtained by using an algorithm PSO. Apart from that, our proposed method overcomes the limitations of wavelet transform based approaches.

The experimental results on five different slices of images show the higher performance of proposed method in term the brightness level, the contrast, the entropy, the gradient and VIFF, FMI indices. For further works, we intend to integrate the parameter optimization in image processing and apply the improvement method in other problems.