 Open Access
 Total Downloads : 17
 Authors : M. Divya, V P S Naidu
 Paper ID : IJERTCONV6IS13001
 Volume & Issue : NCESC – 2018 (Volume 6 – Issue 13)
 Published (First Online): 24042018
 ISSN (Online) : 22780181
 Publisher Name : IJERT
 License: This work is licensed under a Creative Commons Attribution 4.0 International License
MultiImaging Sensor Data Fusion using 2D DST
M. Divya,
Dept. of ECE,
JNTU College of Engineering, Ananthapuramu, India,
V P S Naidu
Multi Sensor Data Fusion Lab, CSIR – National Aerospace Laboratories, Bangalore,
India
Abstract: Image fusion is a process of combining two or more
X (k)
2 N 1 x(n) sin (n 1)(k 1) , 0 n N 1
images into a single image without any loss of information. Nowadays, image fusion is playing a major role in research areas. In this paper, discrete sine transform based multi imaging sensor data fusion algorithms are developed,
n0
N 1
N
0 k N 1
(1)
implemented and tested using fusion quality evaluation metrics. The proposed fusion algorithms are compared with discrete Cosine transform based fusion algorithms and it is observed from the results that they are almost similar and comparable. The proposed fusion algorithms are computationally simple and can be used in real time applications.
The DSTI is orthogonal and it is exactly equivalent to a
DFT of real sequence that is odd around the 0th and middle points, scaled by 0.5. The DSTI is its own inverse.
The two dimensional (2D) discrete sine transform
X (k1 , k2 ) o f an image x(n1 , n2 ) of size N1 N 2 is defined as
2 2 N1 1N2 1
(n 1)(k 1)
Keywords: Discrete Sine Transform, Image Fusion, Discrete
X (k1 , k2 ) x(n1 , n2 )sin 1 1
Cosine Transform.
N1 N2 n1 0 n2 0
N1
(2)
(n2 1)(k2 1)

INTRODUCTION
Oflate, many researchers developed different image fusion algorithms to combine multiple images into a single image.
sin
N2
Where, 0 k1 , k2
N1
1, N 2 1
Image fusion using corresponding pixel averaging is the simplest method of all those different fusion algorithms. In image fusion literature, no researchers use the simple and well proven discrete Sine transform (DST) for fusion application. In this paper, an attempt is made to use DST for developing image fusion algorithms. This paper introduces different DST based image fusion techniques, studied their performance. The goal of this paper is to check the performance of image fusion based on DST algorithm using different block sizes and to compare with DCT based image fusion algorithm [1].
Here, 2D DST is a self inverse [2].
3. IMAGE FUSION
The architecture of 2D DST based image fusion is shown in Fig.1.The images to be fused are divided into non overlapping blocks of size NxN. DST is performed on corresponding blocks of images to be fused and fusion rules are applied on DST coefficients to get the fused DST coefficients. This process is repeated for each block of an image. This gives the fused image as output.
Fusion rules
Image 1

DISCRETE SINE TRANSFORM
The discrete sine transform (DST) is a Fourierrelated transform similar to the discrete Fourier transform (DFT) but using a purely real matrix. It is the imaginary part of a DFT and its length is approximately twice the length of DFT. It is operating on real data with odd symmetry, whereas the DFT of a real and odd function is imaginary and odd. DST expresses of finitely discrete sequence in terms of sine functions oscillating at different frequencies. There are eight standard DST variants of which four are common and widely used for signal analysis.DST is a linear and invertible function. In literature eight DST
Image 2
DST
DST
DST
Fused Image
equations are there in that DST1 to DST4 are widely used. In this paper DST1 is using because it is simple and it has its own inverse so no need of applying inverse DST.
Fig1: Framework of 2D DST based image fusion architecture.
The one dimensional (1D) discrete sine transform X (k ) of a signal x(n) of length N is defined as:
The fusion rules which are used in image fusion process 4. FUSION EVALUATION METRICS
are presented in this section. Let consider X1 be the DST coefficients of image block from image 1 and similarly X 2 be the DST coefficients of image block from image 2.
Image fusion process is used to get good quality image and to evaluate the fusion quality, many fusion quality evaluation metrics are proposed in the open literature.
When the ground truth image is available the following
Assume that the image block is of size the fused DST coefficients.
3.1 DSTav
N N
and
X f be
fusion evaluation metrics are used to evaluate the performance of the different image fusion algorithms [4]:
4.1 SPATIAL FREQUENCY (SF)
The frequency in spatial domain indicates the overall
In this fusion rule, all DST coefficients from both image blocks are averaged to get fused DST coefficients. It is a very simple and basic image fusion technique in DST
activity level in the fused image and it is computed as row and column frequency of the images.
domain.
Spatial frequency criterion is: SF
Where, row frequency of the image:
RF 2 CF 2 (7)
X f (k1, k2 ) 0.5*(X1(k1, k2 ) X 2 (k1, k2 ))
(3)
1 M N
f
f
RF [x (n1, n2) x (n1, n2 1)]2
Where, k1 , k2
= 0,1,2,… …,N 1
MN n11 n22
3.2 DSTmax
Column frequency of the image:
By collecting DC components from both frames the
CF
1 N
[x (n1, n2) x (n11, n2)]2M
f
f
average can be done. From the AC components greatest magnitude AC coefficients are considered and then the detailed coefficients correspond to sharper brightness changes in the images such as edges, object boundaries etc. The DC and AC coefficients are fluctuating around zero
MN n21n12
SF indicates the overall activity level in the fused image. The fused image with high SF will be considered [2, 3].
X f (0,0) 0.5* (X1 (0,0) X 2 (0,0))
1
Where, k1 , k2 = 0,1,2,… …,N 1
(4)
4.2 Peak Signal to Noise Ratio (PSNR)
Its value will be high when the reconstructed and reference images are similar. Higher PSNR value implies better
f 1 2
X1 (k1, k 2 ) X (k ,k )
 X1 (k , k 2 )   X2 (k1 , k 2 ) 
reconstruction. The peak signal to noise ratio is computed as:
2 1 2
X (k , k )
3.3 DSTah
 X1 (k1 , k 2 )   X2 (k1 , k 2 ) 
PSNR 10log
L2
(8)
By collecting the lowest AC components and including DC
10 1 M N x (n1, n2) x
(n1, n2)2
coefficients averaging process can be performed and the
MN
r f
n21n11
AC coefficients which are remained, are chosen based on largest magnitude.
Where, L is the number of gray levels in the image and xr
X (k , k ) 0.5*(X (k , k ) X (k , k )) (5)
is the reference/ground truth image.
f 1 2
1 1 2
2 1 2
Where, k1 , k2
= 0,1,2,… …,0.5N1
4.3 Entropy (H)
f 1 2
X1 (k1 , k 2 ) X (k ,k )
X (k , k )
 X1 (k , k 2 )   X2 (k1 , k 2 ) 
1
 X (k , k )   X (k , k ) 
Entropy is used to measure the information content of an image. Using entropy, the information content of a fused
image is:
2 1 2
1 1 2
2 1 2
L
H
Where, k1 , k2
= 0,1,2,… …,N 1
H hx (i) log 2hx (i)
(9)
3.4 DSTe
In this DC components are averaged together. And the AC
Where,
h
x f
f f
i0
(i) is the normalized histogram of the fused image
coefficients correspond to the frequency band having largest energy considered.
x f and
LH number of frequency bins in the histogram.
X f (0,0) 0.5*(X1(0,0) X 2 (0,0))
X k , k Ej2 Ej2
(6)
Entropy is sensitive to noise and other unwanted rapid
fluctuations. The information entropy measures the
X k , k
1 1 2
richness of information in an image. Hence, entropy is
f 1 2
X 2
k1 , k2
Ej2 Ej2
higher, performance is better [4].
Where, k1 , k2
= 0,1,2,… …,N 1 and
j k1 k2

RESULTS AND DISCUSSIONS
The fused image will get by applying the DST on fused coefficients as: xf DST(X f )
Note: DCT based image fusion algorithms are very similar to DST based image fusion algorithms (Section 3.1 to 3.4) by simply replacing DST by DCT.
The main objective of this paper is to fuse multiple images using 2D DST. The ground truth and source images (images to be fused) are shown in Fig2 and Fig3 respectively. Fused images using different DST algorithms are shown in Figs4 to 7. The computational time of the proposed fusion algorithms are shown in Table1. It is
observed that, in DST based fusion process at 16×16 block size it is taking less time for all fusion rules except for DSTe(at 32×32 it is taking less time). whereas, in DCT at 128×128 block size it is taking less time for all fusion rules except for DSTe(at 64×64 it is taking less time). Image fusion quality evaluation metrics for the proposed fusion algorithms are shown in Table2&3. The metrics shown with bold indicates better results and corresponding algorithms will be the best among others. It is observed that SF is same irrespective of block size. The comparison of DST and DCT based image fusion using block size 8×8 is shown in Table4.It is observed that, fusion algorithm based on DST and DCT are performing almost similar. In fact, DSTav provides better PSNR than DCTav as shown in Table4. The fused images with different fusion algorithms are shown in Figs4 to 7. It is observed that, DST av shows better fusion image compare to other DST based fusion algorithms followed by DSTe. These results are comparable with DCT based fusion algorithms.

CONCLUSION
Multi imaging sensor data fusion algorithms using 2DDST are presented and evaluated using fusion quality evaluation
metrics. The results are compared with DCT based fusion algorithms available in open literature. It is observed that fusion algorithms based DST and DCT are performing almost similar. The proposed fusion algorithms are computationally simple and can be used in realtime image fusion applications.

REFERENCES

VPS Naidu, Discrete Cosine Transform based Image Fusion Techniques, Journal of Communication, Navigation and Signal Processing (January 2012) Vol. 1, No. 1, pp. 3545.

https://en.wikipedia.org/wiki/Discrete_sine_transform.

K. Rao and P. Yip, Discrete Cosine Transform, Algorithm, Advantages, applications, Academic Press, 1990.

S.A. Martucci, Symmetric Convolution and the Discrete Sine and Cosine transforms, IEEE Trans. Sig. Processing, SP42, pp.10381051, 1994.
Table (1): Computational Time (in sec)
Fusion rules 
DST/DCT 
Block Size(rows & columns) 

2×2 
4×4 
8×8 
16×16 
32×32 
64×64 
128×128 
256×256 
512×512 

Av 
DST 
7.4232 
3.8034 
1.9731 
1.8318 
2.6496 
4.7131 
9.1133 
17.9871 
35.9240 
DCT 
22.6578 
5.9152 
1.5946 
0.4896 
0.1859 
0.1617 
0.1085 
0.2009 
0.2738 

max 
DST 
7.8969 
3.4496 
2.0177 
1.8393 
2.6509 
4.7310 
9.0845 
18.1372 
36.0051 
DCT 
23.1537 
6.0213 
1.6209 
0.4881 
0.1886 
0.1691 
0.1164 
0.2028 
0.2831 

ah 
DST 
8.7456 
3.6002 
2.0647 
1.8703 
2.6434 
4.8103 
9.0807 
17.9239 
35.8718 
DCT 
23.7942 
6.1585 
1.6665 
0.5042 
0.1988 
0.1665 
0.1226 
0.2065 
0.2847 

e 
DST 
27.0125 
14.0739 
7.4312 
5.4210 
5.3831 
8.2326 
14.9772 
29.0308 
58.1472 
DCT 
34.4017 
11.9302 
4.8228 
2.3368 
1.4845 
1.4178 
1.8222 
4.2950 
9.7422 
Table (2): Spatial Frequency
Fusion rules 
DST/DCT 
Block Size(rows & columns) 

2×2 
4×4 
8×8 
16×16 
32×32 
64×64 
128×128 
256×256 
512×512 

Av 
DST 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 

DCT 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 
0.0358 

max 
DST 
0.0450 
0.0489 
0.0515 
00531 
0.0535 
0.0546 
0.0555 
0.0548 
0.0569 

DCT 
0.0544 
0.0643 
0.0667 
0.0669 
0.0668 
0.0668 
0.0666 
0.0667 
0.0611 

ah 
DST 
0.0557 
0.0503 
0.0497 
0.0486 
0.0474 
0.0487 
0.0493 
0.0474 
0.0508 

DCT 
0.0544 
0.0528 
0.0510 
0.0501 
0.0496 
0.0493 
0.0491 
0.0489 
0.0463 

e 
DST 
0.0556 
0.0560 
0.0580 
0.0610 
0.0631 
0.0677 
0.0676 
0.0668 
0.0644 

DCT 
0.0544 
0.0642 
0.0666 
0.0668 
0.0668 
0.0667 
0.0667 
0.0667 
0.0520 
Table (3): Entropy of DST/DCT
Fusion rules 
DST/DCT 
Block Size(rows & columns) 

2×2 
4×4 
8×8 
16×16 
32×32 
64×64 
128×128 
256×256 
512×512 

Av 
DST 
4.0546 
4.0232 
4.0217 
4.0152 
4.0181 
4.0182 
4.0174 
4.0182 
4.0176 

DCT 
3.9888 
3.9888 
3.9891 
3.9889 
3.9882 
3.9937 
3.9933 
3.9978 
3.9943 

Max 
DST 
3.9478 
3.9000 
3.9165 
3.9698 
4.0570 
4.1263 
4.2360 
4.5376 
4.6268 

DCT 
4.0179 
4.0090 
4.0099 
4.0006 
3.9488 
3.9918 
4.1587 
3.7418 
5.5020 

Ah 
DST 
3.9478 
3.9480 
3.9783 
4.0183 
4.0775 
4.1074 
4.2049 
4.3844 
4.4422 

DCT 
4.0179 
4.0246 
4.0253 
4.0286 
4.0412 
4.0701 
4.1477 
4.1916 
4.8325 

E 
DST 
3.9464 
3.8561 
3.8397 
3.8748 
3.9539 
4.0498 
4.0651 
4.0914 
4.3567 

DCT 
4.0173 
4.0064 
4.0057 
3.9889 
3.9385 
3.9669 
3.9820 
3.7125 
4.8561 
Table (4): Comparison between DST and DCT
Block Size(8×8) 

DSTav 
DSTmax 
DSTah 
DSTe 
DCTav 
DCTmax 
DCTah 
DCTe 

Computational Time(in sec) 
1.9731 
2.0177 
2.0647 
7.4312 
1.5946 
1.6209 
1.6665 
4.8228 
Spatial 
0.0358 
0.0515 
0.0497 
0.0580 
0.0358 
0.0667 
0.0510 
0.0666 
PSNR 
46.7141 
43.9522 
45.0890 
44.4066 
38.4255 
40.7588 
38.6715 
40.7857 
Entropy 
4.0217 
3.9165 
3.9783 
3.8397 
3.9891 
4.0099 
4.0253 
4.0057 
Fig2: Ground Truth ImageSARAS
Fig3: Source Images (images to be fused) SARAS
Fig4a: Fused image using DSTe fusion algorithm Fig4b: Fused image using DCTe fusion algorithm
Fig5a: Fused image using DSTah fusion algorithm Fig5b: Fused image using DCTah fusion algorithm
Fig6a: Fused image using DSTmax fusion algorithm Fig6b: Fused image using DCTmax fusion algorithm
Fig7a: Fused image using DSTavfusion algorithm Fig7b: Fused image using DCTav fusion algorithm