Multi-Imaging Sensor Data Fusion using 2D DST

Download Full-Text PDF Cite this Publication

Text Only Version

Multi-Imaging Sensor Data Fusion using 2D DST

M. Divya,

Dept. of ECE,

JNTU College of Engineering, Ananthapuramu, India,

V P S Naidu

Multi Sensor Data Fusion Lab, CSIR – National Aerospace Laboratories, Bangalore,

India

Abstract:- Image fusion is a process of combining two or more

X (k)

2 N 1 x(n) sin (n 1)(k 1) , 0 n N 1

images into a single image without any loss of information. Now-a-days, image fusion is playing a major role in research areas. In this paper, discrete sine transform based multi imaging sensor data fusion algorithms are developed,

n0

N 1

N

0 k N 1

(1)

implemented and tested using fusion quality evaluation metrics. The proposed fusion algorithms are compared with discrete Cosine transform based fusion algorithms and it is observed from the results that they are almost similar and comparable. The proposed fusion algorithms are computationally simple and can be used in real time applications.

The DST-I is orthogonal and it is exactly equivalent to a

DFT of real sequence that is odd around the 0th and middle points, scaled by 0.5. The DST-I is its own inverse.

The two dimensional (2D) discrete sine transform

X (k1 , k2 ) o f an image x(n1 , n2 ) of size N1 N 2 is defined as

2 2 N1 1N2 1

(n 1)(k 1)

Keywords:- Discrete Sine Transform, Image Fusion, Discrete

X (k1 , k2 ) x(n1 , n2 )sin 1 1

Cosine Transform.

N1 N2 n1 0 n2 0

N1

(2)

(n2 1)(k2 1)

  1. INTRODUCTION

    Of-late, many researchers developed different image fusion algorithms to combine multiple images into a single image.

    sin

    N2

    Where, 0 k1 , k2

    N1

    1, N 2 1

    Image fusion using corresponding pixel averaging is the simplest method of all those different fusion algorithms. In image fusion literature, no researchers use the simple and well proven discrete Sine transform (DST) for fusion application. In this paper, an attempt is made to use DST for developing image fusion algorithms. This paper introduces different DST based image fusion techniques, studied their performance. The goal of this paper is to check the performance of image fusion based on DST algorithm using different block sizes and to compare with DCT based image fusion algorithm [1].

    Here, 2D DST is a self inverse [2].

    3. IMAGE FUSION

    The architecture of 2D DST based image fusion is shown in Fig.1.The images to be fused are divided into non- overlapping blocks of size NxN. DST is performed on corresponding blocks of images to be fused and fusion rules are applied on DST coefficients to get the fused DST coefficients. This process is repeated for each block of an image. This gives the fused image as output.

    Fusion rules

    Image 1

  2. DISCRETE SINE TRANSFORM

The discrete sine transform (DST) is a Fourier-related transform similar to the discrete Fourier transform (DFT) but using a purely real matrix. It is the imaginary part of a DFT and its length is approximately twice the length of DFT. It is operating on real data with odd symmetry, whereas the DFT of a real and odd function is imaginary and odd. DST expresses of finitely discrete sequence in terms of sine functions oscillating at different frequencies. There are eight standard DST variants of which four are common and widely used for signal analysis.DST is a linear and invertible function. In literature eight DST

Image 2

DST

DST

DST

Fused Image

equations are there in that DST-1 to DST-4 are widely used. In this paper DST-1 is using because it is simple and it has its own inverse so no need of applying inverse DST.

Fig-1: Framework of 2D DST based image fusion architecture.

The one dimensional (1D) discrete sine transform X (k ) of a signal x(n) of length N is defined as:

The fusion rules which are used in image fusion process 4. FUSION EVALUATION METRICS

are presented in this section. Let consider X1 be the DST coefficients of image block from image 1 and similarly X 2 be the DST coefficients of image block from image 2.

Image fusion process is used to get good quality image and to evaluate the fusion quality, many fusion quality evaluation metrics are proposed in the open literature.

When the ground truth image is available the following

Assume that the image block is of size the fused DST coefficients.

3.1 DSTav

N N

and

X f be

fusion evaluation metrics are used to evaluate the performance of the different image fusion algorithms [4]:

4.1 SPATIAL FREQUENCY (SF)

The frequency in spatial domain indicates the overall

In this fusion rule, all DST coefficients from both image blocks are averaged to get fused DST coefficients. It is a very simple and basic image fusion technique in DST

activity level in the fused image and it is computed as row and column frequency of the images.

domain.

Spatial frequency criterion is: SF

Where, row frequency of the image:

RF 2 CF 2 (7)

X f (k1, k2 ) 0.5*(X1(k1, k2 ) X 2 (k1, k2 ))

(3)

1 M N

f

f

RF [x (n1, n2) x (n1, n2 1)]2

Where, k1 , k2

= 0,1,2,… …,N -1

MN n11 n22

3.2 DSTmax

Column frequency of the image:

By collecting DC components from both frames the

CF

1 N

[x (n1, n2) x (n11, n2)]2

M

f

f

average can be done. From the AC components greatest magnitude AC coefficients are considered and then the detailed coefficients correspond to sharper brightness changes in the images such as edges, object boundaries etc. The DC and AC coefficients are fluctuating around zero

MN n21n12

SF indicates the overall activity level in the fused image. The fused image with high SF will be considered [2, 3].

X f (0,0) 0.5* (X1 (0,0) X 2 (0,0))

1

Where, k1 , k2 = 0,1,2,… …,N -1

(4)

4.2 Peak Signal to Noise Ratio (PSNR)

Its value will be high when the reconstructed and reference images are similar. Higher PSNR value implies better

f 1 2

X1 (k1, k 2 ) X (k ,k )

| X1 (k , k 2 ) | | X2 (k1 , k 2 ) |

reconstruction. The peak signal to noise ratio is computed as:

2 1 2

X (k , k )

3.3 DSTah

| X1 (k1 , k 2 ) | | X2 (k1 , k 2 ) |

PSNR 10log

L2

(8)

By collecting the lowest AC components and including DC

10 1 M N x (n1, n2) x

(n1, n2)2

coefficients averaging process can be performed and the

MN

r f

n21n11

AC coefficients which are remained, are chosen based on largest magnitude.

Where, L is the number of gray levels in the image and xr

X (k , k ) 0.5*(X (k , k ) X (k , k )) (5)

is the reference/ground truth image.

f 1 2

1 1 2

2 1 2

Where, k1 , k2

= 0,1,2,… …,0.5N-1

4.3 Entropy (H)

f 1 2

X1 (k1 , k 2 ) X (k ,k )

X (k , k )

| X1 (k , k 2 ) | | X2 (k1 , k 2 ) |

1

| X (k , k ) | | X (k , k ) |

Entropy is used to measure the information content of an image. Using entropy, the information content of a fused

image is:

2 1 2

1 1 2

2 1 2

L

H

Where, k1 , k2

= 0,1,2,… …,N -1

H hx (i) log 2hx (i)

(9)

3.4 DSTe

In this DC components are averaged together. And the AC

Where,

h

x f

f f

i0

(i) is the normalized histogram of the fused image

coefficients correspond to the frequency band having largest energy considered.

x f and

LH number of frequency bins in the histogram.

X f (0,0) 0.5*(X1(0,0) X 2 (0,0))

X k , k Ej2 Ej2

(6)

Entropy is sensitive to noise and other unwanted rapid

fluctuations. The information entropy measures the

X k , k

1 1 2

richness of information in an image. Hence, entropy is

f 1 2

X 2

k1 , k2

Ej2 Ej2

higher, performance is better [4].

Where, k1 , k2

= 0,1,2,… …,N -1 and

j k1 k2

  1. RESULTS AND DISCUSSIONS

    The fused image will get by applying the DST on fused coefficients as: xf DST(X f )

    Note: DCT based image fusion algorithms are very similar to DST based image fusion algorithms (Section 3.1 to 3.4) by simply replacing DST by DCT.

    The main objective of this paper is to fuse multiple images using 2D DST. The ground truth and source images (images to be fused) are shown in Fig-2 and Fig-3 respectively. Fused images using different DST algorithms are shown in Figs-4 to 7. The computational time of the proposed fusion algorithms are shown in Table-1. It is

    observed that, in DST based fusion process at 16×16 block size it is taking less time for all fusion rules except for DSTe(at 32×32 it is taking less time). whereas, in DCT at 128×128 block size it is taking less time for all fusion rules except for DSTe(at 64×64 it is taking less time). Image fusion quality evaluation metrics for the proposed fusion algorithms are shown in Table-2&3. The metrics shown with bold indicates better results and corresponding algorithms will be the best among others. It is observed that SF is same irrespective of block size. The comparison of DST and DCT based image fusion using block size 8×8 is shown in Table-4.It is observed that, fusion algorithm based on DST and DCT are performing almost similar. In fact, DSTav provides better PSNR than DCTav as shown in Table-4. The fused images with different fusion algorithms are shown in Figs-4 to 7. It is observed that, DST av shows better fusion image compare to other DST based fusion algorithms followed by DSTe. These results are comparable with DCT based fusion algorithms.

  2. CONCLUSION

    Multi imaging sensor data fusion algorithms using 2D-DST are presented and evaluated using fusion quality evaluation

    metrics. The results are compared with DCT based fusion algorithms available in open literature. It is observed that fusion algorithms based DST and DCT are performing almost similar. The proposed fusion algorithms are computationally simple and can be used in real-time image fusion applications.

  3. REFERENCES

  1. VPS Naidu, Discrete Cosine Transform based Image Fusion Techniques, Journal of Communication, Navigation and Signal Processing (January 2012) Vol. 1, No. 1, pp. 35-45.

  2. https://en.wikipedia.org/wiki/Discrete_sine_transform.

  3. K. Rao and P. Yip, Discrete Cosine Transform, Algorithm, Advantages, applications, Academic Press, 1990.

  4. S.A. Martucci, Symmetric Convolution and the Discrete Sine and Cosine transforms, IEEE Trans. Sig. Processing, SP-42, pp.1038-1051, 1994.

Table (1): Computational Time (in sec)

Fusion rules

DST/DCT

Block Size(rows & columns)

2×2

4×4

8×8

16×16

32×32

64×64

128×128

256×256

512×512

Av

DST

7.4232

3.8034

1.9731

1.8318

2.6496

4.7131

9.1133

17.9871

35.9240

DCT

22.6578

5.9152

1.5946

0.4896

0.1859

0.1617

0.1085

0.2009

0.2738

max

DST

7.8969

3.4496

2.0177

1.8393

2.6509

4.7310

9.0845

18.1372

36.0051

DCT

23.1537

6.0213

1.6209

0.4881

0.1886

0.1691

0.1164

0.2028

0.2831

ah

DST

8.7456

3.6002

2.0647

1.8703

2.6434

4.8103

9.0807

17.9239

35.8718

DCT

23.7942

6.1585

1.6665

0.5042

0.1988

0.1665

0.1226

0.2065

0.2847

e

DST

27.0125

14.0739

7.4312

5.4210

5.3831

8.2326

14.9772

29.0308

58.1472

DCT

34.4017

11.9302

4.8228

2.3368

1.4845

1.4178

1.8222

4.2950

9.7422

Table (2): Spatial Frequency

Fusion rules

DST/DCT

Block Size(rows & columns)

2×2

4×4

8×8

16×16

32×32

64×64

128×128

256×256

512×512

Av

DST

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

DCT

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

0.0358

max

DST

0.0450

0.0489

0.0515

00531

0.0535

0.0546

0.0555

0.0548

0.0569

DCT

0.0544

0.0643

0.0667

0.0669

0.0668

0.0668

0.0666

0.0667

0.0611

ah

DST

0.0557

0.0503

0.0497

0.0486

0.0474

0.0487

0.0493

0.0474

0.0508

DCT

0.0544

0.0528

0.0510

0.0501

0.0496

0.0493

0.0491

0.0489

0.0463

e

DST

0.0556

0.0560

0.0580

0.0610

0.0631

0.0677

0.0676

0.0668

0.0644

DCT

0.0544

0.0642

0.0666

0.0668

0.0668

0.0667

0.0667

0.0667

0.0520

Table (3): Entropy of DST/DCT

Fusion rules

DST/DCT

Block Size(rows & columns)

2×2

4×4

8×8

16×16

32×32

64×64

128×128

256×256

512×512

Av

DST

4.0546

4.0232

4.0217

4.0152

4.0181

4.0182

4.0174

4.0182

4.0176

DCT

3.9888

3.9888

3.9891

3.9889

3.9882

3.9937

3.9933

3.9978

3.9943

Max

DST

3.9478

3.9000

3.9165

3.9698

4.0570

4.1263

4.2360

4.5376

4.6268

DCT

4.0179

4.0090

4.0099

4.0006

3.9488

3.9918

4.1587

3.7418

5.5020

Ah

DST

3.9478

3.9480

3.9783

4.0183

4.0775

4.1074

4.2049

4.3844

4.4422

DCT

4.0179

4.0246

4.0253

4.0286

4.0412

4.0701

4.1477

4.1916

4.8325

E

DST

3.9464

3.8561

3.8397

3.8748

3.9539

4.0498

4.0651

4.0914

4.3567

DCT

4.0173

4.0064

4.0057

3.9889

3.9385

3.9669

3.9820

3.7125

4.8561

Table (4): Comparison between DST and DCT

Block Size(8×8)

DSTav

DSTmax

DSTah

DSTe

DCTav

DCTmax

DCTah

DCTe

Computational Time(in sec)

1.9731

2.0177

2.0647

7.4312

1.5946

1.6209

1.6665

4.8228

Spatial

0.0358

0.0515

0.0497

0.0580

0.0358

0.0667

0.0510

0.0666

PSNR

46.7141

43.9522

45.0890

44.4066

38.4255

40.7588

38.6715

40.7857

Entropy

4.0217

3.9165

3.9783

3.8397

3.9891

4.0099

4.0253

4.0057

Fig-2: Ground Truth Image-SARAS

Fig-3: Source Images (images to be fused) -SARAS

Fig-4a: Fused image using DSTe fusion algorithm Fig-4b: Fused image using DCTe fusion algorithm

Fig-5a: Fused image using DSTah fusion algorithm Fig-5b: Fused image using DCTah fusion algorithm

Fig-6a: Fused image using DSTmax fusion algorithm Fig-6b: Fused image using DCTmax fusion algorithm

Fig-7a: Fused image using DSTavfusion algorithm Fig-7b: Fused image using DCTav fusion algorithm

Leave a Reply

Your email address will not be published. Required fields are marked *