Convolutional Neural Network based Classification of Brain Tumors images: An Approach

Convolutional Neural Network based Classification of Brain Tumors images: An Approach

Jayashree Shedbalkar

Department of CSE, KLS VDIT, Haliyal

Dr. K. Prabhushetty

Dept. of ECE, SGBIT, Belagavi, India

Abstract Brain tissue biopsy is performed by surgery, which is rarely performed before explicit brain surgery. Improving technology and machine learning can help radiologists in tumor screening without invasive measures. A machine learning algorithm that has achieved great results in image classification and fragmentation is the convolutional neural network (CNN). We are introducing a new CNN model for brain cell division into healthy or malignant. The advanced network is simpler than the already trained networks that already exist, and has been tested on T1 images with comparatively weighted magnification. The best result of the 10- point cross-verification method was obtained by intelligent recording confirmation of additional data sets, and, in that case, the accuracy was 96.50%. With a good knowledge of overall performance and good performance speed, the new CNN architecture could be used as an effective tool to support decisions for radiologists in clinical trials.

KeywordsBrain tumour; image classification; CNN; deep learning

1. INTRODUCTION

   Diagnosing Brain tissues share certain features and challenges in treating tumors in the body, but they also present specific problems related to the unique features of the organ in which they live. It is important to weigh the tissues according to their severity. Although other types of tumors, for example, meningiomas and neurinomas are less common, gliomas as the most common brain tissue present in all the stages. Surgical biopsies, especially when taken with stereo carefully, may miss the worst part of the tumor and therefore reduce the tumor detection. Separation is a necessary and important step in image analysis; it is the process of separating an image into different regions or preventing the sharing of common and similar properties, such as color, texture, contrast, brightness, borders, and gray level. Brain tumors detection involve the process of separating tumor tissues such as edema and dead cells from normal brain tissue using WM, GM, and CSF with the help of MR images or other imaging techniques [2 – 5]. In fact, experts have offered a variety of methods to automatically detect brain tissue and catalog typing using MRI brain imaging from the time of scanning and transporting medical images to a computer. On the other hand, Neural Networks (NN) and Support Vector Machine (SVM) are the most commonly used methods used over the past few years.

   A major problem in classifying MRI images by specific neural networks lies in the number of images in the database. In addition, MRI images are available on a variety of plane,

   so the option of using all the planes will expand the size of the database. Pre-processing is required before feeding the images to the neural network, one of the well-known advantage of use of convolutional neural networks (CNN) is that manual pre-processing and feature extraction is not required.

   The purpose of this study was to classify a given image into a healthy or malignant through CNN. In this paper, we present a new CNN model for brain image processing to healthy or malignant.

2. LITERATURE SURVEY

   In order to know the state-of-the-art methods in the related study, following are the gist of papers.

   (Rehman A et.al, 2020) have worked proposed a method to detect brain tumors and their classification. The selected features are verified by the feed-forward neural network for final isolation. The three BraTS data sets for 2015, 2017, and 2018 are used for research, validation, and achieve accuracy of 98.32, 96.97, and 92.67%, respectively.

   (Thillaikkarasi R et.al, 2019) have worked on in-depth learning algorithm (kernel-based CNN) with M-SVM introduction to identify tumor automatically and efficiently. Image segmentation is done using M-SVM according with selected features. From an MRI scan, the tumor is isolated with the help of a kernel-based CNN-based method. The experimental results of the proposed method indicate that the method used may enable the brain tumor separation to reach approximately 84% accuracy in comparison with existing algorithms.

   (Zhang M etal, 2020) have developed a methodology for detecting brain metastasis through MRI images. A total of 361 patients from 121 patients are used to train and test with fast region-based network with a convolutional neural network (Faster R-CNN). 270 scans of 73 patients were used for training; 488 planes are used in 91 scans of 48 test patients. Examination of data captured in MRI brain showed 96% sensitivity and 20 false metastases by scans. The results showed 87.1% sensitivity and 0.24 false metastases per unit.

   (Todoroki Y et.al, 2019) have proposed a system for the detection of liver transplants with different sizes in multiphase CT images using CNN's multi-channel protocol essential for transplantation. Test results reveal that memory points and dice coefficient can be significantly improved, by reducing the number of false positives and the proposed method are compared with existing methods.

   (Latif G et.al, 2019) have proposed a method using 65-cases with 40,300 MR images obtained from the BRATS-2015 database. The work includes 26 Low-Grade Glioma (LGG) cases and 39 High-Grade Glioma (HGG) tumors. The proposed CNN model exceeds existing methods by obtaining an average accuracy of 98.77%.

   (Vivanti R et.al, 2018) proposed a convolutional neural network (CNN) method for automatic liver detection. Test results on 222 tissues from 31 patients revealed a common fracture error of 17% (std = 11.2) for a distance of more than

   1. mm (std = 1.8), much better than the independent segment alone. Importantly, the durability of the approach is improved from 67% of CNN's global division to 100%.

      (Wang H et.al, 2019) have worked on Focal cortical dysplasia (FCD) related to cortical developmental disorders. FCD automation detection strategies are proposed using MR images. All cortical patches are extracted from each axial plane, and these patches are divided into FCD and non-FCD using a deep convolutional neural network (CNN) with five layers of precision. Program quality testing, nine out of ten FCD images were successfully detected, and 85% of non-FCD images were correctly identified.

      (Kutlu H et.al, 2019) have proposed a method to differentiate liver and brain tumors using convolutional neural network (CNN). Test results show that the proposed method had higher performance than the other classifiers, such as the proximity K (KNN) and the vector support machine (SVM). Results show that a 99.1% accuracy rate is obtained with a liver tumor classification and accuracy rate of 98.6% is achieved in the brain tumor phase.

      Form the literature survey; we can conclude that, ample amount of work is carried in classification of brain tumor images. But, only few researchers have reported their work using CNN. Hence, the proposed work is taken.

3. METHODOLOGY

   The proposed methodology consists of the following blocks as shown in Fig. 1. The work carried is discussed in detail through the following sections.

   Gaussian blur		Morphological operation		Contour detection		Data Augmentation
   	CNN model

   Healthy

   Malignant

   Healthy

   Malignant

   Fig. 1. Block Schematic representation of the proposed study

   1. Image Database

      The image database, images are collected from Kaggle database. A total of 340 images are collected and are increased to 1700 through data augmentation technique. Sample images considered for the work is shown in Fig. 2. (a) through (j).

      a	b	c	d	e
      f	g	h	i	j

      Fig. 2. (a)- (j): Sample images used for the work showing different types of tumors in different planes.

   2. Morphological operation

      Magnetic resonance images from the database were of different sizes and were provided in int16 format. These images represent the input layer of the network, so they were normalized and resized to 256 × 256 pixels.

      In order to augment the dataset, we transformed each image in two ways. The first transformation was image rotation by 90 degrees. The second transformation was flipping images vertically. In this way, we augmented our dataset three times, resulting in 1700 images.

      1. Gaussian Blur:

         The Gaussian smoothing operator is a 2-D convolution operator that is used to `blur' images and remove detail and noise. In this sense it is similar to the mean filter, but it uses a different kernel that represents the shape of a Gaussian (`bell- shaped') hump. This kernel has some special properties which are detailed below.

         The Gaussian distribution in 1-D has the form as given in Equation-1:

         (1)

         Where is the standard deviation of the distribution. We have also assumed that the distribution has a mean of zero (i.e. it is centered on the line x=0). The distribution is illustrated in Figure 3.

         Fig. 3. 1-D Gaussian distribution with mean 0 and =1

         In 2-D, an isotropic (i.e. circularly symmetric) Gaussian has the form as given in Equation-2:

         (2)

         This distribution is shown in Figure 4.

         Fig. 4. 2-D Gaussian distribution with mean (0,0) and =1

      2. Erosion and dilation

      Greyscale image	Binary image by thresholding	Erosion: a 2×2 square structuring element

      Greyscale image	Binary image by thresholding	Erosion: a 2×2 square structuring element

      Binary image	Dilation: a 2×2 square structuring element

      Binary image	Dilation: a 2×2 square structuring element

      The erosion of a binary image f by a structuring element s (denoted f s) produces a new binary image g = f s with ones in all locations (x,y) of a structuring element's origin at which that structuring element s fits the input image f, i.e. g(x,y) = 1 is s fits f and 0 otherwise, repeating for all pixel coordinates (x,y).

      Erosion removes small-scale details from a binary image but simultaneously reduces the size of regions of interest, too. By subtracting the eroded image from the original image, boundaries of each region can be found: b = f (f s) where f is an image of the regions, s is a 3×3 structuring element, and b is an image of the region boundaries.

      The dilation of an image f by a structuring element s (denoted f s) produces a new binary image g = f s with ones in all locations (x,y) of a structuring element's orogin at which that structuring element s hits the the input image f, i.e. g(x,y) = 1 if s hits f and 0 otherwise, repeating for all pixel coordinates (x,y). Dilation has the opposite effect to erosion — it adds a layer of pixels to both the inner and outer boundaries of regions.

      Fig. 5. Morphological operation demonstrating thresholding

      Erosion with small (e.g. 2×2 – 5×5) square structuring elements shrinks an image by stripping away a layer of pixels from both the inner and outer boundaries of regions. The holes and gaps between different regions become larger, and small details are eliminated:

      Erosion: a 3×3 square structuring element

      Fig. 6. Morphological operation demonstrating erosion

      Larger structuring elements have a more pronounced effect, the result of erosion with a large structuring element being similar to the result obtained by iterated erosion using a smaller structuring element of the same shape. If s1 and s2 are a pair of structuring elements identical in shape, with s2 twice the size of s1, then,

      f s2 (f s1) s1 (3)

      Fig. 7. Morphological operation demonstrating dilation

      The holes enclosed by a single region and gaps between different regions become smaller, and small intrusions into boundaries of a region are filled in:

      Dilation: a 3×3 square structuring element

      Fig. 8. Morphological operation demonstrating dilation considering 3×3 square element.

      Results of dilation or erosion are influenced both by the size and shape of a structuring element. Dilation and erosion are dual operations in that they have opposite effects. Let f c denote the complement of an image f, i.e., the image produced by replacing 1 with 0 and vice versa. Formally, the duality is written as

      f s = f c srot (4)

      where srot is the structuring element s rotated by 1800. If a structuring element is symmetrical with respect to rotation, then srot does not differ from s. If a binary image is considered to be a collection of connected regions of pixels set to 1 on a background of pixels set to 0, then erosion is the fitting of a structuring element to these regions and dilation is

      the fitting of a structuring element (rotated if necessary) into the background, followed by inversion of the result.

   3. Contour detection

      Active contour is a type of segmentation technique which can be defined as use of energy forces and constraints for segregation of the pixels of interest from the image for further processing and analysis. Active contour described as active model for the process of segmentation. Contours are boundaries designed for the area of interest required in an image. The main application of active contours in image processing is to define smooth shape in the image and forms closed contour for the region. Active contour models involve snake model, gradient vector flow snake model, balloon model and geometric or geodesic contours.

      Curvature of the models is determined with various contour algorithms using external and internal forces applied. Energy functional is always associated with the curve defined in the image. External energy is defined as the combination of forces due to the image which is specifically used to control the positioning of the contour onto the image and internal energy, to control the deformable changes. For the set of points in an image, the contour can be defined based on forces and constraints in the regions of the image. Different types of active contour models are used in various medical applications especially for the separation of required regions from the various medical images. For example, a slice of brain CT image is considered for segmentation using active contour models. The contour of the image defines the layers of the region in the brain which is shown in the Fig. 9.

      Fig. 9. Segmentation of brain image using active contours.

      Active contours can also be used for segmentation of 3-D images derived from different medical imaging modalities. 2- D slices of image data are used for the separation of target object from the 3-D imges. These 2-D slices of images in all directions along with the segmented target region are subjected to 3-D reconstruction to segregate the regions. Mesh model of the 3-D image is designed before applying active contour model. The mesh helps in the formation of deformable contours of the target object in the directional 2- D slices of the 3-D images [14].

   4. Image augmentation:

      It creates training images through different ways of processing or combination of multiple processing, such as random rotation, shifts, shear and flips, etc. Image Data Generator. An augmented image generator can be easily created using ImageDataGenerator API in Keras. ImageDataGenerator generates batches of image data with real-time data augmentation. The most basic codes to create and configure ImageDataGenerator and train deep neural network with augmented images are as follows.

      datagen = ImageDataGenerator() datagen.fit(train)

      X_batch, y_batch = datagen.flow(X_train, y_train, batch_size=batch_size) model.fit_generator(datagen, samples_per_epoch=len(train), epochs=epochs)

      We can experiment with the following code to create augmented images with the desired properties. In our case, the following data generator generates a batch of 9 augmented images with rotation by 30 degrees and horizontal shift by 0.5. datagen = ImageDataGenerator(rotation_range=30, horizontal_flip=0.5)

      datagen.fit(img)i=0

      for img_batch in datagen.flow(img, batch_size=9):

      for img in img_batch: plt.subplot(330 + 1 + i) plt.imshow(img)

      i=i+1

      if i >= batch_size: break

      Fig. 10. Images after augmentation.

      Details of the augmentation technique applied the proposed study is given below:

      Augmentation_datagen = ImageDataGenerator(

      rotation_range=15, width_shift_range=0.05, height_shift_range=0.05, shear_range=0.05, zoom_range=0.1, horizontal_flip=True, vertical_flip=False, fill_mode='nearest')

   5. CNN Model

      The network architecture consists of input, two main blocks, classification block, and output, as shown in Figure2. The first main block, Block A, consists of a convolutional layer which as an output gives an image two times smaller than the provided input. The convolutional layer is followed by the rectified linear unit (ReLU)

      activation layer and the dropout layer. In this block, there is also the max pooling layer which gives an output two times

      max_pooling2d_2 (MaxPooling2 (None, 26, 26, 64) 0

      smaller than the input. The second block, Block B, is

      different from the first only in the convolution layer, which retains the same output size as the input size of that layer.

      flatten (Flatten) (None, 43264) 0

      The classification block consists of two fully connected

      (FC) layers, of which the first one represents the flattened output of the last max pooling layer, whereas, in the second

      dense (Dense) (None, 64) 2768960

      FC layer, the number of hidden units is equal to the number

      of the classes of tumor. The whole network architecture consists of the input layer, classification block, and output

      activation_3 (Activation) (None, 64) 0

      layer; altogether, there are 22 layers.

      dropout (Dropout) (None, 64)

4. RESULTS AND DISCUSSION

   Table.1. shows different parameter scores before and after augmentation of the images.

   0

   dense_1 (Dense) (None, 1) 65

   Table.1: Performance parameters

   Parameters	before augment	after augment
   Accuracy	0.781250	0.925000
   Precision	0.878049	0.958049
   Recall	0.800000	0.900000
   F1 score	0.837209	0.937209
   Cohens kappa	0.506064	0.706064
   ROC AUC	0.865497	0.965497

   Parameters	before augment	after augment
   Accuracy	0.781250	0.925000
   Precision	0.878049	0.958049
   Recall	0.800000	0.900000
   F1 score	0.837209	0.937209
   Cohens kappa	0.506064	0.706064
   ROC AUC	0.865497	0.965497

   activation_4 (Activation) (None, 1) 0

   =========================================

   Total params: 2,797,089

   Trainable params: 2,797,089

   Non-trainable params: 0

   • Model summary is given as follows:

   Model Summary

   Model: "sequential"

   Layer (type) Output Shape Param #

   =========================================

   conv2d (Conv2D) (None, 222, 222, 32) 320

   activation (Activation) (None, 222, 222, 32) 0

   max_pooling2d (MaxPooling2D) (None, 111, 111, 32) 0

   conv2d_1 (Conv2D) (None, 109, 109, 32) 9248

   activation_1 (Activation) (None, 109, 109, 32) 0

   max_pooling2d_1 (MaxPooling2 (None, 54, 54, 32) 0

   conv2d_2 (Conv2D) (None, 52, 52, 64) 18496

   activation_2 (Activation) (None, 52, 52, 64) 0

   Fig. 11. shows the plot for accuracy metrics taken with accuracy versus epochs. Fig. 12 shows the plot for training and validation accuracy metrics taken with accuracy versus epochs

   Fig. 11. Accuracy versus number epochs

   Fig. 12. Performance of the model

5. CONCLUSIONS

We conclude that the proposed CNN architecture classifies given brain images into healthy or malignant. The developed network is simpler than already-existing pre-trained networks, and it was tested on T1-weighted contrast-enhanced magnetic resonance images. The best result for the 10-fold cross- validation method was obtained for the record-wise cross- validation for the augmented data set, and, in that case, the accuracy was 96.50%.

REFERENCES

1. Rehman A, Khan MA, Saba T, Mehmood Z, Tariq U, Ayesha N. Microscopic brain tumor detection and classification using 3D CNN and feature selection architecture. Microsc Res Tech. 2020 Sep 21. doi: 10.1002/jemt.23597. Epub ahead of print. PMID: 32959422.

2. Çinar A, Yildirim M. Detection of tumors on brain MRI images using the hybrid convolutional neural network architecture. Med Hypotheses. 2020 Jun;139: 109684. doi: 10.1016/j.mehy.2020.109684. Epub 2020 Mar 24. PMID: 32240877.

3. Thillaikkarasi R, Saravanan S. An Enhancement of Deep Learning Algorithm for Brain Tumor Segmentation Using Kernel Based CNN with M-SVM. J Med Syst. 2019 Feb 27;43(4):84. doi: 10.1007/s10916- 019-1223-7. PMID: 30810822.

4. Zhang M, Young GS, Chen H, Li J, Qin L, McFaline-Figueroa JR, Reardon DA, Cao X, Wu X, Xu X. Deep-Learning Detection of Cancer Metastases to the Brain on MRI. J Magn Reson Imaging. 2020 Oct;52(4):1227-1236. doi: 10.1002/jmri.27129. Epub 2020 Mar 13. PMID: 32167652; PMCID: PMC7487020.

5. Latif G, Iskandar DNFA, Alghazo J, Butt MM. Brain MR Image Classification for Glioma Tumor detection using Deep Convolutional Neural Network Features. Curr Med Imaging. 2020 Mar 11. doi: 10.2174/1573405616666200311122429. Epub ahead of print. PMID: 32160848.

6. Todoroki Y, Iwamoto Y, Lin L, Hu H, Chen YW. Automatic Detection of Focal Liver Lesions in Multi-phase CT Images Using A Multi- channel & Multi-scale CNN. Annu Int Conf IEEE Eng Med Biol Soc. 2019 Jul;2019:872-875. doi: 10.1109/EMBC.2019.8857292. PMID: 31946033.

7. Vivanti R, Joskowicz L, Lev-Cohain N, Ephrat A, Sosna J. Patient- specific and global convolutional neural networks for robust automatic liver tumor delineation in follow-up CT studies. Med Biol Eng Comput. 2018 Sep;56(9):1699-1713. doi: 10.1007/s11517-018-1803-6. Epub 2018 Mar 10. PMID: 29524116.

8. Wang H, Ahmed SN, Mandal M. Automated detection of focal cortical dysplasia using a deep convolutional neural network. Comput Med Imaging Graph. 2020 Jan;79:101662. doi: 10.1016/j.compmedimag.2019.101662. Epub 2019 Nov 13. PMID: 31812131.

9. Kutlu H, Avc E. A Novel Method for Classifying Liver and Brain Tumors Using Convolutional Neural Networks, Discrete Wavelet Transform and Long Short-Term Memory Networks. Sensors (Basel). 2019 Apr 28;19(9):1992. doi: 10.3390/s19091992. PMID: 31035406; PMCID: PMC6540219.

10. Ker J, Bai Y, Lee HY, Rao J, Wang L. Automated brain histology classification using machine learning. J Clin Neurosci. 2019 Aug;66:239-245. doi: 10.1016/j.jocn.2019.05.019. Epub 2019 May 31. PMID: 31155342.

11. Hussain Z, Gimenez F, Yi D, Rubin D. Differential Data Augmentation Techniques for Medical Imaging Classification Tasks. AMIA Annu Symp Proc. 2018 Apr 16;2017:979-984. PMID: 29854165; PMCID: PMC5977656.

12. Kammerlander C, Neuerburg C, Verlaan JJ, Schmoelz W, Miclau T, Larsson S. The use of augmentation techniques in osteoporotic fracture fixation. Injury. 2016 Jun;47 Suppl 2:S36-43. doi: 10.1016/S0020- 1383(16)47007-5. PMID: 27338226.

13. Zeng S, Zhang B, Gou J, Xu Y. Regularization on Augmented Data to Diversify Sparse Representation for Robust Image Classification. IEEE Trans Cybern. 2020 Oct 21;PP. doi: 10.1109/TCYB.2020.3025757. Epub ahead of print. PMID: 33085628.

14. Nalepa J, Marcinkiewicz M, Kawulok M. Data Augmentation for Brain-Tumor Segmentation: A Review. Front Comput Neurosci. 2019 Dec 11;13:83. doi: 10.3389/fncom.2019.00083. PMID: 31920608; PMCID: PMC6917660.

15. Porter ND, Verdery AM, Gaddis SM. Enhancing big data in the social sciences with crowdsourcing: Data augmentation practices, techniques, and opportunities. PLoS One. 2020 Jun 10;15(6):e0233154. doi: 10.1371/journal.pone.0233154. PMID: 32520948; PMCID: PMC7286483.

AUTHORS

JAYASHREE SHEDBALKAR has received her Bachelors degree in Computer Science and Engineering from Visvesvaraya technological University, Belagavi, Karnataka Masters in Computer Network and Engineering from from VTU Belgaum. Currently she is an Assistant Professor CSE dept. at KLS VDIT Haliyal, Her research interests include Image Processing and Artificial Intelligence.

JAYASHREE SHEDBALKAR has received her Bachelors degree in Computer Science and Engineering from Visvesvaraya technological University, Belagavi, Karnataka Masters in Computer Network and Engineering from from VTU Belgaum. Currently she is an Assistant Professor CSE dept. at KLS VDIT Haliyal, Her research interests include Image Processing and Artificial Intelligence.

Dr. K. Prabhushetty received his PhD in Electronics from Shivaji University Kolhapur, India. He is having more than three decades of experience in the field of engineering and technology. His research area is Medical Image Processing. He is guiding 7 PhD scholars. Currently he is working as professor in department of Electronics & Communication Engineering affiliated to VTU Belgaum institute.

Dr. K. Prabhushetty received his PhD in Electronics from Shivaji University Kolhapur, India. He is having more than three decades of experience in the field of engineering and technology. His research area is Medical Image Processing. He is guiding 7 PhD scholars. Currently he is working as professor in department of Electronics & Communication Engineering affiliated to VTU Belgaum institute.