Survey on Image Contrast Enrichment using Multi Exposure Image

Survey on Image Contrast Enrichment using Multi Exposure Image

Ms. U. L. Kokate1

1Dept of Computer Science and Engineering,

D.K.T.E. Society’s Textile & Engineering Institute, Ichalkaranji (An Autonomous Institute), India.

Mr. S. B. Bhagate2

2Dept of Computer Science and Engineering,

1. 1. 1. 1. Society’s Textile & Engineering Institute, Ichalkaranji (An Autonomous Institute), India.

            Abstract Capturing natural scenery with rich details and good contrast is the main aim of Digital Photography. Very often the digital camera captures images that do not capture the light correctly, which results in an image that is underexposed or overexposed and ends up looking flat without any discernible features due to low contrast. The HDR paradigm comes to the rescue here, according to this technique images with varying degrees of exposure such as underexposed, overexposed with varying low and high contrast images are combined to achieve a high detail image. This is a revolutionary technology but unfortunately, most of the proposed techniques have some or the other drawback. Therefore, we have envisioned an HDR generation technique achieved through the modification of the CNN algorithm. To achieve this, this paper evaluates a collection of conventional HDR image generation techniques and identifies the shortcomings. The proposed methodology for the generation of HDR images through the use of an improved CNN algorithm will be discussed in future researches.

            KeywordsImage contrast enrichment, HDR, MEF

            1. INTRODUCTION:

               The picture have value of thousand words, which I s true it might take a long text to describe a particular scenario in detail, but a picture is enough to combine it into a much more comprehensive representation. Image processing means the method to perform operations on image and get the enhanced image from the input image. Contrast is nothing but the change in luminance and color that makes an object distinct.In the history of man, it has been observed that pictures seem to be highly valued for their representations, from the cavemen paintings of various scenes to the renaissance painters immaculately realistic paintings and frescos. Humans have given a lot of reverence and importance to images throughout history. In the modern era, there have been a lot of advancements in the imaging industry, once upon a time photographic films were in large scale use and produced images by capturing the light on to a lightsensitive Photographic film, which was then later developed to get the image. We have progressed from that era very recently and diverged towards the digital images, which are images captured with the help of a sensor, which captures the light incident on the sensor and generates an image through pixels. A pixel is the smallest unit of an image that has a particular value and the combination of a huge number of pixels generates the resultant image, due to large advancements in the technology in the recent years. There has been a

               large-scale proliferation of various electronic cameras that are capable of capturing digital images. This has enabled far better access to the masses to capture a memory or a vacation that can be highly valuable as a remembrance of a particular occasion and location as it has already been established that humans have an affinity towards pictures. A camera pointed towards a bright object receives a lot of light, which increases the average maximum brightness of the image, this artifact washes out the images which reduces the details in the image, whereas if the Digital Camera is pointed at a subject with reduced light, the average maximum brightness is lowered significantly and the camera does not get enough light to discern the objects which result in an image which is dark and lacks a lot of detail, to solve this problem, HDR is utilized. The dynamic range of an image is increased by taking the same picture of a subject form the same angle and elevation multiple times. Each time the exposure of the image is adjusted until a collection of images with low, high and medium exposure is achieved. These images represent the varying exposures such as the washed-out elements and the dark elements from a low exposure. These images are then combined together to achieve a much higher Dynamic Range which preserves all the details. The Multi-Exposure Image Fusion method can recover more details of images, which cannot be exposed by the single image contrast enhancer method. The single image contrast enhancer easy to implement in practical, but due to restricted information in a single image it is somehow difficult task. HDR can help reach a reallife quality in images due to the fact that our eyes adjust to various exposures and can process the image in a High Dynamic Range; this is how a human eye visualizes the world. Therefore, the HDR images represent a closer to real-life quality of experience that is highly useful and pleasing to the eyes. There have been many techniques and researches in the field of HDR processing and the creation of HDR-enabled Displays to utilize this revolutionary imaging technique.

            2. LITERATURE REVIEW

               J. Mir presented unique approach towards the generation of accurate and fast High Dynamic Range (HDR) images. The approach elaborated based on the adaptive residual mapping [1]. The authors have also combined High Efficiency Video Coding (HEVC) and optimized the extension layer coding performance according to the HEVC. The resultant technique is capable of

               outperforming the single HLG broadcast pipeline while maintaining backward compatibility to achieve extremely good quality HDR images. The major drawback in this approach is that the system compromises a bit of performance to achieve higher quality HDR images.

               D. Kundu worked to determine a digital pictures quality as described by the observers but done by a computer automatically has seen growing interest in various fields. Usually application where ultimately the classified content is being projected at actual humans is going to interact with or visually perceive the image. Therefore, the authors presented a technique for the automatic quality assessment of HDR (High Dynamic Range) images without the use of any references. The drawback in this technique is that the resultant software is not a specialized system and can also perform on legacy SDR (Standard Dynamic Images) that have 8-bit depth [2].

               Y. Fang described the HDR (High Dynamic Range) images are created to capture and extend the Dynamic ranges of the image as most of the range is not handled well by conventional cameras. The High Dynamic Range images are created by taking multiple shots of the subject in varying exposure levels and then combining them together to achieve the final full detailed image. But most of the time the images get displaced often and combining them can lead to ghosting. Therefore, the researchers have presented a technique for the quality assessment for the HDR de-ghosting technique [3]. The major drawback in this technique is that the authors have not made the database public; therefore, the performance cannot be quantified.

               Y. Liu elaborated on the various technique utilized to achieve better quality and detail from digital images. High Dynamic Range can help emphasize the details in an image by capturing the subject with varying degrees of exposure and then combining the images to achieve a higher detail image. The author worked to utilize the Minimum Detectable Contrast for achieving the HDR image [4]. The presented technique preserves the information related to the contrast to help achieve bettr results. The major drawback of this technique is that it is computationally expensive to deploy this approach.

               M. Azimi performed performance evaluation of various techniques of achieving a High Dynamic Range image, particularly the single-layer approach. The researcher states that there has been an increase in the number of HDR-compatible Television sets that can utilize the high dynamic range for video. Therefore, this study evaluated the SDR10 and HDR10 pipelines for their performance [5]. The extensive experiments concluded that HDR10 is the superior technique of the two. The major drawback in this paper is that it has been limited to the implementation of HDR10 and SDR10 and cannot be reliable as experimental performance analysis. T. Van described that, due to recent advancements in the area of electronic and electronic devices has led to an increase in the

               number of images being taken every day. But as the resolution of the smartphone cameras increase there is a lack of depth or dynamic range in the digital images clicked by a conventional camera. Therefore, the authors have proposed an innovative concept based on the super- pixel illumination invariant motion estimation for the processing of SDR video [6]. The major drawback is that it takes a lot of time to generate an output and increases the time complexity.

               J. Mir presented an innovative technique for achieving a High Dynamic Range in the video to provide a high- fidelity video to the consumers for a future where everybody owns an HDR-enabled Television for more effective viewing experience. The researchers achieved this with the help of HEVC and lambda domain, and the Rate Control has an important task in the transmission and distribution of the video [7]. The proposed technique was extensively tested for its performance and achieved improvements. The major drawback in this technique is that subjective tests were not performed.

               Y. Mai described that, due to recent advancements in electronics and the advent of smartphones, there has been increased consumption of multimedia. Most of it is in the form of images and video. As most of the digital cameras are not capable of achieving real like depth in the photos, HDR is used to achieve higher detail in the images. Therefore, the researcher used a patch-based approach for achieving High Dynamic Range in videos featuring Fast- moving objects [8]. The major drawback in this approach is that the Quality of the resultant image is not very good.

               H. Najaf-Zadeh presented a novel approach for the amalgamation of HDR or High Dynamic Range and Virtual Reality. Most of the time in Virtual Reality, the video quality is very less and bright regions appear washed out and the dark regions lose a lot of detail in them, this leads to a very poor experience. Due to this reason the author designed a framework that utilized the HDR playback for projection on the Virtual Reality headset [9]. The technique has been demonstrated to increase the quality of the video drastically. The major drawback in this technique is the latency observed in displaying complex images.

               Perrin described that there has been a change of preference of the consumer, as there has been a large proliferation of High Dynamic Range televisions and the increasing amount of them have largely been affordable. This is shifting the preference as the consumer is much more interested in HDR content [10]. Therefore, the authors have assessed the Quality of a Dual Layer Backward compatible codec for the accurate HDR and SDR solution for the consumer. The comparison results show that there has to be a bandwidth allocation or delivering the HDR content accurately. The major drawback is that the results show that the HDR after a compression strategy had only acceptable quality for consumption.

            3. RELATED WORK:
               1. 1. System Architecture:

                     Fig.1. System architecture for contrast image enrichment using multi exposure images.

                     Conventional HDR and MEF algorithms are utilized to create a reference image. These images will be utilized further for the comparison purposes.CNN based SICE enhancer is trained using a mapping characteristics between the low contrast and reference image. The consequent photo is decomposed into excessive and low frequency additives by the application of weighted least squares to each channel. The luminance range is enhanced and balanced by the CNN. The first stage combines the luminance and detail enhancer trained in parallel.Both the enhanced components are merged as input for another CNN to enhance the image with the aid of comparing it to the reference photo received in step 1.

                     Example:

                     Fig. 2.1 Input Image Fig. 2.2 Output Image

                     1. Reference Image Creation:

                        MEF and stack based HDR methods are used to regenerate image with high-contrast and good quality than a single exposure image. By using MEF and HDR method reference image of different scenes is generated and get the required output for each sequence. This is the initial step of the proposed model where a reference image is

                        created from the dataset for the given input image. While creating the reference image a dark channel and a bright channel images are considered. And then these images fused to get a perfect model reference image using multi exposure image fusion technique (MEF), where each of the images is considered to build the absolute brightness based on the MEF algorithm. Once the Brightness Enhanced image is obtained then it is subjected to increase its overall channel rate using the High dynamic range.

                     2. CNN Based SICE enhancer:

                        With the help of constructed dataset, we will layout a CNN based SICE enhancer to learn mapping characteristics among the low contrast input photograph and its corresponding reference image. Here in this step Contrast will decided based on the edges in the images, because the edges denotes the number of the objects and complexity in the images. Here for each of the input images by the user is subjected to Edge Detection process, which works based on the following steps.

                        Smoothing – Here image is blurred using the Gaussian blur technique to widen the edges of the objects in the image; for this Gaussian distribution technique is used.

                        Finding the gradients- Here the edges with the maximum width is identified.

                        Non- maximum Suppression – Edges with the maximum width is labeled.

                        Double Thresholding – Here the edges, which are already labeled, are doubled by their width.

                        Edge Tracking- Here all the threshold edges are traversed recursively to mark them in white color and rest of the pixels is marked with the black color.

                     3. Decomposition:

                        Here in this step the number of images from the dataset is linearly decomposed into clusters based on the features. These clusters are then loaded into the multi-threads so that the process of contrast enhancement can be achieved concurrently to reduce the time complexity. The resultant image is decomposed into high and low frequency components by the application of weighted least squares to each channel.

                     4. Luminance Management:

                        The luminance range is enhanced and balanced by the CNN. The first stage combines the luminance and detail enhancer trained in parallel. Here in this step the luminanace of the input image is managed using the following eqauation

                        Luminance= 0.299 * R + 0.587 * G + 0.114 * B

                        (1)

                     5. Optimization:

                        Here in this step of contrast enhancement HDR image is

                        considered as the model image to enhance the contrast of the luminance Managed image by summation of the difference of the Color channel ranges. Both the enhanced components are merged as input for another CNN to enhance the image by comparing it to the reference image obtained in step 3.3.1.

            4. SUMMARY:

               This paper evaluates the various traditional techniques used for the purpose of improving the contrast of the image. The extensive evaluation of the various CNN based High ynamic Range (HDR) image generation techniques has resulted in the observation that most of the techniques are inadequate for application. Most of the prevalent techniques for HDR image creation have been demonstrated to be highly inconsistent with varying accuracy. Therefore, to provide a solution to this problem as well as improve the performance of the HDR generation technique, an improved CNN algorithm is proposed that maintains the accuracy and includes support for high definition images. The proposed methodology will be elaborated in detail in the upcoming editions.

            5. REFERENCES:

1. 1. 1. 1. J. Mir et al, Improved Two-Layer Backward-Compatible HDR VideoCoding: A Performance Comparison with Single-Layer HLG, IEEE International Conference on Consumer Electronics (ICCE), 2017.
         2. D. Kundu et al, No-Reference Quality Assessment of Tone- MappedHDR Pictures, IEEETransactions on Image Processing, 2017.
         3. Y. Fand, H. Zhu, K. Ma, and Z. Wang, Perceptual Quality Assessment of HDR Deghosting AAlgorithms, IEEE International Conference on Image Processing (ICIP), 2017.
         4. Y. Liu et al, An Adaptive Perceptual Quantization Method for HDR Video Coding, IEEE International Conference on Image Processing (ICIP), 2017.
         5. M. Azimi, R. Boitard and M. Pourazad, Performance Evaluation of Single Layer HDRVideo Transmission Pipelines, IEEE Transactions on Consumer Electronics, 2017.
         6. T. Vo and C. Lee, Robust HDR Video Synthesis Using Superpixel-Based IlluminationInvariant Motion Estimation, IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), 2018.
         7. J. Mir, S. Talagala and A. Fernando, Optimization of HEVC – domain Rate ControlAlgorithm for HDR Video, IEEE International Conference on Consumer Electronics (ICCE), 2018.
         8. Y. Mai and C. Chiu, Patch-Based HDR Video Processing for FastMoving Object Reconstruction, International Conference on Computing, Networking and Communications (ICNC): Multimedia Computing andCommunications, 2017.
         9. H. Najaf-Zadeh et al, VR+HDR: A System for View- Dependant Rendering of HDR video in Virtual Reality, IEEE International Conference on Image Processing (ICIP), 2017.
         10. Perrin et al, Quality Assessment of an HDR Dual- LayerBackward-Compatible Codec Compared toUncompromised SDR and HDR Solutions, IEEE Transactions on Broadcasting, 2018.