Study of Filming Condition for Deep Learning Based Crack Detection Method

Study of Filming Condition for Deep Learning Based Crack Detection Method

Tomotaka Fukuoka, Makoto Fujiu, Junichi Takayama

Geosciences and Civil Engineering Kanazawa University, Kakuma-machi Kanazawa city, Ishikawa-ken, Japan

Takahiro Minami

Dept. of Environmental Design Kanazawa University

Kakuma-machi, Kanazawa city, Ishikawa-ken, Japan

Abstract Recently, the study of extending the service life of bridges has gained attention. In Japan, there are about 730,000 bridges with a length of 2 m or longer, and many of these were built during a period of high economic growth, and have now reached the end of their service life. Therefore, their rebuilding and the extension of their service life must be considered.

However, among local public organizations, there are some that have insufficient manpower relative to the number of bridges to manage, as well as insufficient funding for maintenance. Thus, these organizations are unable to perform routine close visual inspections. Specific problems included notably less staff and consulting technicians relative to number of bridges to be managed and high inspection cost preventing from funding for repair.

As issues with the continuing close visual inspection of bridges are surfacing, the remote imaging system is expected to become a new inspection method that replaces close visual inspection.

The practical potential of bridge inspections using images captured with a super-high-resolution camera was examined. A super-high-resolution camera enable us to take a wide area picture of a target bridge from a long distance. An image processing method could improve the efficiency of image based inspection method. For example, a deep learning base image processing method could extract a damaged area on bridge surface automatically with high accuracy faster than human inspection. In generally, the accuracy of image processing method is effected by quality of input image. Filming conditions is one of the factors that determine the quality of photo image. It is important to evaluate an effectiveness of filming condition to improve a reliability of image processing method. In this paper, we evaluate an effectiveness of the filming conditions for image processing method by comparing results of deep learning based crack detection method.

Keywords Bridge inspection; crack detection; image processing; filming conditions; deep learning

1. INTRODUCTION

   Recently, the study of extending the service life of bridges has gained attention. In Japan, there are about 730,000 bridges with a length of 2 m or longer, and many of these were built during a period of high economic growth, and have now reached the end of their service life. Therefore, their rebuilding and the extension of their service life must be considered. An owner of bridge is required to monitoring a bridge with close visual inspection per 5 years according to the national criteria at 2014 in Japan.

   However, some bridge owned by local government have not complete the inspection due to a lack of engineer of bridge inspection or budget. Such bridge is not expected to managed with an aggressive preventive maintenance in future.

   One of the reason of this problem is expensive cost of close visual inspection for bridge. Some bridges are hard to access for engineer does closing and inspection. Such bridge needs scaffolding or an expensive special car to perform close visual inspection. It increases the cost of inspection. In some case, bridge inspection needs a traffic control and it make economic loss. So more reduce economical cost and simplified process of inspection method is required for future bridge maintenance inspection and already many novel inspection methods are proposed [1], [2], [3]. We focus on the remote imaging system which is expected to become a new inspection method that replaces close visual inspection [4]. In this system, an engineer inspects bridge by photo image of target bridge. So engineer no needs to closing target bridge to inspection. This system can solve many problem of current close visual inspection.

   The practical potential of bridge inspections using images captured with a super-high-resolution camera was examined [4]. A super-high-resolution camera enable us to take a wide area picture of a target bridge from a long distance. An image processing method could improve the efficiency of image based inspection method. For example, a deep learning base image processing method could extract a damaged area on bridge surface automatically with high accuracy and faster than human inspection. In generally, an accuracy of image processing method is effected by quality of input image. Filming conditions is one of the factors that determine the quality of photo image. Setting appropriate filming conditions is one way to make quality of photo to high. But the weather changes very often and all bridges are not located on plane field. It is difficult to control filming condition in real bridge inspection. So, it is important to evaluate an effectiveness of filming conditions to improve a reliability of image processing method. Unfortunately, there is no discussion about an effectiveness of filming conditions for accuracy of image processing method using real crack on concrete building.

   In this paper, our purpose is not discovering appropriate filming conditions but measuring degree of effectiveness of filming conditions to image processing method. We focus on two factor of filming conditions: the distance of camera and the lighting. We evaluate these factor using photo image of real crack on surface of concrete building. We compare the

   results of deep learning based crack detection method to evaluate the effectiveness.

2. RELATED WORKS

   There are many kind of method for detecting crack on a concrete surface by image processing. Fujita et al. [5] propose crack detection method with considering effectiveness of noise such as irregular shading and blemishes. They focus on robustness of crack detection. They did not evaluate effectiveness noise. A supervised machine learning method for crack detection is mainstream in recently. Bu et al. [6] proposed Support Vector Machine based crack detection method. They applied three feature extract method from input image; Zenki moment, carbon filter and wavelet transformation. Convolutional Neural Networks (CNN) is one of the popular deep learning method for image processing. Many researchers use it for crack detection task. Cha et al. [7] prepared 40000 images for training model of crack detection. This method can surround crack on concrete surface image with bounding box and has around 98% precision for detection.

   Fig. 1. Example of crack detection by DeepCrack

3. EVALUATAION

   Before you begin to format your paper, first write and save the content as a separate text file. Keep your text and graphic files separate until after the text has been formatted and styled. Do not use hard tabs, and limit use of hard returns to only one return at the end of a paragraph. Do not add any kind of pagination anywhere in the paper. Do not number text heads- the template will do that for you.

   Finally, complete content and organizational editing before formatting. Please take note of the following items when proofreading spelling and grammar:

   1. Image processing based crack detection method

      We adapt semantic segmentation method as an image processing based crack detection method. We use DeepCrack

      [8] model for crack detection. This model can output crack area with pixel unit (Fig. 1). This model trained with a paired input mage: raw image and annotation image (Fig. 2). We used 13,700 concrete bridge surface images as a training data. All image has same size 256 × 256 pixel. 137 images are extracted from the photo image of real concrete bridge surface. Such photo images had taken by a super-high- resolution camera which has a resolution of about 100 million pixels count. The others images are created by augmentation with deep learning method [9]. We have trained 137 augmentation models with the above 137 read concrete surface images. We created augmentation images by 137 annotation patterns (Fig. 3).

      Fig. 2. Example of training data set

      Fig. 3. Example of image augmentation

   2. Test dataset

      We set four different crack on concrete building as a target crack of test data. We used a super-high-resolution camera which has a resolution of about 100 million pixels count to take picture. We set six filming points to evaluate effectiveness of filming distance (Fig. 4). We put two floodlights (intensity is 5500 lumen) near target crack (distance is 50 cm). To evaluate effectiveness of light, we set four light condition: turn off, right one is running, left one is running, both lights are running (Fig. 5).

      In this paper, we focus on effectiveness of filming condition. But, the target crack area is too narrow than original photo image size. It would make decrease precision of crack detection and make difficult to evaluate an effectiveness of filming condition. All test images are clipped square surrounding crack area of each test data (Fig. 6).

      The crack annotation data of test data made by tracing crack area on the clipped images at each shooting distance. Note that different shooting distance annotation data are not the same (Fig. 7). The annotator could trace small crack on image taken by close shooting distance. On the other hand, the annotator could distinguish only big crack on image taken by far shooting distance. The crack annotation data made by one annotator.

      Fig. 4. Example of each shooting distance images

      Fig. 5. Example of light conditions

      Fig. 6. Example of test image clipping target crack area

      Fig. 7. Comparison of enlarged raw crack images and annotated images

   3. Results of crack detections

   We evaluate the effectiveness of shooting distance and light field by comparing precision and recall. These are results of image processing to detect crack on concrete surface of each conditions. We show an average result of crack detection of four different crack images. The precision and the recall of crack detection was calculated by the rate of concordance of AI detected crack and annotator annotated crack in pixel unit.

   Fig. 8 show precision values of different shooting distance and light of field. In the case of change shooting distance, worst precision value is 0.280 and best precision value is

   0.423. The difference is 0.143. In the case of light of field, we compare both light off case and the other case. When comparing the precision value of both light off and one of the other light condition, it decrease 0.036 in worst case and increase 0.023 in best case.

   Fig. 9 show recall precision values of different shooting distance and light of field. In the case of change shooting distance, worst precision value is 0.542 and best precision value is 0.859. The difference is 0.317. The recall values of far shooting distance become higher than one of close shooting distance. It is because of difference of resolution of test data. In the case of far shooting distance, annotator could not distinguish small crack on image. Because of a big crack which is easy to detect by crack detection remains in far shooting distance data, the recall value become high when shooting distance is far (Fig. 10). Note that this result does not ensure any far shooting distance make high recall value in always. In the case of light of field, comparing recall value as with comparing of precision value. When comparing the recall value of both light off and one of the other light condition, it decrease 0.084 in worst case and increase 0.022 in best case.

   Fig. 8. Comparison of enlarged raw crack images and annotated images

   Fig. 9. Comparison of enlarged raw crack images and annotated images

   Fig. 10. Comparison of enlarged raw crack images and annotated images

4. CONCLUSION

A bridge inspection needs much cost and an alternative is necessary for future bridge inspection in Japan. One of an alternative method of the current close visual inspection is the bridge inspections using images captured with a super-high- resolution camera. An image processing method could make efficiency of image based inspection but be effected by filming conditions of input image.

We have evaluated effectiveness of shooting distance and light of field for deep learning based crack detection by comparing precision and recall. The results of evaluation show the concrete effectiveness of filming conditions.

The results of this evaluation are not robust because of small test dataset. In the future work, we make big test dataset by decreasing filming conditions and target crack on concrete surface. We will make clear robust effectiveness of filming conditions and the reason of filming conditions change the results of crack detection.

REFERENCES

1. Menendez, E., Victores, J.G., Montero, R., Martínez, S., Balaguer, C.:Tunnel structural inspection and assessment using an autonomous robotic system. Automation in Construction, 87, pp. 117-126, 2018.

2. Pham, N.H., La, H.M., Ha, Q.P., Dang, S.N., Vo, A.H., Dinh, Q.H.: Visual and 3D mapping for steel bridge inspection using a climbing

   robot. ISARC 2016 – 33rd International Symposium on Automation and Robotics in Construction, pp. 141-149, 2016.

3. Xie, R., Yao, J., Liu, K., Lu, X., Liu, Y., Xia, M., Zeng, Q.: Automatic multi-image stitching for concrete bridge inspection by combining point and line features. Automation in Construction, 90, pp. 265-280, 2018.

4. H. Minami, M. Fujiu, J. Takayama, S. Suda, S. Okumura, K. Watanabe.: A basic study on diagnostic imaging technology for bridge inspection using super high resolution camera, Journal of Sociotechnology Research Network, 15, pp. 54-64, 2018. (in Japanese)

5. Y. Fujita, Y. Mitani, Y. Hamamoto.: A method for crack extraction on concrete surfaces using Image processing techniques. Journal of The Japanese Society for Non-Destructive Inspection, 56(7), pp. 371-377, 2007. (in Japanese)

6. Bu, G., Chanda, S., Guan, H., Jo, J., Blumenstein, M., Loo, Y.-C.: Crack detection using a texture analysis-based technique for visual bridge inspection. Electronic Journal of Structural Engineering, 14(1), pp. 4148, 2015.

7. Cha, Y.-J., Choi, W., Büyüköztürk, O.: Deep learning-based crack damage detection using convolutional neural networks. Computer- Aided Civil and Infrastructure Engineering, 32(5), pp. 361-378, 2017.

8. Z. Qin, Z. Zheng, L. Qingquan, Q. Xianbiao, W. Qian, W. Song.: DeepCrack: Learning Hierarchical Convo-lutional Features for Crack Detection, IEEE Transactions on Image Processing, 28(3), pp.1498- 1512, 2019.

9. T. Fukuoka, T. Minami, W. Urata, M. Fujiu, J. Takayama.: A data augmentation with Pix2Pix for deep learning method for detecting crack on an actual concrete, Journal of Japan Society of Civil Engineers, Ser. F4 (Construction and Management), 17(2), pp. I_27- I_35, 2019. (in Japanese)