U-Net (V2) Based Railway Track Crack Detection & Risk Analysis using Computer Vision

U-Net (V2) Based Railway Track Crack Detection & Risk Analysis using Computer Vision

Kiran Kumar Vangari (1)

Assistant Professor, Department of CSE (Artificial Intelligence & Machine Learning) CMR Technical campus Kandlakoya(V), Medchal Road, Hyderabad, Telangana 501401, India.

Sasi Priya Avula (2)

Department of CSE (Artificial Intelligence & Machine Learning) CMR Technical campus Kandlakoya(V), Medchal Road, Hyderabad, Telangana 501401, India.

Naga Venkata Venu Vardhan Anguluri (3)

Department of CSE (Artificial Intelligence & Machine Learning) CMR Technical campus Kandlakoya(V), Medchal Road, Hyderabad, Telangana 501401, India.

Rama Rao Guguloth (4)

Department of CSE (Artificial Intelligence & Machine Learning) CMR Technical campus Kandlakoya(V), Medchal Road, Hyderabad, Telangana 501401, India.

Abstract – Railroad transportation is an essential component of modern transportation infrastructure, and the preservation of the structural state of the railroad tracks is critical for the reliable and safe operation of railroad infrastructure. The continuous mechanical load, fatigue of materials, environmental factors and prolonged use may cause cracks to appear on the surface of the railroad tracks. In case when cracks remain undetected in the early stages, this may result in infrastructure degradation, high maintenance cost, downtime and even safety issues. The conventional railway track inspection process is done through visual manual inspections, which are time-consuming, error-prone and labour-intensive. In this work we present an approach to detect cracks on railroad tracks and assess the risks with the help of deep learning U-Net V2 algorithm.As an input image of the railroad tracks is provided. Following the preprocessing stage, the system uses the U-Net V2 model for crack segmentation. Then, the crack positions are detected through the use of the U-Net V2 model pixel by pixel. At the last stage, the system analyzes the crack features for the severity of the damage. Lastly, the system classifies the risk level of the cracks as either low, medium, or high, helping the maintenance crews prioritize their work. The proposed system can be classified as an automatic inspection system. The main purpose of such a system is to reduce the need for manual inspection of railroad tracks by human operators. In addition, it is aimed to assist in maintenance. That is, detection of potential defects before they cause any problem. Such an approach allows for the inspection of items overtime and detection of warning signs of any

malfunction that might occur. Thus, repairing can be performed beforehand not in a rush manner. Fewer interruptions, lower costs, security, etc. The system is completely autonomous. However, people still check whether the system functions correctly. This is a part of the equipment maintenance process.This new method utilizes Railroad Track Surface Faults Dataset. The data set consists of 5250 pictures of the railroad track with surface faults. The system is designed on such technologies as Python, OpenCV, Tensorflow, Keras, Numpy, Pandas, Scikit-learn, and Matplotlib. Such parameters will be used to measure the performance of the system in terms of image segmentation and classification: IoU, Dice coefficient, precision, recall, F1 score and accuracy.

Keywords: Identification of Railroad Tracks, Crack Detection, Computer Vision, U-Net V2, Deep Learning, Image Segmentation, Risk Assessment, Predictive Maintenance, Railroad Safety

1. INTRODUCTION

   Railway transportation is an integral part that connects cities and is meant for the purpose of transporting people and goods. The safety of railway transportation systems relies greatly on the infrastructure of railway tracks. The defects on the surface of railway tracks include cracks that are caused by excessive use of railway tracks, heavy traffic, mechanical loads, environment, and material degradation.

   The defect that occurs most frequently is a crack. In case a crack is detected at its early stages, it is very likely to prevent the propagation of this defect and the negative impact on the

   reliability of a railway track. Detection of cracks on time is vital since this will provide the opportunity to ensure the safety of railway transportation by carrying out the necessary repairing works. The detection of cracks is performed by means of visual inspection carried out by maintenance specialists.

   The advancements made in the fields of Artificial Intelligence (AI), Deep Learning (DL) and Computer Vision (CV) can be leveraged to build automated image inspection systems for detection of defects on railroad tracks. This approach replaces the cumbersome and time-taking task of manual image inspections. Specifically, semantic segmentation makes it possible to automatically detect anomalies and localize them within an image.

   In this paper, we propose a novel deep learning solution for automated crack detection and severity assessment on the railroad tracks. The proposed solution employs U-Net V2 image segmentation architecture. The system is able to accept images of the railroad tracks via the web interface, validate and preprocess them and run the crack detection algorithm using the segmentation model. The identified cracks are then classified according to their severity.

   The method presented above is an automated algorithm for crack detection in railroad tracks. For pixel level image segmentation and an attempt to detect cracks, we have used U-Net V2 model. The risk assessment method has been added to the crack detection algorithm. This method classifies detected cracks into three categories: Low Risk, Medium Risk and High Risk.

   Contributions from this work include the following:

   • Detection of Railroad Track Cracks via Automated Computer

     Vision-Based System.

   • Implementation of a U-Net V2-based segmentation model for pixel-based crack detection.

   • Extension of the crack detection task towards risk analysis for railroad tracks.

   • Implementation of a track inspection workflow for automated track inspection.

   • Implementation of an automated anomaly detection system for track maintenance prediction.

2. LITERATURE REVIEW

   Railway track inspection plays an important role since early detection of surface defects helps avoid further deterioration and provides increased safety of the railway network. Manual visual inspection is a conventional inspection technique of railroads. It is challenging since the process requires time and mistakes may easily be made during the process. A novel approach for automatic railroad defect inspection using computer vision and deep learning has drawn increasing attention from researchers. The first computer vision techniques for the detection of surface defects in railroads have been traditional image processing techniques that include image enhancement, thresholding, edge detection, morphology, and hand-crafted features. Such approaches are able to detect the surface defects successfully in well-controlled environments. Nevertheless, their performance drops sharply in cases when there are some differences in imaging conditions, surface textures, complexity of background, environment, and defects themselves.

   Convolutional Neural Networks (CNNs) have been broadly utilized for automated detection of railway defects. CNNs can classify railroad images in terms of existence and/or the kid of defect and lessen the labor required for the feature extraction process. However, image-level classification does not address the problem of presence and location of a defect within an image, and it is crucial in railway crack detection, because railway cracks can be smaller and/or irregularly located. Semantic segmentation is aimed at solving the problem of pixel-level classification, where each pixel of an image is classified to a certain class. A network architecture called U-Net was proposed by and has been broadly used for image segmentation tasks since then. The encoder learns features of high levels from the input image, whereas the decoder up-samples it to recreate the image and keep its spatial information. The encoder and decoder are coupled with skip-connections, concatenating feature maps from the encoder to the corresponding level of the decoder.

   As seen from the literature review, the majority of the convolutional networks are employed for image classification. Yet, in the case of railway crack detection, there is a need to identify not only the presence of the crack but also its location in the image. On the contrary, semantic segmentation is capable of identifying the fine details of objects, including cracks with irregular shapes. This is especially important for railroad crack inspection as railroad cracks usually possess a complicated pattern.

   Many improvements have been proposed to enhance the U-Net architecture: for instance, U-Net V2 is designed to efficiently learn abstract and context-aware features. The skip connection modifications in the U-Net V2 allow the decoder to efficiently learn encoder features. Railway crack image identification requires precise context information due to complicated patterns and fine details of cracks. Improvements of context modelling in U-Net V2 provide many opportunities for railroad crack segmentation.

   The automatic railroad surface defect detection problem has received a lot of attention in the literature by applying it to publicly available railroad datasets. For example, the Railroad Track Surface Faults Dataset contains images of a variety of railroad surface defects including cracks, flakings, grooves, joints, shellings, spallings and squats. The dataset is a collection of diverse railway track surface defects under varying lighting conditions. A wide range of train samples can make the semantic segmentation model generalize well in the test phase and therefore improve the segmentation results. Previous researches have revealed the capability of computer vision and deep learning-based algorithms for the problem of railway defect detection, but there are still some limitations.

   Previous methods are characterized by the low contrast of crack images, complexity of background texture, non-uniform lighting, and complex patterns of crack images. In addition, a crack itself in an image does not give any crack-specific information (e.g., severity of surface damage). Crack severity estimation will be important for a maintenance crew and will give them additional information about the severity of the crack. That is why this work offers a U-Net V2 based railroad track crack detection and risk analysis system.

   Detected cracks will be classified into Low, Medium and High risk categories to give the maintenance crew additional information on severity of the detected crack. Thus, it could be stated that all discussed literature proves the potential of using

   computer vision and deep learning techniques for railroad track inspection purposes. However, it is needed to develop an integrated crack detection and risk analysis system to give railroad track maintenance crew information on location and severity of the identified cracks. This work contributes to this purpose.

3. SYSTEM ARCHITECTURE AND METHODOLOGY

   The expected outcome of the proposed system will be an automated railway track inspection system with computer vision and U-Net V2 deep learning model for cracks detection and risk assessment. The architecture of the system consists of different modules for processing of the railway track image, beginning with the input up to producing the inspection report.

   The process begins with capturing or uploading of the image of the railroad track by the Maintenance Engineer using the web- based inspection system. The input image is verified that it is suitable for further processing.

   Afterwards, the pre-processed image is submitted to the U-Net V2 model. The U-Net V2 performs pixel segmentation and detects the cracks on the track. The output of the U-Net V2 model gives the position and size of the detected cracks.

   The severity of the defect is determined through risk analysis module by taking out some crack feature parameters from the detected crack region. The identified crack is categorized into either Low, Medium or High risk based on the severity analysis.

   The Maintenance Engineer is notified of the final inspection result. The result can be saved in the database for future use. An inspection report can be created as well based on the results obtained from the detection and risk analysis. System Administrator: Is responsible for system users, datasets, AI models, inspection reports and system configuration.

   The entire process of the proposed system can be represented in the following architecture: Maintenance Engineer Image Upload/Capture Image Validation Image Pre-processing

   U-Net V2 Model Crack Segmentation Crack Feature Extraction Severity Analysis Risk Classification Inspection Result/Report Database.

   As seen above, the proposed system uses modular architecture by separating the image processing part, deep learning based segmentation and the risk analysis in different modules. Thus, the system provides the Automated Inspection System with a clear workflow.

   Figure 1. Architecture of the System for Railway Track Crack Detection and Risk Analysis

   The methodology involved in the proposed system comprises a series of steps involved in image processing, deep learning, segmentation, and risk assessment.

   Figure 2. The Methodology of the Suggested Railroad Track Crack Detection and Risk Analysis System

4. EXPERIMENTAL SETUP

   This research study focuses on developing a deep learning and computer vision based railroad track crack detection and risk analysis system. The experimental study has been performed on the Railroad Track Surface Faults Dataset which includes 5250 real-world images under various environmental and lighting conditions. Seven types of surface faults are present in the railroad track dataset, and the proposed system focuses on cracks.

   For experimentation, Python 3.10+ was used as the primary programming language. Image processing and computer vision experiments have been conducted using OpenCV 4.10.0. For U- Net V2 deep learning model development, TensorFlow 2.x and Keras 3.5.0 were used. In experimental software environment, NumPy version 2.1.2 was used for mathematical computations and also Pandas 2.2.3, Matplotlib 3.9.2, and Scikit-learn 1.5.2. Visual Studio Code 1.95+ was the development environment used for the study.

   The hardware environment includes an Intel Core i5 CPU, 16 GB RAM, and at least 256 GB SSD storage capacity. An NVIDIA GPU was used in PC to develop and run deep learning models. Internet connection was necessary during development and other related activities.

   RailVision Homepage Rail tracks images can be uploaded using the formats: JPG, JPEG or PNG. The application has sophisticated calibrations and the option of “Run U-Net++ Defect Analysis”. This starts the defect detection process. Sample Images may be used for testing purposes.

5. RESULTS AND ANALYSIS

   1. Rail Surface Localization and Crack Segmentation

      This explains the rail surface localization and crack segmentation technique using the deep learning model of U-Net++. This figure rpresents the original rail surface image, Region of Interest (ROI) detection, predicted crack segmentation mask, and the

      final defects overlay. These steps help in detecting possible cracks and locating the areas of cracks on the railway surface.

      The output of the segmentation is the visualization of the cracks over the image of the rail surface. This is used as a starting point for other analysis such as area estimation of the cracks and calculation of coverage of the rail surface. The following figure explains the way RailVision processes railroad images.

   2. Quantitative Crack Measurement and Risk Assessment

      They display the outcomes of quantitative measurements based on the predicted crack segmentation. These include the number of crack pixels, number of rail surface pixels, estimated area coverage of the crack, number of distinct regions of cracks, skeleton length, and estimated crack opening width.

      The interface also contains data about detection accuracy, risk assessment, evaluation model, and image telemetry. This data helps the user to assess what kind of defects are expected to be found and make preliminary risk assessment regarding potential rail damage.

   3. Dataset Audit and Pipeline Verification

      The dataset audit interface shows information about the image dataset that is utilized in the RailVision pipeline. The information includes the total number of images, the number of crack images, presence of segmentation masks, and the distribution of images in train-test-validate sets.

      There is also an interface which displays the summary of defect categories and dataset structure. This information helps in verifying the availability of dataset and prepares the data for training, validating, testing, and verification of the machine learning pipeline.

   4. Performance Evaluation

      Estimated Value

Performance Indicator

Performance of the suggested U-Net V2 would be analyzed using the actual performance measures taken from the experiments. Below is an example of a performance evaluation table which can be used:

The real figures have to be found out through the test outcomes.

1. Track Detection History

The detection history module contains the history of the railway track images that have been detected before. The details shown include detection ID numbers, filenames, thumbnail pictures, time stamps, detection result status, risk level, and confidence score.

The user interface also provides for the search, filter, sort and download of PDF reports for easy access to historic information. This is possible due to the ability of the system to perform systematic record keeping.

1. CONCLUSION AND FUTURE SCOPE

   1. Conclusion

      Development of U-Net V2 based Railway Track Crack Detection and Risk Analysis The computer vision system is meant to detect

      and analyze any cracks found in images taken of railway tracks. The computer vision technology and U-Net V2 are used to detect and analyze cracks found in images of the railway tracks. The system classifies the railway tracks into three different categories namely low, medium or high risk depending on the nature of the crack. This system will reduce manual inspection processes, time spent on inspections and also help detect defects in railway tracks. The proposed computer vision system would make a valuable contribution to the field of railroad track inspection and maintenance.

   2. Future Scope

      The system can be enhanced in the following manner:

      1. Real-Time Detection: This model can easily be adapted to process video streams in real-time.

      2. Detection of Multiple Defects: The future scope of this system is to detect and analyze multiple defects including flakings, grooves, joints, shellings, spallings, and squats.

      3. Larger Dataset: The model can be further trained using a more extensive dataset that consists of various track, weather and lighting conditions.

      4. Enhanced Risk Analysis: Additional measurements and parameters related to the severity of the crack can be incorporated for accurate classification of the risk.

      5. Predictive Maintenance: The system can be expanded to predict future track failures using historical data of the track.

      6. Automated Notifications: An automated notification system can be setup to send alerts in case of high-risk cracks.

      7. Mobile and Edge Computing: The model can be optimized for mobile and edge computing platforms.

2. REFERENCES

• [1] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Medical Image Computing and Computer- Assisted Intervention, ser. Lecture Notes in Computer Science, 2015, vol. 9351.

• [2] “U-Net V2: Rethinking the Skip Connections of U-Net for Medical Image Segmentation,” in Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI), 2025.

• [3] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge: MIT Press, 2016.

• [4] OpenCV, “OpenCV Documentation,” OpenCV Foundation.

• [5] Google, “TensorFlow Documentation,” TensorFlow.

• [6] F. Chollet, Deep Learning with Python, 2nd ed. Shelter Island, NY, USA: Manning Publications, 2018.

• [7] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, pp. 84-90, 2017.

• [8] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for recognising images,” in Proceedings of the IEEE

  Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

• [9] Railroad Track Surface Faults Dataset, a dataset of real railroad track surface images and several types of surface defects including cracks, flakes, grooves, joints, shellings, spallings, and squatting.

• [10] NumPy Developers, “NumPy Documentation,” NumPy.

• [11] Scikit-learn Developers. Scikit-learn Documentation. Scikit-learn.