AI-Based Coma Patient Movement Monitoring

AI-Based Coma Patient Movement Monitoring

Dr. Kiran G M

Professor, Dept. of CSE SIET, Tumkur, India

Bhoomika H K

Dept. of CSE SIET, Tumkur, India

Chandana V

Dept. of CSE SIET, Tumkur, India

Kusuma G

Dept. of CSE SIET, Tumkur, India

Monika H C

Dept. of CSE SIET, Tumkur, India

Abstract – Coma patients require continuous observation because even small body movements may provide useful information about their condition. However, continuously monitoring patients manually can be difficult for healthcare workers, especially when several patients need attention at the same time. To address this problem, this paper presents an AI-based coma patient movement monitoring and alert system. The system uses computer vision and deep learning techniques to monitor and analyze patient movements. A camera is used to continuously capture the patient’s movements, and Media Pipe Pose is used to identify important body landmarks from the video frames. The extracted landmark information is processed by a Long Short-Term Memory (LSTM) model to recognize movement patterns over time. When a significant or abnormal movement is detected, the system generates an alert so that the caregiver can check the patient immediately. A monitoring dashboard is also provided to display the patient’s current movement status and maintain movement records for further observation. The proposed system provides a non-contact method for continuous patient monitoring and can help reduce the effort required for manual observation. It can also support healthcare professionals by providing timely movement information and alerts, particularly in situations where continuous direct supervision is not possible.

The interface can be accessed by an authorized doctor through the provided public IP address, allowing the doctor to view the patient’s movement remotely without being physically present near the patient. This can be useful when the doctor is located in another room or at a different location and needs to check the patient’s current movement status.

The system also includes an automatic email alert mechanism. When movement is detected, the system automatically captures a screenshot of the event and sends it to the doctor’s registered email address along with an alert message. The screenshot provides visual information about the detected event, allowing the doctor to understand what occurred and decide whether further attention is required.

This feature helps reduce the possibility of important movements being missed during continuous monitoring.

The proposed system therefore combines real-time movement detection, AI-based analysis, remote access, and email-based notification into a single monitoring platform. The system is designed to assist healthcare professionals rather than replace medical judgment.

INTRODUCTION:

A coma is a serious medical condition in which a person remains unconscious and has little or no response to the surrounding environment. Patients in a coma require continuous observation because changes in their body movement may provide useful information about their condition. In hospitals, this observation is generally performed by doctors, nurses, or caregivers. However, manually monitoring several patient movements and providing timely alerts to healthcare professionals can be difficult.

Advances in Artificial Intelligence (AI), Computer Vision, and Deep Learning have created opportunities for automated observation of patients. Camera-based monitoring provides a non-contact approach in which the patient’s movements can be observed continuously without disturbing the patient. In this work, Computer Vision techniques are used to detect and analyze the movement of a coma patient from a live camera feed.

The proposed system uses Media Pipe Pose to identify the patient’s body landmarks from the captured video frames. These landmarks represent different parts of the human body, such as the shoulders, elbows, wrists, hips, knees, and ankles. The extracted landmark information is analyzed over a sequence of frames to identify changes in the patient’s body movement. An LSTM (Long Short-Term Memory) model is used to learn the temporal patterns in the movement data and identify significant patient movements.

One of the key features of the proposed system is remote monitoring through a public IP address. The live monitoring interface can be accessed by an authorized doctor remotely.

The system also provides automatic screenshot capture and email alerts when significant movement is detected.

LITERATURE REVIEW:

An IoT-based healthcare monitoring system for coma patients in 2021 proposed continuous monitoring using sensors and communication technologies. The system was designed to collect patient-related parameters and transmit them to caregivers or medical professionals, allowing the patient’s condition to be monitored remotely. Such systems can reduce the need for continuous physical observation and can provide timely information when a patient’s condition changes. However, sensor-based monitoring requires additional hardware and physical contact with the patient and provides limited information about the exact type or location of body movement. A computer-vision-based system can overcome some of these limitations by analyzing patient movement directly from video.

Research on LSTM-based human activity recognition in 2022 demonstrated the effectiveness of Long Short-Term Memory networks for recognizing activities from sequential movement data. Because LSTM networks retain information from earlier stages of a sequence, they are well suited to analyzing movements that evolve across multiple time steps. The findings indicated that LSTM architecture can learn time-dependent patterns within movement sequences and use them to distinguish human activities. Nevertheless, the research concentrated on ordinary human activity recognition rather than movement assessment in unconscious or critically ill individuals.

A study on video-based patient monitoring using deep learning in 2023 investigated the use of continuous video streams and machine-learning techniques for monitoring patient behaviour in healthcare environments. The research demonstrated that video-based systems can provide non- contact observation and can help identify changes in patient activity. However, existing approaches have mainly concentrated on general patient or ICU monitoring and have limited integration of real-time dashboards, automatic screenshot capture, and email-based alerts for doctors.

Research on pose-estimation-based human movement analysis in 2023 demonstrated that computer vision can identify important body landmarks from video and convert them into numerical coordinates for movement analysis. Pose estimation techniques can detect body points such as the shoulders, elbows, wrists, hips, knees, and ankles, allowing the system to track changes in body position over time. This approach is useful for healthcare applications because it provides a non-contact method of monitoring patient movements without requiring wearable sensors. However, pose estimation performance can be affected by poor lighting, camera position, occlusion, and blankets covering parts of the body.

A 2024 study on human sleeping pose estimation for in-bed patient monitoring investigated computer vision and deep learning to identify the posture of a person lying on a bed. The system used infrared images and pose-estimation techniques to analyze body position, making it suitable for environments where normal cameras may have difficulty operating under low-light conditions. The research demonstrated the potential of pose estimation for non- contact monitoring of bedridden individuals. However, the system primarily focused on sleeping posture and did not specifically classify abnormal movements of coma patients using Media Pipe and LSTM.

A 2024 review examined how machine-learning and deep- learning approaches are being applied to the diagnosis and prognosis of disorders of consciousness. The study reported that AI-based methods can help differentiate different consciousness states and support prediction of patient recovery. The review highlighted several limitations, including small datasets, differences among patients, the absence of standardized data, and inadequate clinical validation. Although these approaches provide valuable support for neurological assessment, they mainly focus on diagnosis and prognosis rather than continuous observation of physical movements. Therefore, a real-time vision-based monitoring system can provide complementary information about patient movement and activity.

Lee and Laureys presented a review of Artificial Intelligence and Machine Learning in Disorders of Consciousness, including coma, unresponsive wakefulness syndrome, and minimally conscious states. The study examined AI techniques for diagnosis, prognosis, and assessment of patients with disorders of consciousness. The findings showed that machine learning can assist clinicians in identifying different states of consciousness and predicting patient outcomes using information obtained from EEG, neuroimaging, and clinical data. However, most existing approaches depend on neurological or physiological information and do not provide continuous camera-based monitoring of patient body movements.

Dai et al. proposed an automated ICU agitation monitoring system using continuous video streaming and deep learning to monitor patient movements. Patient motion in the recorded video was quantified using MediaPipe together with the Background Subtractor MOG2 technique. Among the evaluated approaches, LSTM produced the strongest result, reporting 92% accuracy and an AUC value of 0.91. The study demonstrated that computer vision and LSTM can support continuous, non-contact monitoring of critically ill patients. However, the system mainly focused on ICU agitation and sedation monitoring rather than specifically monitoring coma patient movements or providing automated email alerts.

METHODOLOGY:

1. Video Data Collection and Preprocessing:

   The first stage of the proposed system is the collection and preprocessing of live video data from a camera positioned near the coma patient’s bed. The camera continuously captures the patient’s movements and provides the video frames to the monitoring system.

   Fig.1: shows the patient video capture and preprocessing process.

2. Human Pose Estimation Using Media Pipe:

   After obtaining the video frames, Media Pipe Pose is used to identify the patient’s body structure and estimate important body landmarks. The pose-estimation model detects key points such as the shoulders, elbows, wrists, hips, knees, and ankles. These landmarks provide information about the position and movement of different parts of the patient’s body. Instead of directly processing the complete image, the system uses the detected landmark coordinates for movement analysis. This reduces the complexity of the input data and allows the system to perform movement monitoring in real time.

   Fig.2: shows the Media Pipe Pose landmark detection process.

3. Feature Extraction and Sequence Formation:

   Movement-related features are derived from the coordinates produced by MediaPipe Pose. The position of each detected landmark is represented using numerical coordinates, which are collected continuously from consecutive video frames. These coordinates are organized into a sequence so that changes in body position over time can be identified. The sequence-based representation is important because a patient’s movement cannot always be determined from a single frame. The extracted features are therefore prepared as temporal data for input to the LSTM model.

   Fig.3: shows the feature extraction and sequence formation process.

4. LSTM-Based Movement Classification:

   The extracted sequence of body-movement features is provided to a Long Short-Term Memory (LSTM) neural network. LSTM is selected because it can learn relationships between data points occurring at different time steps. The model analyzes consecutive body positions and identifies movement patterns from the patient. Based on the learned patterns, the system determines whether significant patient movement has occurred. Training the LSTM with sequential movement data enables it to distinguish patterns and classify movement while monitoring is in progress.

   Fig.4: shows the LSTM-based movement classification process.

5. Real-Time Movement Monitoring:

   After the LSTM model performs the classification, the result is displayed through the monitoring system. The live camera feed allows the patient’s movement to be observed continuously, while the detected movement information is displayed on the monitoring interface. The system continues processing incoming frames and updating the patient’s movement status. This provides continuous observation and reduces the requirement for a doctor or caregiver to manually monitor the patient at all times.

   Fig.5: shows the automatic screenshot capture and event recording process.

6. Screenshot Capture and Event Recording:

   When significant movement is detected, the system automatically captures a screenshot from the live video feed. The screenshot provides visual evidence of the patient’s condition at the time the movement was identified. Each detected event can be stored together with information describing the movement and its corresponding timestamp. This information can be useful for reviewing the patient’s movement history and providing the doctor with additional evidence about the detected event.

   Fig.6: shows the email alert notification sent to the doctor with the captured screenshot.

7. Email Alert Notification:

   Following movement detection, the system automatically generates an alert for the authorized doctor. The captured screenshot is attached to the email notification, allowing the doctor to understand the patient’s condition without continuously watching the live video. This feature helps reduce response time when significant movement occurs and provides an additional layer of patient safety. The alert mechanism is designed to notify the doctor immediately after the movement event is detected.

   Fig.7: shows the automatic email alert sent to the doctor with the captured screenshot and movement details.

8. Remote Monitoring Through Public IP Address:

A web-based interface allows an authorized doctor to reach the monitoring system remotely through its public IP address. The authorized doctor can use the public IP address to access the monitoring system and observe the patient’s live movement from a different location. This feature enables remote supervision and reduces the need for the doctor to remain physically near the patient’s bed. The combination of live monitoring and automatic email notification provides both continuous observation and event- based communication.

Fig.8: shows the remote monitoring dashboard accessed by the doctor through the public IP address.

RESULTS AND ANALYSIS:

The proposed AI-based coma patient movement monitoring and alert system was developed to provide continuous and automated monitoring of patient movements. The system successfully captures live video through a camera and processes the video frames for movement analysis. MediaPipe Pose obtains the necessary body landmarks, and these numerical features are subsequently supplied to the LSTM network. The LSTM model analyzes the sequence of movements and identifies whether significant patient movement has occurred.

When movement is detected, the system automatically captures a screenshot and records the movement event with the corresponding information in the monitoring dashboard. The detected results are stored in the dashboard, allowing the doctor to review the patient’s movement history. In addition, an automatic email alert containing the movement information and captured screenshot is sent to the doctor. The monitoring dashboard can also be accessed remotely through a public IP address, enabling the doctor to observe the patient without being physically present near the monitoring system.

Overall, the system provides real-time movement detection, event recording, screenshot capture, dashboard-based monitoring, and email notification in a single integrated framework. These results demonstrate that the proposed system can reduce the need for continuous manual observation and provide timely information to the doctor when patient movement is detected.

Implementation Challenges:

• Real-time video processing: Handling continuous camera input smoothly.

• Movement detection: Detecting small patient movements accurately.

• Pose estimation: Maintaining landmarks when the patient’s body is partially hidden.

• LSTM training: Preparing suitable movement sequences for the model.

• Dashboard integration: Displaying and storing detected movements correctly.

• Email alerts: Ensuring alerts and screenshots are sent successfully.

• Remote access: Maintaining stable dashboard access through the public IP.

• System integration: Connecting all modules and making them work together reliably.

  DISCUSSION:

  The proposed AI-based coma patient movement monitoring and alert system provides an automated approach for continuously observing patient movements. Combining MediaPipe Pose with LSTM allows body motion to be evaluated directly from the live camera stream, reducing dependence on constant manual supervision. Body- landmark measurements provide the movement features, while the LSTM examines their progression over time to perform classification.

  One of the important advantages of the proposed system is the integration of movement detection with real-time monitoring and alert mechanisms. When movement is detected, the system captures a screenshot and stores the event information in the dashboard. An email notification containing the movement details and screenshot is also sent to the doctor. This provides timely information and helps the doctor review the patient’s condition remotely.

  The remote-access feature further improves the usability of the system by allowing the doctor to access the monitoring dashboard through a public IP address. The dashboard provides the recorded movement information and helps in reviewing previous events. However, the system’s performance may be affected by factors such as camera position, lighting conditions, patient visibility, and body occlusion. Overall, the integration of AI-based movement analysis, dashboard monitoring, screenshot capture, and email notification provides a practical framework for assisting continuous coma patient monitoring.

  Furthermore, the system provides a centralized platform for monitoring and managing detected movement events. By combining real-time video analysis, LSTM-based movement classification, dashboard storage, screenshot recording, and email notification, the proposed system creates a complete monitoring workflow. This integration can help caregivers and doctors receive movement-related information quickly and maintain a record of detected events for later review. The system therefore demonstrates the potential of AI and computer vision techniques to support efficient and continuous monitoring of coma patients.

  CONCLUSION AND FUTURE SCOPE

  Conclusion:

  The proposed AI-based coma patient movement monitoring and alert system provides an automated approach for continuously monitoring patient movements using live video analysis. The integration of MediaPipe Pose and LSTM enables the system to analyze body movements and identify movement events. When movement is detected, the system captures a screenshot, records the event in the dashboard, and sends an email notification with the captured image to the doctor.

  The system also provides an effective remote monitoring facility through a public IP address, allowing doctors to access the monitoring dashboard from a different location. The dashboard stores the detected movement events and provides a record that can be reviewed when required. Whenever movement is identified, the system automatically captures the relevant screen and sends an email notification to the doctor along with the screenshot and movement information. This reduces the dependency on continuous manual observation and helps provide timely awareness of patient activity. The integration of video monitoring, pose estimation, LSTM-based movement analysis, dashboard storage, screenshot capture, email notification, and remote access makes the proposed system a complete monitoring framework. Although the system is intended to assist doctors and caregivers rather than replace clinical observation, it illustrates how AI and computer vision can be applied to assist with continuous monitoring of patients in a coma.

  FUTURE WORK:

• The system can be further improved to detect very small and subtle movements of coma patients with higher accuracy. Advanced computer vision techniques can be incorporated to reduce false detections and improve the reliability of movement monitoring.

• In the future, the LSTM model can be trained using a larger and more diverse dataset containing different patient postures, movements, camera angles, and environmental conditions. This can improve the model’s ability to recognize movements in real-world situations.

• The proposed system can be extended to identify and classify different types of patient movements. This would provide more detailed information to doctors instead of simply indicating whether movement has occurred.

• Future development can focus on improving system performance under different lighting conditions, camera positions, partial body visibility, and changes in patient posture. This can make the system more suitable for continuous monitoring in different hospital environments.

• Cloud technology can be integrated to securely store movement events, screenshots, timestamps, and monitoring records. This would allow authorized doctors to access historical patient information whenever required.

• A dedicated mobile application can be developed to allow doctors to monitor patients remotely using smartphones. The application can display movement events and provide immediate notifications whenever unusual activity is detected.

• Along with email notifications, SMS, mobile push notifications, or other communication methods can be added. Multiple alert mechanisms can help ensure that important movement events reach the doctor without delay.

• The system can be extended to monitor multiple patients simultaneously. Each patient can have a separate monitoring profile and dashboard containing their movement records, screenshots, and alert history.

• The system can be enhanced to automatically generate daily or weekly reports containing deected movement events, timestamps, screenshots, and alert information. Such reports can help doctors review the patient’s monitoring history more efficiently.

ACKNOWLEDGMENT:

We would like to express our sincere gratitude to our project guide for the valuable guidance, continuous support, and constructive suggestions throughout the development of our project. We also thank the Head of the Department and all the faculty members of the Department of Computer Science and Engineering for their encouragement, technical support, and for providing the necessary facilities and resources. We are grateful to our institution for giving us the opportunity and support to carry out this work successfully. Finally, we would like to thank our parents, friends, and everyone who directly or indirectly supported and encouraged us throughout the completion of the AI-Based Coma Patient Movement Monitoring and Alert System.

REFERENCES:

1. M. Dai et al., An automated ICU agitation monitoring system for video streaming using deep learning classification, BMC Medical Informatics and Decision Making, vol. 24, 2024, doi: 10.1186/s12911-024-02479-2.

2. M. Lee and S. Laureys, Artificial intelligence and machine learning in disorders of consciousness, Current Opinion in Neurology, vol. 37, no. 6, pp. 614620, 2024, doi: 10.1097/WCO.0000000000001322.

3. S. Sabbu et al., LSTM-Based Neural Network to Recognize Human Activities Using Deep Learning Techniques, Applied Computational Intelligence and Soft Computing, 2022, doi: 10.1155/2022/1681096.

4. M. A. Khatun et al., Deep CNN-LSTM With Self-Attention Model for Human Activity Recognition Using Wearable Sensor, IEEE Journal of Translational Engineering in Health and Medicine, vol. 10, 2022, doi: 10.1109/JTEHM.2022.3177710.

5. S.-H. Yang, D.-G. Baek, and K. Thapa, Semi-Supervised Adversarial Learning Using LSTM for Human Activity Recognition, Sensors, vol. 22, no. 13, p. 4755, 2022, doi: 10.3390/s22134755.

6. Y. Li and L. Wang, Human Activity Recognition Based on Residual Network and BiLSTM, Sensors, vol. 22, no. 2, p. 635, 2022, doi: 10.3390/s22020635.

7. Smart Health Monitoring System for Coma Patients using IoT, IEEE Conference Publication, 2023.

8. Health Care System for Coma Patients, International Journal of Emerging Technologies and Innovative Research, vol. 8, no. 7, pp. f271f278, 2021.

9. Real-Time Health Monitoring System using IoT for Comatose Patients, 2021.