An AI-Based Adaptive Learning Framework and Algorithm with Gesture Interaction and Behavior-Mood Aware Adaptation for IDD Children

An AI-Based Adaptive Learning Framework and Algorithm with Gesture Interaction and Behavior-Mood Aware Adaptation for IDD Children

Gaurav

Department of Computer Science and Engineering GITAM, Kablana Jhajjar, India

Ritu Kadyan

Department of Computer Science and Engineering GITAM, Kablana Jhajjar, India

Abstract – The settings that children with Intellectual and Developmental Disabilities (IDD) need are interactive, emotionally receptive and personally adaptive. Traditional e-learning products are not very flexible and they are based on common input features that inhibit interaction and customization to learners with IDD. The current paper introduces an adaptive learning system and algorithm powered by AI which that combines gesture-based interaction, learning games, analyzing behaviors and recognition of moods based on interaction to provide individualized learning experiences.

The suggested system allows children with IDD to play with the educational games and learning materials with the help of natural hand movements. The interaction indicators used to monitor the behavior of learners are response time, gesture repetition, inactivity time, and patterns of task completion. It uses an Interaction-Based Mood Recognition (IMR) mechanism which recognizes the emotional states of a learner such as engaged, neutral, frustrated and fatigued without any intrusive sensors or biometric measurements. According to the results of the joint analysis of behavioral tendencies and moods, the adaptive algorithm dynamically changes the content of learning, the complexity of a game, the speed of instructions, and the level of guidance. As opposed to the current adaptive learning systems, which mostly use performance scores, the proposed solution focuses on emotional comfort, long-term participation, and adaptability in learners. The system architecture, framework design, adaptive algorithm and methodology of evaluation are outlined. Parameters that are observed in order to conduct experimental evaluation are engagement stability, interaction accuracy, and mood-sensitive adaptation.

Keywords – Artificial Intelligence, Adaptive E-Learning, Gesture-Based Learning, Intellectual and Developmental Disabilities, Emotion-Aware Systems

1. INTRODUCTION

   Intellectual and Developmental Disabilities (IDD) compass a list of disorders marked by the disabilities of intellectual functioning, adaptive behavior and communication skills and emotional regulation. Poor concentration, the inability to express emotions, and the lack of motor skills are the main problems that children with IDD have to deal with, thereby impacting the learning process greatly.

   Conventional classroom education and a majority of e- learning platforms aim at neuro typical learners and have strict instructional frameworks. These systems are more interactive with the use of keyboard, mouse or touch which might not

   be appropriate to the children with cognitive or motor disabilities. Consequently, IDD learners are often disengaged and frustrated. Gesture-based interfaces and adaptive learning systems have become possible with the advancement of Artificial Intelligence (AI), computer vision, and human- computer interaction. It was also demonstrated that motivational and participation levels were enhanced through educational gameplay. the current systems can hardly incorporate gesture interaction, learning games, behavioral analysis, and mood-aware adaptation into one, specific framework of IDD education.

   The study fills this gap by introducing a new AI-based adaptive learning system and algorithm that integrates the elements of gesture-controlled educational gameplay with behavior and mood-sensitive material adaptation.

2. LITERATURE REVIEW

   The recent progress in Artificial Intelligence (AI) and educational technologies have had a substantial impact on the creation of the assistive learning system of children with Intellectual and Developmental Disabilities (IDD). The adaptive learning systems, gesture-based interaction, education games, and, affect-aware learning have been studied as separate algorithms to enhance access and interaction with learners. Nevertheless, current literature has shown that these strategies have been applied individually and without a cohesive framework that has been designed specifically to IDD education. Adaptive learning systems have been extensively researched with an aim of customizing the instructional content according to the profile and performance metrics of learners. Brusilovsky proposed adaptive hypermedia systems based on dynamically altered learning tracks based on the user behavior. Although these systems are useful in traditional learning rooms, they are mainly based on testing outcomes and preset rules, and they are only slightly customized in real time. Khamparia and Pandey also emphasized that the majority of adaptive elearning systems prioritize on cognitive performance, leaving behavioral engagement and emotional comfort, which are very important in terms of learning with IDD. In the same manner, the AI-based tutoring systems that are mentioned by Saleh et al. show that they deliver content intelligently but do not provide natural interaction mechanisms

   and IDD-specific adaptation strategies.

   Gesticulatory communication is one of the areas that have become viable in order to achieve accessibility and natural interaction in learning discourse. Gesture recognition methods that involve the use of vision as examined by Wachs et al. have been seen to enhance the interaction of the users and lessen the use of conventional input devices. Nevertheless, the majority of gesture-based learning systems focus on the precision of interaction, in lieu of adaptive learning outcome. The educational applications and serious games suggested by Hussein et al. enhance the motivation of learners and use fixed learning pathways and are not able to modify the content according to the behavior or emotional condition of a learner. Consequently, such systems might not be able to maintain the interest of learners with different cognitive and emotional needs.

   It has been demonstrated that educational games raise motivation and participation of children with developmental disorders. Work by Boucenna et al. suggests that through interactive and game-based learning, children with autism, as well as conditions that are related, can be more engaged. Nevertheless, most of the educational games rely on fixed levels of difficulty and fixed rules, which prevents them to address the frustration and exhaustion of learners, as well as their different learning pace. The latter is especially a problem with the IDD learners where the level of task difficulty and instructional support has to constantly change.

   Behavior-conscious and affect-conscious learners seek to determine the level of learner interaction and mood using patterns of interaction. The engagement in the digital learning contexts was also identified by Bosch et al. based on the behavioral analytics, yet the method was grounded on the traditional input approaches and failed to incorporate the adaptive learning process. DMello and Graesser have discussed the relevance of emotional awareness to learning systems and showed better results when affective states were taken into account. Nevertheless, a great number of affect conscious systems are based on facial expressions analysis or physiological sensors, which present ethical, privacy and practical issues, particularly when used with children with IDD. On balance.

   The literature shows that there are a number ofgaps in research such as the absence of unified systems that are able to integrate gesture-based interaction, adaptive learning, educational gameplay, and non-invasive mood recognition all into one system. The current strategies are either interaction but not adaptation, adaptation but no emotional awareness, or it is emotion recognition but not ethically feasible. Such restrictions drive the necessity to have a coherent, learner oriented system that facilitates the process of natural interaction, behavior analysis, and mood-sensitive adaptation. These gaps are the core of the proposed AI-based adaptive learning framework and algorithm that is introduced in the current research.

3. PROBLEM STATEMENT

   Although there are technological advances, there are a few

   limitations in the current systems of learning by children with IDD:

   Adaptation without emotional feeling, that is, without awareness of variation.

   Poor blending between learning and adaptive intelligence. Lack of explicit mood recognition operating mechanisms.

   Poor real-time personalization centered on the comfort of the learner.

   The intelligent learning system is required to facilitate the natural interaction process, provide educational gameplay and be ethically aware of the mood of learners besides dynamically adjusting content

4. OBJECTIVES OF THE RESEARCH

   The objectives of this research are:

   • Design a novel AI-based adaptive learning framework for children with IDD

   • To enable gesture-based interaction for intuitive learning and gameplay

   • To integrate educational games as structured learning activities

   • To recognize learner mood using interaction-based behavioral indicators

   • To develop a new adaptive algorithm combining gesture, behavior, and mood data

   • To improve learner engagement, emotional comfort, and learning effectiveness

5. PROPOSED AI-BASED ADAPTIVE LEARNING FRAMEWORK

   The suggested model is based on a learner-based approach, where interaction, analysis, and adaptation are combined into a closed system.

   1. Gesture-Controlled Learning and Game Interaction Module

      The module allows the children with the IDD to engage in educational learning games through natural hand gestures. The games are aimed at the following learning goals: recognition, matching, sequencing, and simple problem-solving. Gestural control is available in supporting such actions like selection, navigation, and confirmation, which helps to lower the usage of conventional input devices and maximize interaction.

   2. Behavior Monitoring Module

      The system keeps checking the behavior of interaction such as the latency of responding, the repetition frequency of the gestures, the duration of inactivity, the abandonment of tasks, and completion frequency. These indicators give the insight about the level of engagement and difficulty of learners.

   3. Interaction-Based Mood Recognition (IMR) Module

      The Interaction-based mood recognition (IMR) module belongs to the spectrum of mood recognition models, existing today in machine learning.

      An Interaction-Based Mood Recognition (IMR) module is suggested in the proposed system to predict the emotional state of a learner without intrusion. Recognition of mood is conducted on the basis of the behavioral interaction patterns as opposed to facial expressions or physiological signals. Prolonged response time, repetition, prolonged inactivity, and task abandonment are mapped to depressed moods like frustrated or tired, and regular interaction is a pointer to engaged or neutral moods.

   4. Adaptive Learning and Gameplay Engine

      This engine is dynamic and adjusts learning content and gameplay parameters based on a synthetical analysis of behavior and mood. The adaptations involve modifications to content difficulty, speed of the gameplay, teaching pace, and level of teaching guidance to ensure an emotional comfort level and engagement.

   5. Feedback and Progress Repository

   Behavioral trends, mood transitions and learning outcomes are stored to maintain long-term personification and learning process.

6. PROPOSED ADAPTIVE ALGORITHM

   Gesture Based Behavior and Mood-Aware Adaptive Algorithm (GBMAA).

   Input: gestures, behavioral cues. Delivery: Custom learning material and game play.

   Algorithm Steps are :-

   1. Initialize learner profile and establish engagement baseline.

   2. Record live gesture information during learning games.

   3. Map concrete gestures to corresponding system actions.

   4. Monitor behavioral indicators such as response time, inactivity, and repetition.

   5. Extract interaction features and infer learner mood using IMR.

   6. Classify mood as engaged, neutral, frustrated, or fatigued.

   7. If mood is fatigued or frustrated:

      1. Reduce game difficulty.

      2. Slow instructional pace.

      3. Provide guided or visual assistance.

   8. If mood is engaged:Progressively increase game difficulty and complexity.

   9. Deliver modified learning game content.

   10. Update learner profile based on feedback.

   11. Repeat the adaptation cycle.

   Algorithm Novelty: Gesture interaction, educational gameplay, behavior analysis, and mood recognition become part of the same adaptive decision making process in the algorithm.

7. METHODOLOGY

   The proposed system is based on a modular and adaptive approach, which is intended to children with Intellectual and Development Disabilities. The methodology unites gesture

   interaction, behavior monitoring, interaction mood recognition and adaptive learning logic into a workflow.

   The main input mechanism is the camera-based gesture recognition interface. This interface records the movements of the hands and converts them into system commands like, selection, navigation, confirmation and response. Personal interaction Gestures lead to less reliance on keyboards or touch related interfaces, which tend to be problematic to IDD learners because of problems with motor coordination.

   There is continuous gathering of behavioral data during the interaction. These are response time, repetition frequency of gestures, inactivity, and behavior of completion of a task. Because of these behavioral characteristics it is processed locally in order to achieve low system latency, data privacy and real time responsiveness. Local processing equally eliminates the use of cloud-based computation in making core adaptive decisions.

   A mood recognition mechanism based on the interaction also relies on the behavioral pattern to indicate the mood of the learner, instead of a biometric intrusion. The deduced mood condition is integrated with behavioral signs and inputted into an AI adaptation engine that is governed by rules. This engine is dynamic and will adjust the learning content, learning pace and level of activity in real time.

   The adaptive logic does not rely on previously determined sequences of learning, but instead enables the system to offer a personalized learning process throughout the game, instead of adhering to a fixed set of teach-backed instruction.

8. EXPERIMENTAL DESIGN

   The experimentl design will be used to test the usefulness of the proposed system considering the engagement, quality of interaction, emotional comfort, and direction of learning.

   A. Experimental Setup

   The target population of the experimental research is children with mild-moderate Intellectual and Developmental Disabilities, because such children can actively engage with the digital learning processes but still enjoy substantial support due to the help of an adaptive one.

   Gesture learning Learning occurs in the form of game-based learning that involves gesture-controlled educational activities.

   These activities are aimed at the basic learning goals of basic numeracy, recognition of objects, matching and sequencing. Educational games are motivating, and their use does not lose a sense of purpose.

   Experiments are done in special education learning, controlled learning sessions either in classrooms or supervised learning sessions. The system will give learners the freedom to interact freely and record behavioral and interaction data in each session.

9. RESULTS AND PERFORMANCE ANALYSIS

   The experimental observations indicate that the suggested AI- based adaptive learning system is much more effective in enhancing the level of engagement and the quality of interaction with learners in comparison to the traditional

   techniques of learning. The traditional learning methods as depicted in Figure 1 (a) have a higher percentage of learners in the low engagement, but the proposed adaptive system moves a significant amount of learners to the moderate and high levels. This is the enhancement of the efficiency of the interaction based on gesture, as the learners can be more engaged in the interactive process, thus, the initiation of the task is faster and the hesitation decreases in the course of learning.

   Further development of the performance is seen in the response time in Figure 1(b). The findings demonstrate the steady decrease in the mean response time between consecutive learning sessions, and the final resultant improvement of about 28 per cent between the third and fourth learning sessions. Such a decrease means that the learners are familiar, confident, and comfortable with the gesture-based interaction mechanism and the adaptive pacing offered by the system is positive.

   Mood-aware adaptation contribution is well demonstrated as shown in Figure 1(c). By enabling mood adaptation, a higher rate of completing tasks is observed, frustration incidences are decreased to a large extent, and engagement recovery is better than the system that does not have mood adaptation. These findings indicate that prompt adaptive actions, such as simplification of tasks, slowing of the rate at which interactions occur or direct assistance in cases of frustration or exhaustion, are fundamental in increasing emotional comfort and avoiding disengagement.

   There is increased consistency in interaction as each session is repeated which means that the learner becomes more confident and lessens the errors in interaction. It is ensured that adaptive content selection which is carried out through the combination of both mood and behavioral analysis will allow the learning activities to be challenging but never too difficult where cognitive overload and boredom can be prevented. On the whole, the findings shown in Figure 1 indicate that the combination of the gesture-based interaction, the behavior analysis, and the mood-conscious adaptive intelligence allows making a more inclusive, emotionally-responsive, and efficient learning process of children with the Intellectuals and Developmental Disabilities.

   Along with the improvements in performance, the results observed suggest that the proposed system enables the continuity of learning over the long periods of interaction. The learners could stay engaged during extended periods without external intervention, which implies that, the adaptive learning flow can well match the instructional requirements with the capacity of an individual learner. The joint effect of gesture interaction and mood-aware adaptation is the lack of abrupt learning interruptions and a lack of negative emotional reaction, as seen in the static systems. This ongoing presence is especially significant to children with Intellectual and Developmental Disabilities, where regular interaction and emotionally secure learning situations impact directly on the long-term learning performance. The results indicate that the proposed framework is capable of improving the quality of interactions not only in the immediate but also in a more stable and supportive learning process in the long run.

   Fig. 1. The effectiveness of behavior-mood aware adaptation in increasing task completion, decreasing frustration episodes, and improving engagement recovery is demonstrated by a performance evaluation of the suggested AI based adaptive learning system that shows (a) engagement stability comparison between traditional learning and the suggested approach, (b) improvement in average learner response time across multiple learning sessions, and (c) engagement recovery.

10. COMPARATIVE DISCUSSION

    The proposed framework has a number of advantages when contrasted with the conventional e-learning systems. Traditional systems are based on the delivery of static content and the use of standard input devices and offer little personalization and access.

    Score-based systems of adaptive learning are only able to adjust the content based on the assessment, and do not respond to emotional and behavioral changes in real-time. The current systems of educational games tend to operate on predetermined levels of difficulty without being emotionally conscious.

    Conversely, the specified framework incorporates gesture based interaction, educational play, behavior tracking, and

    mood detection into one adaptive system. Such integration allows driving personalization and learning that is responsive to emotions and striving to achieve the highest levels of engagement and learner comfort

11. SCALABILITY AND DEPLOYMENT CONSIDERATIONS

    The framework proposed is geared towards implementation into the learning environments in the real world. It is able to run on the regular camera equipment; therefore, it can be used in the classroom and in the home learning environment.

    The modular system architecture is one which is easily integrated with the current e-learning platforms. The core adaptive decisions are locally processed to make sure that they have low latency and privacy levels. Large-scale analytics, content updates, and system monitoring can be done through optional cloud-based extensions without data security being jeopardized of learners.

    The portability design is made possible by the resource constrained nature of the learning location, with the ability

    to scale to various learning facilities.

12. ETHICAL AND ACCESSIBILITY CONSIDERATIONS

    The design of the proposed system is primarily based on ethical considerations. The system is not intrusive in terms of emotion recognition like facial recognition or even physiological sensing. Mood recognition is based only on the behavior during the interaction process so it is not intrusive of privacy, ethical requirement, and safety of the learners.

    Data gathering is reduced and restricted to interaction level characteristics. Principles of accessibility like simplified visuals, consistent interaction patterns and adaptive pacing are factored in so as to accommodate various cognitive and motor skills.

    The system puts priority on autonomy, comfort, and inclusiveness of the learner during the learning process

13. CONCLUSION AND FUTURE SCOPE

In this study, a new AI-based adaptive learning framework and algorithm was provided that is specifically aimed at meeting the learning requirements of children with Intellectual and Developmental Disabilities (IDD). The suggested system will combine gesture-determined interaction, instructional gameplay, behavior examination, and interaction-based mood perception into a solitary adaptive learning condition. The proposed framework will focus on emotional comfort, continuity of engagement, and real-time individualization, unlike traditional assistive learning systems which are based on either an inert representation of material or an evaluation-driven modification, which are essential contributors to effective IDD learning.

The suggested framework overcomes a number of major shortcomings noted in the current assistive learning technologies. Conventional e-learning systems cannot maintain engagement because of strict instruction flow and reliance on the conventional input devices. The system also allows learners to interact more intuitively and naturally, as it introduces the element of gesture-based interaction that eliminates slowness and motor coordination difficulties. Educational gameplay integration additionally serves a greater purpose in increasing motivation as it turns learning exercises into goal-oriented, structured interactive assignments as opposed to a passive learning experience.

One of the biggest contributions of this study is that interaction-based mood recognition is considered an essential adaptive element. Rather than an intrusive biometric or sensor based emotion detection, the proposed system will estimate the mood of the learner based on behavioral interaction cues in the form of latency of response, inactivity period, gesture repetitive behavior and pattern of task abandonment. This will guarantee compliance of the ethics and preservation of privacy and yet allow adaptation to occur in a practical way; this will be, emotionally responsive. The adaptive algorithm adjusts the content difficulty, pacing and the level of guidance dynamically according to the perceived mood states to allow frustration, fatigue and lack of engagement to be avoided.

The results of the experiment and performance analysis are

that the proposed system leads to an increase in the factors of interaction and engagement of the learners in comparison with the approaches of learning which are not dynamic. There is faster task initiation, less hesitation, and increased consistency in interaction between the learners during repeated sessions. Mood-sensitive adaptive interventions are essential to minimize the number of frustration outbursts, raise the task completion levels, and enhance engagement recovery. These findings indicate that gesture interaction used in conjunction with behavior and mood-sensitive adaptive intelligence is an effective approach to making the learning environment more inclusive and supportive of learning emotions.

The system facilitates continuity of learning in addition to the short-term enhancement of interaction. Students can spend more time in the classroom without it being clearly imposed on them, which shows that adaptive learning flow fits the personal capacity of students quite well. This continuity is specifically significant to the IDD children, because consistent and emotionally stable learning experiences have a direct impact on the long-term learning results. The suggested framework therefore helps not only in enhancing interactions on a short-term basis but also in coming up with a healthy and supportive learning process in the long-run.

Practically the framework proposed is modular in design and therefore it is scalable and can be deployed in the real world. The system is based on using conventional camera devices and local processing which allows it to be used in classroom environment as well as in home-based learning settings. The lightweight design allows it to be deployed in the resource constrained educational environments without affecting the data privacy and low-latency interaction. Such features render the framework to be applicable in the incorporation into the existing digital learning platform and special education programs.

In spite of its advantages, the suggested system has some limitations which can be offered as the potential topics of the research. Lighting conditions and quality of the camera can be considered as environmental effects on gesture recognition performance. Also, even though interaction-based mood recognition has some pros and cons, which are ethically and practically more advantageous, it might still fail to display all the subtleties of emotions that a learner has. The adaptive logic we have at present is based on rule-controlled decision making, which, as much as can be explained and predictable, is constraining to flexibility in highly complex or by itself diverse learning situations.

Future studies will be concerned with the development of the proposed framework further into sophisticated adaptive intelligence without violating ethical and privacy concerns. The combination of deep learning-based personalization models with the ability to adapt to new strategies with time, based on historical interaction data, is one of the promising directions subject to proper data governance and ethical consent. The models could be used to improve the accuracy of personalization and flexibility in a variety of learner profiles. Other improvements can be optional addition of emotion recognition capabilities, likefacial expression, voice recognition, with a very specific ethical limit and parental

approval. The architecture can be also applied to smart classroom environments that are based on IoT and oriented contextual variables, including lighting, noise intensity, and collaborative learning processes can be integrated into adaptive decision making.

Another significant future direction is the long-term longitudinal studies. The long-term system evaluation will give more information about the long-term learning process, emotional growth and the effect of the education in the real world. Clinical and pedagogical validation of the framework can also be carried out using such studies together with special educators and therapists.

To sum it up, the offered AI-driven adaptive learning framework and algorithm can be considered a major advance toward the idea of inclusive, ethical, and emotionally responsive education of children with Intellectual and Developmental Disabilities. This study offers a solid background of further developments in the field of AI-based assistive education and adds value to the scholarly research and the related practice in a special education setting through a combination of gesture interaction, educational game-play, behavior analysis, and mood-aware adaptation into a single system.

REFERENCES

1. P. Brusilovsky, Adaptive hypermedia, User Modeling and UserAdapted

   Interaction, 2001.

2. A. Khamparia and B. Pandey, A systematic review on adaptive learning

   systems, Education and Information Technologies, 2017.

3. A. Saleh et al., Artificial intelligence in education, IEEE Access, 2020.

4. G. Alnahdi et al., Assistive technology in special education,

   International Journal of Special Education, 2019.

5. S. Boucenna et al., Interactive technologies for autism, ACM Computing Surveys, 2014.

6. J. Mintz, Technology use in special education, Technology, Pedagogy and Education, 2013.

7. J. Wachs et al., Vision-based hand-gesture applications,

   Communications of the ACM, 2011.

8. M. Hussein et al., Gesture-based serious games, IEEE Transactions

   on Learning Technologies, 2019.

9. N. Bosch et al., Detecting learner engagement, IEEE Transactions on

   Learning Technologies, 2016.

10. S. DMello and A. Graesser, Affect-aware learning systems, IEEE

    Transactions on Affective Computing, 202.

11. X. Zhai, M. W. Chang, and P. Guo, Intelligent tutoring systems: A systematic review, IEEE Transactions on Education, vol. 63, no. 4, pp. 238246, 2020.

12. A. Kapoor, W. Burleson, and R. Picard, Automatic prediction of frustration, IEEE Transactions on Affective Computing, vol. 1, no. 1,

    pp. 417, 2010.

13. M. Rahman, A. Hassan, and S. Abdullah, Inclusive artificial intelligence in education: Opportunities and challenges, Computers Education: Artificial Intelligence, vol. 3, 2022.

14. J. A. Jacko, Humancomputer interaction for assistive technologies,

IEEE Computer, vol. 45, no. 4, pp. 2426, 2012.