IntelShare: A Survey and Proposal of Interactive Machine Learning Systems for Human-AI Collaboration

IntelShare: A Survey and Proposal of Interactive Machine Learning Systems for Human-AI Collaboration

Aman Mohamed K

Dept. of Computer Science & Engineering Federal Institute of Science and Technology (FISAT), Angamaly, Kerala, India

Azim Palakkal

Dept. of Computer Science & Engineering Federal Institute of Science and Technology (FISAT), Angamaly, Kerala, India

Chethna Joy

Dept. of Computer Science & Engineering Federal Institute of Science and Technology (FISAT), Angamaly, Kerala, India

Anirudh Anilkumar

Dept. of Computer Science & Engineering Federal Institute of Science and Technology (FISAT), Angamaly, Kerala, India

Cyril Pius

Dept. of Computer Science & Engineering Federal Institute of Science and Technology (FISAT) Angamaly, Kerala, India

Abstract – Interactive Machine Learning (IML) aims to de- mocratize artificial intelligence by involving humans directly in the training loop through feedback, correction, and iterative refinement. However, many existing IML systems either require specialized hardware, focus on a single modality, or are tai- lored to narrow domains such as robotics or manufacturing, which limits their accessibility for non-technical users. This survey reviews representative work in IML, including tangible interfaces, human-in-the-loop frameworks, interactive imitation learning, audio-based IML, and edge-AI robotics. We analyze their architectures, feedback mechanisms, and deployment strate- gies, and summarize their strengths and limitations. Building on these insights, we present IntelShare, a humanAI collaboration framework that combines multimodal edge sensing, real-time feedback-driven adaptation, explainability tools, and cloud-based model sharing. IntelShare targets non-expert users and small organizations, enabling them to train, personalize, and share lightweight models using intuitive camera and microphone inputs. The survey highlights open challenges in scalability, feedback quality, and trust, and outlines future research directions for accessible and sustainable interactive AI.

Index Terms – Interactive Machine Learning, Human-in-the- Loop, Edge AI, Explainable AI, Model Sharing, Human Computer Interaction.

1. INTRODUCTION

   Artificial Intelligence (AI) and deep learning have trans- formed domains such as computer vision, speech processing, and robotics. Yet building and adapting machine learning (ML) models typically requires programming expertise, curated datasets, and substantial compute resources. For educators, small businesses, and individual learners, these requirements create significant barriers to entry.Several studies emphasize that data collection and annotation are among the most ex-

   pensive stages in ML development, particularly for small- scale users and educational settings [25], [26]. Interactive approaches have therefore gained attention as a means to reduce development time and improve accessibility [27], [28]. Interactive Machine Learning (IML) seeks to lower these barriers by embedding the human in the learning loop. Instead of preparing a fully labelled dataset before training, users iteratively provide examples and corrections while the model updates in near real time. This process can improve usability, reduce data requirements, and make model behavior more

   transparent.

   Despite promising results, existing IML systems often suffer from one or more of the following limitations:

   • Support for a single modality (e.g., only images or only audio);
   • Dependency on specialized or expensive hardware;
   • Focus on narrow, domain-specific tasks (e.g., a specific robot platform);
   • Limited support for model export, reuse, and sharing across applications;
   • Weak or absent explainability tools for non- technical users.

   In this survey, we:

   1. Review key IML systems from tangible interfaces, human-in-the-loop frameworks, robotics, and web- based ML tools;
   2. Provide a comparative analysis of methodologies, feed- back mechanisms, and evaluation metrics;
   3. Identify gaps that motivate a more unified and accessible framework;
   4. Present IntelShare, an interactive ML architecture de- signed for multimodal inputs, real-time adaptation, and cloud-based model sharing;
   5. Discuss open challenges and research directions for future work.

   The rest of the paper is organized as follows. Section II in- troduces background on human-in-the-loop learning and IML. Section III surveys representative IML systems. Section IV provides a comparative analysis and discusses limitations. Section V presents the IntelShare framework. Section VI dis- cusses open challenges and future directions, and Section VII concludes.

2. BACKGROUND AND MOTIVATION
   1. Human-in-the-Loop Machine Learning

      Human-in-the-loop (HITL) ML refers to learning paradigms where humans contribute labels, corrections, or higher-level guidance at different stages of the training pipeline. Typical interaction patterns include:

      • Interactive correction: Users correct misclassifications as the system runs, triggering model updates.
      • Active learning: The system selectively queries the user on uncertain samples.
      • Machine teaching: Experts design teaching strategies, curricula, or constraints for the learner.

        Recent surveys emphasize adaptive interaction strategies in which the level of human involvement varies based on model confidence and task complexity [29], [30]. The importance of human guidance in improving system usability and reliability has been highlighted in several studies [3], [8].

   2. Interactive Machine Learning Systems

      IML systems integrate HITL concepts into end-to-end ap- plications, often featuring:

      • A user interface (tangible or graphical) for feedback;
      • An ML engine capable of fast or incremental updates; and
      • Visualization or explainability components that help users interpret predictions and their impact.

        Visualization and explanation tools are essential for helping users understand model behavior and correct errors effec- tively [15], [20]. Web-based educational tools further demon- strate the role of intuitive interfaces in promoting ML liter- acy [21].

   3. Need for Multimodal, Accessible IML

      Most existing tools assume technical users or single- modality setups. However, real-world interaction often in- volves combined visual and audio context, as well as resource- constrained edge devices. Motivated by this, we aim for an IML framework that:

      • Supports multimodal sensing (camera, microphone, con- textual sensors);
      • Runs on affordable edge hardware while synchronizing with the cloud;
      • Allows non-experts to train, correct, and share models;
      • Exposes interpretable feedback on why a prediction was made.

   Multimodal learning frameworks enable systems to exploit complementary visual and auditorycues, leading to improved robustness and accuracy [22]. Such approaches are especially valuable in resource-constrained edge environments [11].

3. SURVEY OF INTERACTIVE MACHINE LEARNING SYSTEMS

   In this section we summarize representative IML works across different interaction modalities and application do- mains.

   1. Tangible and Physical IML Interfaces

      TofuML introduces a spatio-physical interactive ML device that uses a tangible paper mat with spatial zones and sensors to capture user interactions [1]. Users place objects on the mat and provide feedback via physical manipulation while a lightweight model retrains on-device in real time.Similar tangible and physical interfaces have been shown to enhance learning engagement among novice users [1], [24].

   2. Robot Learning from Natural Interaction

      A humanoid robot framework leverages natural language instructions, user corrections, and large language models (LLMs) to incrementally learn robot behaviours from interac- tion [2]. User feedback is stored and reused for similar future tasks.Interactive imitation and incremental learning techniques have been widely studied for improving robot adaptability in dynamic environments [2], [4].

   3. Human-in-the-Loop ML Surveys

      A survey on human-in-the-loop ML classifies systems by feedback type, user control, and application domain, and identifies best practices for latency, usability, and feedback frequency [3]. Another survey on interactive imitation learning in robotics focuses on combining demonstrations with online corrections to improve robustness [4].

   4. Human-Interactive Robot Learning in Manufacturing

      Human-interactive robot learning frameworks for smart manufacturing use sensor-rich robots whose policies are up- dated based on operator corrections, targeting usability and efficiency in industrial settings [5].Human-centered learning systems are increasingly adopted in industrial automation to improve safety and productivity [5].

   5. Web-Based IML: Teachable Machine

      Teachable Machine is a browser-based tool that lets users train classifiers on webcam or microphone data using transfer learning in TensorFlow.js [6]. It is widely used in education due to its simplicity and zero-install interface.Browser-based tools such as Teachable Machine demonstrate how transfer learning can lower entry barriers for non-experts [6], [21].

   6. Foundational Interactive ML Frameworks

      Foundational work on IML introduces real-time feedback loops where users correct classifier predictions and the system updates immediately [7]. Complementary work studies the role of users in interactive ML and introduces confidence-based querying and adaptive interfaces [8].

   7. Audio-Centric IML and Visualization

      Audio-based IML systems rely on CNN-based spectrogram classifiers with interactive visualization of audio features and model evolution to improve user understanding and feedback quality [9].Interactive audio recognition systems benefit from visualization-based feedback that improves labeling accuracy and user understanding [9], [15].

   8. Edge AI for Service Robots

      Edge AI frameworks deploy models on devices such as Raspberry Pi or Jetson Nano using TensorFlow Lite and ROS, enabling low-latency inference and online adaptation with optional cloud synchronization [10].Edge-based deployment enables low-latency learning and inference while reducing dependence on centralized cloud infrastructure [10], [11].

   9. Related Topics: Edge ML, XAI, and Model Sharing

   Recent work on on-device and edge ML focuses on effi- cient architectures and compression techniques for resource- constrained devices [11], [12]. Explainable AI (XAI) ap- proaches such as saliency maps, feature attribution, and concept-based explanations aim to make model decisions more transparent to non-experts [13], [14]. Interactive visualization tools connect XAI with human feedback to support debugging and iterative improvement [15].

   Federated and collaborative learning approaches explore how multiple users or devices can train shared models without centralizing raw data, which is relevant to collaborative model sharing in IntelShare [16], [17]. Few-shot and incremental learning techniques enable fast adaptation to new classes using limited labelled examples, which is crucial for practical interactive systems [18], [19].

4. COMPARATIVE ANALYSIS AND LIMITATIONS

   Table I summarizes key properties of representative systems. From this comparison and broader literature, several limi-

   tations emerge:

   • Modality gap: Systems are often limited to a single modality.
   • Hardware coupling: Many frameworks are tied to spe- cific robot platforms or custom devices.
   • Export and sharing: Robust model export, versioning, and sharing across applications are rarely addressed.
   • Explainability: Comprehensive, user-friendly XAI tools are uncommon.
   • Scalability and collaboration: Collaborative learning between users via cloud backends is rarely explored.

     These gaps motivate the IntelShare framework described below. Similar limitations regarding scalability, explainability,

     TABLE I

     Comparison of Representative IML Systems

     Work	Modality / Domain	Hardware	Export / Ex- plain
     TofuML [1]	Tangible vision, education	Custom de- vice	Limited export, basic viz
     Robot incr. [2]	Robot behaviour	Humanoid robot	Behaviour memory only
     Teachable Machine [6]	Webcam / mic, general	Browser, we- bcam	Model export; limited XAI
     Audio IML [9]	Sound recog- nition	Desktop / web	Visualization- based XAI
     Edge AI [10]	Service robots	Pi / Jetson	Cloud sync, task-specific

     and reuse have been reported in recent surveys on interactive and edge-based learning systems [3], [11].

5. INTELSHARE: A UNIFIED FRAMEWORK FOR INTERACTIVE ML
   1. System Overview

      IntelShare is a humanAI collaboration and model sharing platform designed to address the above limitations. The end- to-end pipeline is:

      Edge Sensing Data Preprocessing Interactive ML Engine Action Execution Model Export, Sync & Sharing.

      Edge devices (Raspberry Pi, Jetson Nano, smartphones) capture images, audio, and optional contextual data (IMU, GPS). The preprocessing modules perform normalization, noise reduction, and temporal segmentation. A lightweight interactive ML engine performs incremental updates without interrupting real-time inference. A companion React / React Native application exposes feedback controls and visualiza- tions, while Supabase or AWS-based services manage cloud synchronization, authentication, and shared model reposito- ries.The proposed architecture is aligned with modular and reusable ML development practices emphasized in recent visualization and toolkit-based studies [20]. Incremental and few-shot learning techniques further support rapid adaptation with limited training samples [18], [19].

   2. Functional Components
      1. Data Acquisition ad Preprocessing: This module:
         • Captures multimodal sensor data;
         • Supports manual and semi-automatic labeling via the companion app;
         • Applies normalization, augmentation, and feature extrac- tion.
      2. Model Training and Reinforcement: This module:
         • Uses compact CNN backbones (e.g., MobileNet-style);
         • Performs incremental learning with feedback- weighted loss functions;
         • Syncs edge and cloud training where needed.
      3. Inference and Explainability: This module:
         • Runs low-latency inference on edge devices;
         • Generates saliency maps, confidence plots, and attention heatmaps;
         • Logs decisions and feedback for debugging and evalua- tion.
      4. Companion Application: The companion app:
         • Provides buttons for Correct/Wrong feedback;
         • Displays predictions, confidence values, and explana- tions;
         • Visualizes learning curves and version history.
      5. Cloud Integration and Sharing: The cloud backend:
         • Stores model versions and associated metadata;
         • Enables secure upload/download and sharing of models;
         • Can aggregate anonymized feedback for community mod- els.

           Collaborative and federated learning approaches enable secure and scalable model sharing without exposing raw data [16], [17], [23].

   3. Interactive Workflow

      The IntelShare workflow can be summarized as follows:

      1. Data collection and labeling using a camera/microphone.
      2. Initial training of a classifier on top of pre-trained embeddings.
      3. Real-time inference during use.
      4. User feedback on predictions, especially errors.
      5. Incremental update of the classifier/head using new samples.
      6. Packaging and exporting stable models with metadata to the cloud.
6. CHALLENGES AND FUTURE DIRECTIONS

   Despite significant progress in interactive machine learning, several challenges remain that limit widespread adoption. Issues such as noisy user feedback, scalability across users and devices, limited explainability for non-expert users, and resource constraints on edge hardware continue to affect system performance and reliability. Addressing these chal- lenges opens important future research directions, including robust feedback modeling, federated learning, user-centered explainability, and energy-efficient model adaptation. Recent research highlights the need for reliable feedback aggregation, robust optimization, and transparent explanations in interactive systems [13][15].

   1. Feedback Quality and Noise

      Noisy or inconsistent user feedback can degrade model quality. Future work includes confidence-based filtering, in- terface guidance, and robust optimization.

   2. Scalability and Federated Learning

      Scaling to many users and devices motivates federated and decentralized learning approaches that allow shared improve- ment while preserving privacy [16], [17].

   3. Trust, Explainability, and Ethics

      Explainability must be understandable to non-experts, and systems must address issues of bias, accountability, and re- sponsible use of trained models [13], [14].

   4. Energy and Resource Constraints

   Edge deployment requires models and update strategies that balance accuracy, latency, and energy consumption, e.g., through quantization and pruning [11], [12].Model compres- sion and lightweight architectures are commonly used to address resource limitations on embedded and mobile plat- forms [11], [12].

7. CONCLUSION

We surveyed representative interactive ML systems that cover tangible interfaces, robotics, web-based ML tools, audio- centric frameworks, and edge robotics. We analyze their in- teraction mechanisms, deployment strategies, and limitations in terms of modality, hardware coupling, explainability, and model sharing.

To address these limitations, we presented IntelShare, a multimodal, edge-centric, and explainable IML framework with integrated model export and cloud-based sharing. In- telShare aims to make AI more accessible for educators, small businesses, and individual learners. Future work includes real- world deployments, federated extensions, and deeper user studies on usability and learning impact.

ACKNOWLEDGMENT

The authors thank the faculty and staff of the Department of Computer Science and Engineering, FISAT, for their guidance and support during the development of IntelShare.

REFERENCES

1. A. Campero, E. Kiciman, A. Kapoor, and B. Hartmann, TofuML: A spatio-physical interactive machine learning device for novices, in Proc. ACM UIST, 2021.
2. T. Ishikawa, K. Okada, and M. Inaba, Incremental learning of humanoid robot behavior from natural interaction and large language models, in Proc. IEEE/RSJ IROS, 2024.
3. S. Amershi, M. Cakmak, W. Bradley, and A. Kapoor, Human-in- the- loop machine learning: A state of the art, Knowledge-Based Systems, 2022.
4. L. Feng, J. Wang, B. Li, and S. Song, Interactive imitation learning in robotics: A survey, IEEE Trans. Robotics, 2022.
5. Y. Wang, H. Liu, and X. Chen, Human-interactive robot learning for smart manufacturing: A human-centric perspective, Robotics and Computer-Integrated Manufacturing, 2023.
6. N. Jonas, A. Jacoby, P. Papoutsakis, and D. Ha, Teachable machine: Approachable web-based tool for exploring machine learning classifica- tion, Google AI Research, 2020.
7. H. Lieberman, F. Paterno`, and V. Wulf, Interactive machine learning, in Proc. CHI Workshop, 2003.
8. S. Amershi, M. Cakmak, W. Knox, and T. Kulesza, Power to the people: The role of humans in interactive machine learning, AI Magazine, 2014.
9. J. Fails and D. Olsen, Investigating audio data visualization for inter- active sound recognition, in Proc. IUI, 2020.
10. M. Chen, Y. Hao, L. Hu, M. S. Hossain, and A. Ghoneim, Edge AI for service robots: Real-time learning and adaptation in dynamic environments, IEEE Internet of Things Journal, 2023.
11. Y. Kang, J. Lin, and Z. Wang, A survey on on-device and edge ma- chine learning for mobile and IoT applications, IEEE Communications Surveys & Tutorials, 2020.
12. Y. Cheng, D. Wang, P. Zhou, and T. Zhang, A survey of model compression and acceleration for deep neural networks, IEEE Signal Processing Magazine, 2018.
13. A. Adadi and M. Berrada, Peeking inside the black-box: A survey on explainable artificial intelligence, IEEE Access, 2018.
14. W. Samek, T. Wiegand, and K. Mu¨ller, Explainable artificial in- telligence: Understanding, visualizing and interpreting deep learning models, IEEE Signal Processing Magazine, 2017.
15. T. Kulesza, M. Burnett, W.-K. Wong, and S. Stumpf, Principles of explanatory debugging: Interactivemachine learning for explainability, in Proc. IUI, 2015.
16. Q. Yang, Y. Liu, T. Chen, and Y. Tong, Federated learning: Concepts, applications, and challenges, ACM Trans. Intelligent Systems and Technology, 2019.
17. S. Savazzi, S. Kianoush, and M. Nicoli, Opportunities of federated learning in connected, smart factories: Privacy and efficiency, IEEE Communications Magazine, 2021.
18. Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, Generalizing from a few examples: A survey on few-shot learning, ACM Computing Surveys, 2020.
19. G. M. D. Cavalcanti, T. M. Nunes, and R. B. Araujo, Class- incremental learning: A survey, IEEE Trans. Neural Networks and Learning Sys- tems, 2022.
20. F. Hohman, M. Kahng, R. Pienta, and D. H. Chau, Combining machine learning and visualization for interactive model building, in Proc. IEEE VAST, 2019.
21. M. W. Johnson and C. J. Beck, Web-based interactive machine learning tools for education and computational thinking, IEEE Transactions on Education, 2021.
22. S. Arif, H. Zhang, and F. Yin, Multimodal interactive learning with audio-visual inputs on resource-constrained devices, IEEE Internet of Things Journal, 2022.
23. J. Vanschoren, Meta-learning platform: Reusable machine learning models and pipelines, ACM SIGKDD Explorations, 2019.
24. A. Holzinger, G. Langs, H. Denk, K. Zatloukal, and H. Mu¨ller, Hu- mancomputer interaction for interactive machine learning systems, Communications of the ACM, 2019.
25. J. Howe, The Rising Cost of Data Annotation, IEEE Data Eng. Bull., 2020.
26. K. Lee, Barriers to Machine Learning Education, IEEE Trans. Educ., 2021.
27. S. Amershi et al., Interactive Machine Learning: An Overview, ACM CSUR, 2014.
28. A. Gupta and R. Singh, A Survey on Data Labeling, ACM Comput. Surv., 2022.
29. Y. Liu and M. Zhou, Adaptive Human-in-the-Loop Learning, Proc. AAAI, 2020.
30. P. Donmez, J. Carbonell, and J. Schneider, Efficiently Learning the Accuracy of Labels, Proc. KDD, 2009.