Kibo : The Companion Robot

Kibo : The Companion Robot

Dr. C. Vivek

Assistant professor Department of AI&DS Panimalar engineering college, Chennai, India

P Nithesh Kumar

UG Scholar Department of AI&DS Panimalar engineering college, Chennai, India

S Mukesh

UG Scholar Department of AI&DS Panimalar engineering college, Chennai, India

S Nithin Raj

UG Scholar Department of AI&DS Panimalar engineering college, Chennai, India

Abstract – Kibo, a virtual pet robot created with a Raspberry Pi 4, is presented in this project. To create a captivating companion, Kibo combines vocal interaction, computer vision, and mood simulation. Its LED/LCD facial display creates a natural contact by displaying emotions including happiness, melancholy, curiosity, and drowsiness. With configurable wake words and commands, an integrated voice assistant facilitates seamless communication, and a mood engine modifies Kibos behavior in response to human involvement and idle time. Simple actions like tilting the head are added using servo motors that are managed by a PCA9685 driver, adding to its realistic appearance. The system, which is modular in design, integrates display, audio, vision, and mood management onto a single platform.

Index Terms – Human-Robot Interaction, Voice Assistant, Mood Simulation, Facial Expression Display, Raspberry Pi 4

1. INTRODUCTION

   Robotics has developed into fields that directly include human connection and companionship, going well beyond industrial automation and production. In addition to carrying out functional tasks, modern robots are supposed to be able to interact, communicate, and adjust to human emotions. A new class of robots intended for personal interaction, entertainment, and education has emerged as a result of this change. The virtual pet robot is one such idea that combines intellect, emotions, and interaction to function as a companion. In order to provide a more engaging and natural relationship with users, we present Kibo, a virtual pet robot that combines voice communication, emotional simulation, and digital expressions. As the primary processing unit, the Raspberry Pi 4 is at the heart of Kibo. The Raspberry Pi was selected because of its adaptability, low cost, and capacity to manage numerous input-output functions, which makes it appropriate for real- time applications. The three main themes of Kibos design are vocal communication, digital expressions for emotions, and mood simulation for behavioral adjustments. When combined, these characteristics enable Kibo to provide an experience more akin to that of a live companion, transcending its status as a straightforward programmed machine. Kibo uses a digital facial display, which can be used with a TFT LCD screen or an LED matrix, to convey emotions. The robot uses this interface to display emotions like joy,

   sorrow, curiosity, or drowsiness. In addition to making interaction more interesting, the robots capacity to display emotions strengthens the bond between the human and the machine. Servo motors are used to carry out

   Fig. 1. FLOW DIAGRAM

   basic motions like head tilting for increased realism, giving Kibo a sense of bodily expression to go along with its virtual face. Kibos voice interaction capabilities is another important feature. The robot can reply to commands, answer questions, and carry out activities thanks to a customisable voice assistant that was created using platforms like Mycroft AI, Rhasspy, or Google Assistant SDK. A more customized experience is made possible by the ability for users to define particular wake words and commands. Because Kibo can listen, process, and react in real time, this feature makes interactions with it feel participatory.

   The mechanism that simulates moods gives Kibo a person- ality. Kibo behaves differently depending on user involvement and idle time than traditional robots, which always react in the same way. For instance, regular engagement can keep Kibo happy and energetic, while prolonged inaction can make it seem drowsy or exhausted. Because of this adaptive na- ture, Kibo is less like a mechanical device and more like a live companion. Each componentvision, audio, display, and moodis designed and tested independently before being incorporated into the entire system as part of the projects modular development approach. This approach guarantees that the system will continue to be adaptable and simple to upgrade

   in the future. After being joined, the modules work together using multithreading, which enables Kibo to control speech,

   mood, and display simultaneously without interruption.

   Kibo was created in order to investigate practical and affordable methods of human-robot interaction. The cost and complexity of many of the sophisticated robots on the market limit their accessibility for experimentation and learning. Kibo uses open-source technologies and reasonably priced hardware to prove that. To sum up, Kibo is intended to serve as an interactive companion robot that can speak, express emotions, and adjust to user behavior. Its creation demonstrates how open-source frameworks and widely available technologies, like as Raspberry Pi, may be used to build intelligent and expressive robotic systems. The following chapters will further explain the background, design, implementation, testing, and future possibilities of this project.

2. RELATED WORKS

   In this paper, a low-cost robot that uses machine learning to identify human facial emotions is constructed using a Raspberry Pi and ROS. The technology uses trained models to classify emotions like happy, sadness, and rage after capturing facial expressions with a camera and processing them in real time. The authors highlight accessibility and cost while demonstrating that embedded systems are capable of achieving high-performance emotion detection[1].

   This work uses a Raspberry Pi and a galvanic skin response (GSR) sensor to combine physiological responses with face emotion detection. It increases the accuracy of emotion catego- rization by combining bio-signal analysis with visual emotion detection from facial features. The dual-input method makes it easier to distinguish between emotions that are similar, like surprise and terror. The promise of low-cost affective computing systems in robotics and healthcare is highlighted in the article[2].

   This study investigates the use of Convolutional Neural Net- works (CNNs) on Raspberry Pi for real-time emotion identifi- cation. The suggested model was trained on the FER2013 and CK+ datasets, is lightweight, and is designed for embedded devices. The CNN architecture consists of efficient convo- lutional, pooling, and dense layers. Live video streams are captured by a camera module, and the system instantaneously recognizes the emotions on the faces[3].

   The quadruped robot simulation e-Inu, which demonstrates emotional consciousness, is presented in this work. It uses emotion-inspired parameters that are modified by ambient conditions and user interaction to model behavior. The system simulates mood swings and behaviors by combining emotional computing and reinforcement learning. Psychological theories like the circumplex model of affect serve as the foundation for the emotional model[4].

   The authors used Natural Language Processing (NLP) and a Raspberry Pi to create an Internet of Things (IoT) voice assistant. Task automation, text-to-speech, and speech recog- nition are all done by the system independently of cloud services. It makes it possible to get information, do web searches, and operate household equipment. Th project shows

   how to incorporate open-source tools such as pyttsx3 and SpeechRecognition into a Raspberry Pi environment to create a cost-effective personal assistant[5].

   This community project explains how to use SaraKIT, a Home Assistant and Raspberry Pi extension, to create offline and hybrid voice control. The method balances privacy and ef- ficiency by combining offline speech processing with specific cloud-based improvements. Setup, voice model customization, and connection with smart home devices are all explained in the Reddit thread[6].

   The Raspberry Pi-powered robotic dog MiRo-E is intended for emotional engagement and education. It uses sound mod- ules, cameras, and sensors to replicate realistic behaviors. The robot uses LED face expressions, auditory cues, and gestures to convey emotions. MiRo-E, which is based on open- source hardware, lets students and developers experiment with robotics, AI, and machine learning[7].

   This open-source GitHub project describes how to build a Raspberry Pi 5-powered voice-activated desk pet robot. For interaction, it makes use of OLED-based face feedback, speech recognition, and natural language answers. The robot makes simple gestures, exhibits emotions, and responds to spoken orders from the user. Real-time processing and Python-based voice control are integrated into the system design[8].

   This research suggests a modular, dispersed pressure sensor- based tactile interaction system for robot dogs. It improves robots emotional and physical reactivity by giving them the ability to sense pressure, touch, and human gestures. By simulating biological skin, the tactile feedback technology enables subtle contact. It offers a scalable and affordable design when used in conjunction with a Raspberry Pi for data processing[9].

   Emo is a robot designed to be a personal companion and mimic human emotions and behavior. Emo reacts to human behavior and ambient cues by using voice synthesis, facial expression recognition, and Raspberry Pi. The robot may use auditory cues and face motions to convey joy, interest, or grief. The documentation on Hackster.io describes how to integrate AI models and configure hardware for emotion classification[10].

   This community-driven project investigates how to use a Raspberry Pi and open-source tools to create a pet robot with AI capabilities. Voice interaction, gesture recognition, mood detection, and other real-time progress aspects are shared by the builder. The robot reacts to human emotions and words in a dynamic way. Python-based control, AI model training, and hardware selection are all covered in the Reddit conversation[11].

   This GitHub project uses STM32 microcontrollers and Raspberry Pis to recognize emotions while balancing mecha- nisms. The robot employs a camera module to detect facial emotions and a gyroscope for balance. Robot movement patterns are influenced by the classification of emotions by machine learning models. For example, identifying anger results in defensive posture, whereas identifying happiness results in joyful motion[12].

   Fig. 2. ARCHITECTURAL DIAGRAM

   This study introduces a dual-processor emotional interaction robot that uses an STM32 for motion control and a Raspberry Pi for facial recognition. The robot uses animated facial displays and body actions to mimic human emotions after recognizing them from facial expressions. The system archi- tecture improves response time and efficiency by separating motor control from emotion calculation. The focus of the paper is on embedded optimization and real-time interaction[13].

   This arXiv paper describes Robot Voice, an Arduino- based robot that uses real-time voice commands to react to human speech. For responses, it incorporates basic emotional tone modulation and offline speech recognition. The goal is to create readily programmable, reasonably priced robots for enthusiasts and education. The authors analyze various microphones and processing units for accurate voice identi- fication[14].

   This study describes a visual and auditory recognition robot for home security and defense that is built on a Raspberry Pi. Voice commands are used to control the robot, while image processing is used to identify intruders. Alerts and remote monitoring are available via an Android app. For increased safety, it incorporates motion tracking and AI-powered facial recognition[15].

3. IMPLEMENTATION

   The implementation of Kibo, a virtual pet robot, was carried out through a modular and systematic approach to ensure reliability, scalability, and ease of integration. The process involved hardware assembly, software configuration, develop- ment of individual functional modules, and final integration into a single interactive system.

   1. Hardware Implementation

      The hardware foundation of Kibo is built on the Raspberry Pi 4, which serves as the central processing unit due to its high performance and support for multiple peripheral interfaces. A camera module was connected through the CSI interface for computer vision tasks, while audio communication was achieved using a USB microphone and speaker. An LED

      matrix or TFT LCD was employed for the robots face,

      enabling dynamic emotional expressions. For physical move- ments, servo motors were connected through the PCA9685 driver board, which allows precise multi-channel control using the I2C protocol. The motors were powered by an independent regulated 5V supply, ensuring stable operation. A common ground connection between all components was maintained to avoid electrical inconsistencies.

   2. Software Display

      The Raspberry Pi was configured with Raspberry Pi OS (64-bit). Required libraries and dependencies were installed, including OpenCV for vision processing, Pygame for facial animations, and Adafruit CircuitPython libraries for servo motor control. For audio interaction, different voice assistant frameworks such as Mycroft, Rhasspy, and Google Assistant SDK were tested and integrated. Each function of the robot was designed as a separate Python module:

      • face display.py for emotional expressions,
      • voice module.py for speech recognition and responses,
      • mood engine.py for mood simulation,
      • movement control.py for servo actions,
      • main.py as the central integration script.

      This modular software design ensured easier debugging and allowed each feature to be tested independently before system- wide integration.

   3. Emotional Display

      The robots digital face was implemented to display multiple states, including happiness, sadness, curiosity, and sleepiness. On an LCD, Pygame was used to animate eyes and mouth, whereas the rpi-rgb-led-matrix library was employed for LED- based displays. The facial module dynamically updated ex- pressions based on input from the mood engine and user interactions, providing visual cues that made communication more engaging.

   4. Voice interaction

      The speech interface enabled natural communication be- tween users and Kibo. The assistant continuously listened for a wake word, after which it could process commands such as displaying a particular emotion or initiating a small movement. Customizable commands were mapped to corresponding robot behaviors. The choice of using offline assistants like Mycroft or Rhasspy ensured functionality without internet dependency, while online integration with Google Assistant provided ex- tended conversational capabilities.

   5. Mood Simulation

      A lightweight mood simulation algorithm was developed to provide adaptive responses. The system maintained parameters such as happiness and energy level, which varied based on user interaction frequency and idle duration. For example, consistent engagement increasedhappiness, while inactivity reduced energy, leading to a sleepy expression. This approach gave Kibo a sense of personality and prevented repetitive or mechanical responses.

   6. Motion Control

      Servo motors were programmed to perform subtle, human- like gestures such as head tilts or nods. Movements were synchronized with facial expressions to enhance realism. For instance, when Kibo displayed happiness, a slight head tilt was introduced, while a neutral or sleepy expression was represented with slower movements.

   7. System Integration

      All modules were combined in the main.py script using multithreading, allowing simultaneous execution of tasks. This ensured that voice commands, mood simulation, facial display, and motor movements operated in parallel without interfering with each other. Priority rules were also definedfor example, direct user commands were set to override idle mood expres- sionsso that real-time responsiveness was maintained.

   8. Testing and Optimization

   Extensive testing was conducted under different environ- mental conditions. The voice assistant was tuned for improved recognition of different accents, the mood system was cal- ibrated for realistic time intervals, and servo motors were adjusted for smoother movements. The facial display was optimized for clear visibility and quick transitions. These refinements contributed to stable system performance and improved user experience.

4. RESULT AND DISCUSSION

   The implementation of Kibo was evaluated by testing each module individually and then in a fully integrated environment. The facial display system successfully represented multiple emotions, including happiness, sadness, curiosity, and sleepi- ness, with smooth transitions between expressions. On both the LED matrix and TFT LCD, the expressions were clear and easily distinguishable, providing effective non-verbal commu- nication. The voice interaction module demonstrated reliable performance during testing. Wake words were consistently recognized, and predefined commands such as displaying a specific emotion or initiating servo movements were executed with minimal delay. Offline frameworks like Mycroft and Rhasspy showed faster response times, while the Google As- sistant SDK provided more advanced conversational features when connected to the internet. Accuracy rates remained high under normal indoor conditions, though background noise slightly reduced performance.

   1. Evaluation Methodology

      Kibos performance was evaluated using both fully inte- grated system evaluation and individual module testing. A variety of environmental factors, including background noise and lighting intensity, were used to evaluate each of the basic subsystems: servo motor control, voice interaction, mood simulation, and facial display. The modules were integrated utilizing a multithreaded control framework on a Raspberry Pi 4 to guarantee seamless, concurrent execution after their standalone dependability was confirmed. In order to replicate

      Fig. 3. ACCURACY COMPARISON

      common user engagement scenarios, the testing placed a strong emphasis on response accuracy, synchronization, and system stability during real-time interactions. Kibos capacity to behave consistently even when performing simultaneous mechanical, visual, and auditory tasks was validated by the integrated testing.

   2. Facial Display and Emotional Expression

      Happiness, melancholy, curiosity, and drowsiness were among the various emotional states that Kibos facial display module was able to successfully transmit onto a TFT LCD screen and an LED matrix. Emotional shifts were smooth, pre- serving organic coherence throughout interaction scenes. The expressions that were on exhibit were understandable, distinct, and accurately conveyed Kibos mood at any given moment, allowing for intuitive nonverbal communication. Emotional reactions were made more realistic by the use of controlled blinking animations and gradual brightness gradients. The de- signs ability to replicate human-like expressions and promote empathy during user contact was validated by user input, which showed that emotional states were simple to read.

   3. Voice Interaction and Command Recognition

      During testing, the voice interaction module performed consistently. Consistent recognition of wake words like Hey Kibo and minimal latency execution of predefined commands, such as requesting the display of a certain mood, a servo motion, or the transition to sleep mode, were observed. For testing, the Google Assistant SDK and offline frameworks (Mycroft, Rhasspy) were combined. While Google Assistant offered a richer conversational experience with natural lan- guage comprehension, offline alternatives offered faster reac- tion times because of local processing. The overall accuracy of the commands was more than 90% in typical indoor settings. High ambient noise, however, marginally decreased recognition efficiency, suggesting that directional microphones

      Fig. 4. MOOD SIMULATION

      or noise-filtering algorithms could be used to improve the system.

   4. Mood Simulation and Behavioral Adaptivity

      Kibos realism was greatly improved by the mood sim- ulation engine. Internal emotion parameters were updated regularly according to the frequency and length of user in- teractions. While inaction gradually reduced energy, resulting in a drowsy or neutral appearance, prolonged engagement raised happiness and interest levels, resulting in more active emotions. Unlike static-response robots, Kibo has a feeling of personality thanks to its adaptive emotional modeling. The dynamic simulation produced a responsive and changing character by effectively bridging the gap between robotic behavior and emotional experience. Such flexibility suggests the possibility of more emotional human-robot connection, especially in situations involving companionship or education.

   5. System Integration and Overall Performance

   Using Raspberry Pi multithreading, Kibos system func- tioned flawlessly once all the modules were fully integrated. Without any discernible lag, voice recognition, mood updates, display control, and servo actuation were all carried out simultaneously. Expressive realism was improved by the servo motor subsystem, which was managed by the PCA9685 driver and provided fluid, calibrated head movements in rhythm with emotional states. The system remained stable in a variety of lighting and acoustic settings while operating continuously. Overall system responsiveness and emotional consistency were unaffected, while there was a slight deterioration in recognition due to excessive noise or dim lighting.

5. CONCLUSION

This work presented the design and development of Kibo, a virtual pet robot built on Raspberry Pi 4, integrating voice interaction, facial expression display, mood simulation, and basic motion control. The system was implemented using a modular approach, where each feature was developed in- dependently and later combined into a unified framework through multithreading. Testing confirmed that Kibo was able

to respond accurately to voice commands, display a range of emotions, and adapt its behavior based on interaction patterns, giving it a more engaging and lifelike presence.

The project demonstrates that interactive companion robots can be developed with low-cost hardware and open-source software, making them accessible for educational and re- search purposes. By combining emotional intelligence with simple movements and expressive displays, Kibo provides a meaningful step toward improving human-robot interaction. The outcomes highlight the potential of such systems not only for entertainment but also for applications in learning environments, personal assistance, nd emotional support.

REFERENCES

1. J. Mart´nez and J. Vega, ROS System Facial Emotion Detection Using Machine Learning for a Low-Cost Robot Based on Raspberry Pi,

   Electronics, vol. 12, no. 1, article 90, Dec 2022. MDPI

2. From face detection to emotion recognition on the framework of Rasp- berry pi and galvanic skin response sensor for visual and physiological biosignals, Journal of Electrical Systems and Information Technology, vol. 10, article 24, Apr 2023. SpringerLink
3. Real-Time Emotion Recognition Using Convolutional Neural Network: A Raspberry Pi Architecture Approach, in MICAI 2023 International Workshops, 2023. ACM Digital Library
4. A. Chakravarty et al., e-Inu: Simulating A Quadruped Robot With

   Emotional Sentience, arXiv, Jan 2023. arXiv

5. P. Rajakumar et al., IoT Based Voice Assistant using Raspberry Pi and Natural Language Processing, in ICPECTS 2022, Dec 2022.

   ResearchGate

6. Offline & Hybrid Voice Control for Home Assistant with SaraKIT,

   Reddit project summary, May 2024. Reddit+1.

7. Meet your new robotic best friend: the MiRo-E dog Raspberry Pi,

   Raspberry Pi news, date unspecified (post-2022). Raspberry Pi

8. Desk Pet Robot driven by Raspberry Pi 5, ARK GitHub project, real- time voice-controlled pet with OLED feedback. GitHub
9. L. Zhan et al., Enable Natural Tactile Interaction for Robot Dog based on Large-format Distributed Flexible Pressure Sensors, arXiv, Mar 2023. arXiv
10. Emo Your personal companion robot, Hackster.io project, published May 2023. Hackster
11. Building an AI Pet Robot, Reddit build-in-public project, Jan 2025.

    Reddit

12. Emotion-Recognition-Balance-Robot-Project, GitHub, Raspberry Pi

    and STM32 balance and emotion robot. GitHub

13. Y. Zhou et al., An Emotional Interaction Robot with Facial Expression

    Recognition Realized on Raspberry Pi and STM32, in IASC 2021,

    Springer, 2022. SpringerLink

14. V. Teeda et al., Robot voice a voice controlled robot using arduino,

    arXiv, Feb 2024. arXiv

15. P. Tripathi et al., Raspberry Pi Controlled Home Security and Defense Robot with Visual and Audio Recognition with Android Application Control, in ICCDC 2021, Springer, 2022. SpringerLink