FACEFIT AI: An AI-Powered Skincare and Styling Assistant

FACEFIT AI: An AI-Powered Skincare and Styling Assistant

M. Varun Reddy , Ms. Sita Sowjanya Prakhya , K. Sai Charan

Department of Information Technology, MVSR Engineering College, Nadergul, Hyderabad

Abstract FaceFit AI is an AI-powered personal styling and skincare recommendation platform designed to provide highly personalized fashion and wellness guidance based on individual user features. The system utilizes computer vision techniques such as MediaPipe and YOLOv8 to analyze facial attributes including skin tone, face shape, and skin conditions. Based on this analysis, the platform generates customized outfit recommendations, skincare routines, and style guidance using large language models and retrieval-augmented generation (RAG).

A Digital Closet feature allows users to upload clothing images, enabling AI-driven mix-and-match outfit combinations tailored to event type, personal style, and context. The system also incorporates features such as virtual try-on using real-time computer vision, event-based outfit planning, outfit photo analysis, and an intelligent AI stylist chatbot for interactive assistance. Additionally, it includes product recommendation, budget filtering, and notification-based scheduling to enhance user experience.

The platform is implemented using a Flask-based backend, React frontend, and MongoDB database, ensuring scalability and efficient data management. Unlike existing systems that focus on isolated functionalities, FaceFit AI provides a unified and intelligent solution integrating styling, skincare, and AI-driven personalization into a single platform.

Keywords Artificial Intelligence, Personalized Styling, Skincare Recommendation, Computer Vision, Virtual Try-On, Face Analysis, Digital Closet, RAG, Chatbot, MongoDB

1. INTRODUCTION

   The global fashion and skincare industry is undergoing rapid transformation with the integration of Artificial Intelligence (AI) and machine learning technologies. Despite the growth of e-commerce and digital platforms, users continue to face several challenges, including selecting outfits that match their personal features, maintaining consistent skincare routines, and accessing reliable styling guidance. Most existing applications address these problems individually, resulting in fragmented solutions that fail to provide a complete and personalized user experience.

   FaceFit AI is proposed as a comprehensive AI-powered personal styling and skincare platform that addresses these limitations by integrating multiple intelligent features into a single system. The platform utilizes computer vision techniques such as MediaPipe and YOLOv8 to analyze facial attributes, including skin tone, face shape, and skin conditions. Based on this analysis, the system generates personalized outfit recommendations, skincare routines, and style guidance using large language models and retrieval-augmented generation (RAG).

   In addition to analysis and recommendations, FaceFit AI includes advanced functionalities such as a Digital Closet for wardrobe management, AI-driven mix-and-match outfit generation, virtual try-on using real-time computer vision, event-based outfit planning, and an intelligent AI stylist chatbot for interactive assistance. The platform also incorporates features such as product recommendations, budget filtering, notification-based outfit scheduling, and skin progress tracking, providing a complete and practical solution for users.

   The motivation for developing FaceFit AI arises from key observations: (1) users lack personalized and accessible guidance for fashion and skincare decisions; (2) existing skincare applications provide limited diagnostic insights without actionable recommendations; (3) virtual try-on

   solutions are often incomplete and not tailored to individual user characteristics; and (4) there is no unified platform that combines AI-based styling, skincare intelligence, and real-time user interaction.

   This paper presents the design and development of FaceFit AI, including its system architecture, key features, methodology, and implementation details. The remainder of the paper is organized as follows: Section II reviews related work; Section III defines the problem statement; Section IV describes the proposed system; Section V explains the methodology; Section VI presents datasets and tools used; Section VII discusses expected results; and Section VIII concludes the paper.

2. RELATED WORK

   A significant body of research has explored the application of Artificial Intelligence in skincare analysis and fashion recommendation systems. However, most existing solutions focus on limited functionalities, and no single system integrates all the capabilities provided by FaceFit AI into a unified platform.

   1. AI-Based Skincare and Cosmetics Recommendation

      Kavyashree et al. [1] proposed an AI-based cosmetics suggestion system using Convolutional Neural Networks (CNNs) to classify skin types from image data. The system was trained on a large dataset and recommended skincare products based on detected skin type. While effective in classification, it is limited to skincare recommendations and does not integrate fashion or virtual try-on features.

      Wati et al. [2] implemented the C4.5 Decision Tree algorithm for facial mask product selection, achieving an accuracy of 84.09%. The model considers attributes such as skin type, age, gender, and lifestyle. However, the system lacks advanced AI-based personalization and is restricted to a specific skincare category.

   2. Privacy-Preserving and IoT-Based Skin Monitoring

      Sharma and Bhadula [3] proposed a secure federated learning framework for IoT-enabled skincare systems. This approach ensures data privacy while generating personalized recommendations. However, it does not include fashion recommendation or styling capabilities.

      Kaneko and Matsui [6] developed a continuous skin monitoring system using bioelectrical sensors to track skin conditions such as moisture and oil levels. Their study showed improvements in skin health over time. However, the system depends on hardware sensors and does not utilize computer vision or integrated AI-based styling.

   3. AI-Based Fashion Recommendation Systems

      Deekshitha et al. [4] proposed an AI-driven fashion recommendation system combining content-based filtering, collaborative filtering, and deep learning techniques. The system achieved high recommendation accuracy and explored generative methods for outfit visualization. However, it does not integrate skincare analysis or personalized recommendations based on facial attributes.

   4. Advanced AI Techniques for Skin Analysis

      Gade and Mahaveerakannan [7] explored Vision Transformers (ViT) along with modern healthcare technologies to improve skin condition detection. Their approach improved diagnostic accuracy and accessibility. However, it focuses only on medical analysis and does not include fashion, styling, or user interaction features.

   5. Comparison with Proposed System

   The above systems address individual aspects such as skincare recommendation, monitoring, or fashion suggestion. In contrast, FaceFit AI integrates multiple features including facial analysis, digital wardrobe management, AI-based outfit recommendations, skincare intelligence, chatbot interaction, and real-time assistance into a single platform.

   td bgcolor=”#DCDDDE”>

   Existing Systems

   Feature	FaceFit AI
   Face & Skin Analysis	Partial / Limited	Advanced (MediaPipe + YOLOv8)
   Virtual Try-On	Limited / Single-feature	Real-time accessory try-on
   Digital Wardrobe	Not available	AI-powered closet + mix-match
   Skin Tracking	Sensor/manual	Image-based AI tracking
   Outfit Recommendation	Basic filtering	AI + context-aware system

   TABLE I. COMPARISON OF EXISTING SYSTEMS VS. FACEFIT AI

   Skincare Recommendation	Product-based	RAG-based intelligent system
   AI Chatbot	Not available	LLM-based AI stylist
   Event-Based Planning	Not available	Supported
   Price Alerts	Limited	Integrated alerts
   System Integration	Fragmented	Unified platform

3. PROBLEM STATEMENT

   Users today face multiple interconnected challenges in fashion and skincare that existing digital systems fail to address in a unified manner:

   • Difficulty in selecting appropriate outfits based on individual characteristics such as skin tone, face shape, and occasion without professional guidance.

   • Lack of effective methods to monitor and track skin conditions such as acne, pigmentation, and dark circles over time with actionable recommendations.

   • Limited availability of virtual try-on systems that support multiple categories and adapt to individual user features.

   • Absence of personalized and intelligent recommendation systems that combine both fashion and skincare insights.

   • Lack of integrated platforms that provide features such as wardrobe management, AI-based styling, skincare guidance, and real-time interaction in one place.

   Existing solutions address these challenges separately, resulting in a fragmented user experience. Therefore, there is a need for a unified, AI-powered platform that combines computer vision, machine learning, and intelligent recommendation systems to deliver personalized styling and skincare assistance in a single, interactive system.

4. PROPOSED SYSTEM

   FaceFit AI is proposed as a comprehensive AI-powered platform that integrates multiple intelligent features to address both fashion and skincare needs. The system follows a user-centric approach, starting with facial analysis and extending to personalized outfit recommendations, skincare guidance, digital wardrobe management, and real-time user interaction through an AI chatbot.

   1. System Architecture Overview

      The FaceFit AI architecture is designed as a multi-layered system consisting of the following components:

      1. User Interface Layer Developed using React, providing an interactive and responsive frontend for user interaction.

      2. Backend Services Layer Built using Flask, handling API requests, business logic, and communication between components.

      3. AI and Processing Layer Includes computer vision and AI models such as MediaPipe for facial landmark detection and YOLOv8 for skin condition analysis, along with large language models for intelligent recommendations.

      4. Data Layer MongoDB is used for storing user profiles, wardrobe data, preferences, and analysis results.

      5. External Services Layer Integrates APIs for product recommendations, notifications (email/WhatsApp), and other supporting functionalities.

         AI Pipeline

         The system workflow begins with user image input through upload or camera capture. MediaPipe Face Mesh extracts facial landmarks to determine face shape, while YOLOv8 detects skin conditions such as acne and dark spots. OpenCV-based processing is used for skin tone estimation.

         The extracted features are then passed to the recommendation engine, which generates personalized outfit suggestions and skincare routines. A retrieval-augmented generation (RAG) system is used to provide accurate and context-aware skincare advice. The AI Stylist chatbot interacts with users and provides real-time guidance based on user queries and stored preferences.

   2. Key Features

   The core functionalities of FaceFit AI include:

   • AI Face and Skin Analysis

     Detects skin tone, face shape, and skin conditions using MediaPipe and YOLOv8.

   • Digital Closet (Smart Wardrobe) Allows users to upload clothing items and generates AI-based mix-and-match outfit combinations.

   • Event-Based Outfit Recommendation Suggests outfits tailored to specific occasions, user preferences, and context.

   • Virtual Try-On

     Provides real-time accessory overlay (such as glasses and jewellery) using computer vision techniques.

   • AI Stylist Chatbot

     A conversational assistant powered by large language models for personalized styling and skincare guidance.

   • Skincare Recommendation System Uses RAG-based intelligence to suggest ingredient-level skincare routines.

   • Skin Progress Tracking

     Enables users to track skin condition changes over time using image-based analysis.

   • Product Recommendation & Budget Filtering Suggests relevant fashion and skincare products based on user preferences and budget.

   • Outfit Scheduler & Notifications

     Allows users to schedule outfits and receive reminders via email or WhatsApp.

   • Outfit Photo Analyzer

   Evaluates user-uploaded outfit images and provides improvement suggestions.

   TABLE II. TECHNOLOGY STACK OF FACEFIT AI

   Layer	Technology / Tool
   Frontend	React (Vite), HTML, CSS
   Backend	Flask (Python)
   AI Models	YOLOv8, MediaPipe, OpenCV
   AI Intelligence	Groq LLaMA (LLM), RAG (LangChain + ChromaDB)
   Database	MongoDB Atlas
   APIs	Serper (Product Search), Open-Meteo (Weather), Twilio (WhatsApp), Gmail SMTP
   Dev Tools	Python, JavaScript, OpenCV, Axios

5. METHODOLOGY

   The development of FaceFit AI follows a hybrid methodology combining Agile Software Development for iterative system development and CRISP-DM (Cross-Industry Standard Process for Data Mining) for structured AI model design. This approach ensures both efficient software engineering practices and a systematic workflow for building and integrating intelligent features.

   1. Agile Development Process

      The system is developed using Agile methodology with iterative sprints, allowing continuous improvement and feature integration. The development cycle includes requirement analysis, design, implementation, testing, and deployment, along with regular feedback.

      The project is divided into the following functinal modules:

      1. User registration and face analysis

      2. Digital Closet and wardrobe management

      3. Outfit recommendation system

      4. Virtual try-on implementation

      5. Skincare recommendation and skin tracking

      6. AI chatbot and product recommendation system

      7. Notification system and final integration

         This modular approach enables efficient development, easier debugging, and continuous enhancement of features.

   2. CRISP-DM for AI Model Development

      The AI components in FaceFit AI follow the CRISP-DM methodology consisting of six phases:

      • Business Understanding:

        Identifying user requirements such as personalized outfit selection, skincare guidance, and real-time recommendations.

      • Data Understanding:

        Analysis of image data related to facial features and skin conditions to understand patterns and model requirements.

      • Data Preparation:

        Preprocessing of images including resizing, normalization, and enhancement to improve model performance.

      • Modeling:

        Implementation of computer vision models such as YOLOv8 for skin condition detection and MediaPipe for facial landmark extraction. Large Language Models (LLMs) are used for chatbot-based recommendations, along with RAG for accurate skincare guidance.

      • Evaluation:

        Performance evaluation based on detection accuracy, recommendation relevance, and user interaction quality.

      • Deployment:

      Integration of AI models into the Flask backend using REST APIs, ensuring seamless communication with the React frontend.

   3. AI Model Pipeline

   The core AI pipeline begins with user image input through camera capture or image upload. MediaPipe Face Mesh extracts facial landmarks to determine face shape. YOLOv8 is used to detect skin conditions such as acne, dark spots, and dark circles. OpenCV-based processing is applied for skin tone estimation.

   The extracted features are passed to the recommendation engine, which generates personalized outfit suggestions and skincare routines. A retrieval-augmented generation (RAG) system enhances the accuracy of skincare recommendations by retrieving relevant knowledge.

   The AI Stylist chatbot interacts with users and provides context-aware suggestions based on user preferences, stored data, and previous interactions. The entire pipeline is integrated with the backend and database to ensure real-time performance and scalability.

6. DATASETS

   FaceFit AI utilizes publicly available datasets and pre-trained models for developing and evaluating its computer vision components. These datasets support tasks such as facial analysis and skin condition detection.

   TABLE III. DATASETS USED IN FACEFIT AI

   Dataset	Source	Purpose
   CelebA	Kaggle (Jessica Li)	Face attributes, skin tone, face shape
   Acne Detection	Roboflow (Kritsakorn)	Acne detection and tracking
   DermNet	Kaggle (Shubham Goel)	Skin condition reference and classification

   1. CelebA Dataset

      The CelebA dataset [8] contains more than 200,000 facial images annotated with multiple attributes such as skin tone, facial structure, and appearance features. In FaceFit AI, this dataset is referenced for understanding facial attribute patterns such as face shape and skin tone.

      Preprocessing techniques such as resizing, normalization, and image enhancement are typically applied to improve model performance. The dataset supports the facial analysis module implemented using MediaPipe and OpenCV-based techniques.

   2. Acne Detection Dataset

      The Acne Detection dataset [9] from Roboflow contains annotated images of facial acne across different severity levels. This dataset is used as a reference for developing and validating the skin condition detection component.

      In FaceFit AI, a YOLOv8-based model is used to detect skin conditions such as acne, dark spots, and dark circles. The dataset supports accurate detection and enables features like skin tracking and personalized skincare recommendations.

   3. DermNet Skin Disease Dataset

   The DermNet dataset [10] from Kaggle includes a wide variety of dermatological images covering multiple skin conditions. This dataset is used as a knowledge reference for understanding different skin issues.

   In FaceFit AI, this dataset supports the skincare recommendation system by providing insights into various skin conditions, helping generate more accurate and meaningful suggestions through the RAG-based approach.

7. EXPECTED RESULTS AND DISCUSSION

   FaceFit AI is designed as an integrated AI-powered platform combining computer vision, machine learning, and intelligent recommendation systems. Based on the implemented models, system architecture, and testing, the following outcomes are expected.

   1. Skin Analysis Performance

      The face and skin analysis module, implemented using MediaPipe and YOLOv8, is expected to provide accurate detection of facial features such as face shape, skin tone, and skin conditions (acne, dark spots, and dark circles).

      The YOLOv8-based detection model is expected to achieve high precision in identifying skin conditions under normal lighting conditions. The system ensures reliable performance for real-time usage and supports continuous monitoring through repeated scans.

      Fig 1: Face and Skin Analysis Output

   2. Outfit Recommendation Accuracy

      The outfit recommendation system is designed to generate context-aware suggestions based on user attributes, wardrobe data, and event type. The system is expected to produce relevant and personalized outfit combinations with high user satisfaction.

      The mix-and-match engine, combined with predefined styling rules and AI-based filtering, ensures that recommended outfits maintain color harmony, occasion appropriateness, and style consistency.

      Fig 2: Outfit Recommendation

   3. Virtual Try-On Performance

      The virtual try-on module uses real-time computer vision techniques to overlay accessories on the users face. The system provides smooth and responsive visualization of items such as glasses, jewellery, and other accessories. Facial landmarks detected using MediaPipe ensure accurate positioning of accessories.

      Unlike computationally intensive generative models, this approach enables faster processing and real-time performance, making it suitable for practical deployment and enhancing user interaction

      Fig 3: Virtual Try-On

   4. Skin Tracking and Improvement Analysis

      The skin tracking feature allows users to monitor changes in skin conditions over time using periodic image uploads. The system is expected to provide meaningful insights into improvements or changes in skin health.

      Users can observe trends such as reduction in acne or dark spots, enabling better skincare decisions and consistency in routines.

      Fig 4: Skin Tracking

   5. User Engagement and System Usability

      Features such as the AI Stylist chatbot, digital wardrobe, product recommendations, and outfit scheduler are expected to significantly enhance user engaement.

      The chatbot provides real-time interaction and personalized guidance, while additional features such as budget filtering and notifications improve usability. The integrated nature of the platform is expected to result in a seamless and user-friendly experience.

      Fig 5: Login Page

   6. Outfit Scheduler and Notification System

      The outfit scheduler allows users to plan their outfits for upcoming events and receive timely reminders. Users can select an outfit, assign a date and time, and receive notifications via email or WhatsApp.

      This feature improves user convenience by helping them stay organized and prepared for events. The integration of scheduling and notification services ensures that users do not miss important occasions or outfit plans.

      Fig 6: Outfit Scheduler and Notification System

   7. Digital Closet and Wardrobe Management

      The digital closet feature allows users to upload and manage their clothing items in a centralized system. Each item is analyzed using AI to identify attributes such as category, color, and style.

      The system enables users to generate mix-and-match outfit combinations based on their existing wardrobe. It also filters outfits based on event type and user preferences, helping users efficiently utilize their clothing collection.

      This feature enhances organization, reduces decision-making time, and provides a personalized styling experience using the users own wardrobe.

      Fig 8: Outfit Planning

      I. Body Shape Analysis

      The body shape analysis feature uses computer vision techniques to analyze the users body proportions and classify them into categories such as hourglass, rectangle, pear, or apple. MediaPipe Pose is used to detect key body landmarks and measure ratios.

      Based on the detected body shape, the system provides personalized outfit recommendations, including what styles to prefer and what to avoid. This helps users choose clothing that enhances their appearance and fits their body structure.

      Fig 9: Body Shape Analysis

      J. Event Planner

      Fig 7: Digital Closet

   8. Outfit Planning and Event Management

   The outfit planning feature allows users to organize and prepare outfits for upcoming events based on date, occasion, and personal preferences. The system generates suitable outfit combinations by considering factors such as event type, weather conditions, and user style.

   Users can view multiple outfit options and select the most appropriate one for their event. The system ensures that all recommendations are context-aware and aligned with user attributes.

   This feature helps users plan ahead, reduces last-minute confusion, and ensures a well-coordinated and appropriate appearance for different occasions.

   The event planner feature assists users in preparing for upcoming events by generating complete styling plans. Users can input event details such as type, date, and preferences, and the system provides outfit suggestions along with supporting recommendations.

   The planner may also include skincare routines, grooming tips, and a checklist to ensure the user is fully prepared. This feature provides a structured approach to styling and improves overall planning efficiency.

   Fig 10: Event Planner

   K. Outfit Photo Analyzer

   The outfit photo analyzer allows users to upload images of their outfits and receive AI-based evaluation. The system analyzes aspects such as color combination, event suitability, and overall style coherence.

   It provides a rating along with specific suggestions for improvement and alternative outfit ideas. This helps users understand styling mistakes and make better fashion decisions.

   Fig 11: Outfit Photo Analyzer

8. CONCLUSION

This paper presented FaceFit AI, an AI-powered personal styling and skincare platform designed to provide a unified and intelligent solution for fashion and wellness decision-making. The system integrates computer vision techniques such as MediaPipe and YOLOv8 with modern AI approaches including large language models and retrieval-augmented generation (RAG) to deliver personalized outfit recommendations, skincare guidance, and interactive user assistance.

FaceFit AI combines multiple functionalities such as facial analysis, digital wardrobe management, AI-based outfit recommendation, virtual try-on, skin tracking, and chatbot interaction into a single platform. The system is implemented using a Flask-based backend, React frontend, and MongoDB database, ensuring scalability, real-time performance, and efficient data management.

The use of Agile development and CRISP-DM methodology enabled structured development and seamless integration of AI components. Compared to existing systems, FaceFit AI provides a more comprehensive and practical solution by combining styling, skincare, and intelligent interaction within a single application.

Future work includes improving model accuracy, enhancing virtual try-on capabilities, expanding dataset diversity for better generalization, and further optimizing the AI chatbot for more advanced personalization and user interaction.

REFERENCES

1. N. Kavyashree et al., “Artificial Intelligence based Smart Cosmetics Suggestion System based on Skin Condition,” in Proc. ICACRS, 2022, pp. 797801.

2. M. Wati et al., “Implementation of the C4.5 Decision Tree Algorithm Method for Selection of Facial Mask Skin Care Products,” in Proc. ICOIACT, 2022, pp. 402406.

3. S. Sharma and S. Bhadula, “Secure Federated Learning for Intelligent Industry 4.0 IoT Enabled Self Skin Care Application System,” in Proc. ICAAIC, 2023, pp. 11641170.

4. C. Deekshitha et al., “AI-Driven Personalized Outfit Recommendation with NLP and GAN Enhanced Chatbot,” in Proc. ICITSM, EAI, Apr. 2025.

5. S. Madhan et al., “Text2FashionGAN: Augmenting Personalized Style Recommendations with cGANs and Word2Vec,” in Proc. ICECA, 2023, pp. 321328.

6. H. Kaneko and K. Matsui, “Design and Implementation of a User-Driven System for Continuous Monitoring and Behavior Change Support in Skin Condition Improvement,” in Proc. ICOIN, 2025, pp. 443448.

7. N. V. R. Gade and R. Mahaveerakannan, “Improving Access to Skin Care with a Multi-Factorial Strategy using Vision Transformers,” in Proc. ICSES, 2024, pp. 11891193.

8. Z. Liu et al., “Deep Learning Face Attributes in the Wild,” in Proc. ICCV, 2015. [Dataset: kaggle.com/datasets/jessicali9530/celeba-dataset]

9. Kritsakorn, “Acne Detection Dataset,” Roboflow Universe, 2023. [Online]. Available: universe.roboflow.com/kritsakorn/acne-kbm0q

10. S. Goel, “DermNet Skin Disease Dataset,” Kaggle, 2021. [Online]. Available:

kaggle.com/datasets/shubhamgoel27/dermnet