HDI-PEARL: Hybrid Derma Intelligence for Smart Skin Analysis and Recommendation

HDI-PEARL: Hybrid Derma Intelligence for Smart Skin Analysis and Recommendation

Neha Beegam PE

Assistant Professor

Devasoorya K, Muhammed Nihal K, Mursila, Vinny VS

Students, Department of Computer Science and Engineering, Federal Institute of Science and Technology Ernakulam, India

Abstract – Understanding skin type and identifying common skin concerns are essential steps in choosing appropriate skin-care products; however, many individuals nd it difcult to accurately determine their skin condition without professional consultation. To address this challenge, this study introduces PEARL, an articial intelligencebased skin analysis and product recommendation system that enables users to evaluate their skin using facial images and receive personalized skincare suggestions. The system incorporates a secure login and registration process supported by an SQLite database, after which users can either capture a sele using their device camera or upload an existing facial image. The image is analyzed using computer vision techniques along with a Convolutional Neural Network (CNN) model implemented with TensorFlow and OpenCV, allowing the system to identify the users skin type and detect common concerns such as acne, pigmentation, and enlarged pores. Due to the limitations of publicly available datasets for reliable skin type detection, a custom dataset was developed and carefully prepared to improve classication accuracy and better represent real-world skin conditions. Following the analysis, the system generates suitable skincare product recommendations through a rule-based mechanism that maps detected skin conditions to relevant products. The entire platform is developed as a web application, utilizing HTML, CSS, and JavaScript for the frontend and Python Flask for the backend, with the overall aim of providing users with a simple, accessible, and effective tool for understanding their skin condition and selecting products tailored to their individual needs.

Index Terms – Skin analysis, skin type detection, acne detection, pigmentation detection, computer vision, convolutional neural network, product recommendation system.

1. Introduction

   In recent years, skincare has become an important aspect of personal health and wellbeing. People are increasingly interested in maintaining healthy skin and selecting products that suit their individual skin conditions. However, under-standing ones own skin type can be challenging. Skin varies signicantly from person to person, and individuals may experience multiple concerns such as acne, pigmentation, dryness, excessive oiliness, or enlarged pores. Without proper knowledge, many people rely on trial and error when choosing skincare products, which can sometimes lead to ineffective results or even worsen existing skin problems.

   Traditionally, skin analysis is performed by dermatologists or skincare specialists who examine the skin and provide professional advice. While professional consultations provide accurate guidance, they are not always accessible to everyone.

   In many cases, visiting a dermatologist may be costly, time-consuming, or inconvenient. As a result, there is growing interest in technological solutions that can help individuals understand their skin condition without requiring frequent visits to skincare professionals.

   The widespread availability of smartphones and digital cameras has made image-based analysis a practical approach for building automated skin analysis systems. At the same time, rapid advancements in articial intelligence (AI)particularly in computer vision and deep learninghave enabled machines to analyze images with high accuracy.Deep learning models such as Convolutional Neural Networks (CNNs) are widely used in various applications including facial recognition, medical imaging, and object detection [9], [12]. These models are capable of identifying complex visual patterns in images and can therefore be applied to analyze facial skin characteristics [3], [4].

   In this work, we present PEARL, an AI-based skin analysis and product recommendation system designed to help users evaluate their skin condition and receive personalized skin-care guidance. The system is implemented as a web-based application with a user-friendly interface. Users can create an account, log in securely, and access features such as skin analysis, previous result history, and feedback options.

   After logging in, users can capture a sele using their device camera or upload an existing facial image. The image is processed using computer vision techniques and analyzed by a trained CNN model implemented with TensorFlow and OpenCV. Through this analysis, the system identies the users skin type and detects common skin concerns such as acne, pigmentation, and open pores.

   One of the major challenges encountered during the de-velopment of this system was the lack of reliable datasets specically designed for skin type classication. Many publicly available datasets contain images with inconsistent lighting, heavy makeup, or unrealistic conditions that do not accurately represent real-world skincare scenarios. To overcome this issue, a custom dataset was created and used for training the classication model. This allowed better control over image quality and improved the reliability of the trained model.

   After identifying the skin characteristics, the system gen-erates product recommendations tailored to the users skin type and detected conditions. A rule-based recommendation

   mechanism is used to map specic skin concerns to appropriate product categories such as cleansers, serums, moisturizers, and sunscreens. In addition to providing product suggestions, the system also offers basic skincare information to help users better understand their skin and skincare routines.

   The objective of this project is to combine articial intelli-gence techniques with practical skincare knowledge to develop an accessible tool for automated skin analysis. By offering personalized recommendations and easy-to-understand results, the proposed system aims to help users make better skincare decisions and improve their overall skincare routine.

2. System overview

   The proposed system, PEARL, is an AI-based skin anal-ysis and skincare recommendation platform that helps users understand their skin condition and select suitable skincare products. The system uses image processing and machine learning techniques to analyze facial images and classify skin types [19], [20].. The application is implemented as a web-based system that allows users to register, log in, analyze their skin condition using a camera or uploaded image, and receive product recommendations based on the detected skin type. The main goal of the system is to provide an accessible and automated solution for skin analysis without requiring professional dermatological consultation. The system consists of three main components: User Interface, Backend Processing System, and Machine Learning Model. The user interface is developed using HTML, CSS, and JavaScript, while the backend server is implemented using Python Flask. The machine learning model is built using TensorFlow and Convolutional Neural Networks (CNN) for skin classication.

   1. System Architecture

      The architecture of the proposed system follows a clientserver model. The user interacts with the web application through a browser. The frontend communicates with the backend server through HTTP requests. The backend processes the requests, performs image analysis using the trained machine lerning model, and returns the results to the frontend. The major components of the system include the Frontend Interface which provides the user interface where users can interact with the system including the login page, skin analysis page, product recommendation page, and history page. The Backend Server is developed using the Flask framework and handles user authentication, image processing, database operations, and communication with the machine learning model. The Machine Learning Model uses a Convolutional Neural Network trained to classify different skin types such as Oily Skin, Dry Skin, Normal Skin, and Sensitive Skin, and also detects common skin problems such as acne and pigmentation. The Database uses SQLite to store user data, login information, and previous analysis results.

      As shown in Fig. 1, the system begins with the user uploading a facial image through the web interface. The image is sent to the Flask backend where image preprocessing and analysis are performed. The processed image is then evaluated by the

      Fig. 1. System Architecture of the AI-Based Skin Analysis and Product Recommendation System

      trained CNN model to detect skin type and skin conditions [21], [22]. .Based on the detected features, the rule-based recommendation module generates suitable skincare product suggestions. The results and user information are stored in the database, allowing users to access their analysis history and recommendations through the dashboard.

   2. Functional Requirements

      Functional requirements describe the features and operations that the system must perform. User Registration requires that the system should allow new users to create an account by providing basic details such as username, email, and password. User Login requires that registered users must be able to log in securely using their credentials. Skin Image Upload requires that users should be able to upload a facial image or capture an image using their device camera. Skin Analysis requires that the system must analyze the uploaded image and determine the users skin type and possible skin conditions. Product Recommendation requires that based on the detected skin type and condition, the system should suggest suitable skincare products. Analysis History requires that users should be able to view previous analysis results stored in their history. Logout functionality must be provided to securely end user sessions.

   3. Non Functional Requirements

      Non-functional requirements describe the quality attributes of the system. Performance requires that the system should process and analyze uploaded images within a few seconds. Security requires that user data must be securely stored and protected, and authentication mechanisms such as JWT tokens are used to ensure secure communication. Usability requires that the user interface should be simple and easy to use so that users can easily upload images and view results. Scalability requires that the system architecture should allow future expansion, such as supporting more skin conditions or additional datasets. Reliability requires that the system should operate consistently without frequent crashes or errors.

   4. Technologies used

   The system uses several technologies for development and implementation. Programming Language: Python is used as the main programming language for backend development. Framework: Flask is used as the web framework for building the backend server. Machine Learning: TensorFlow is used for training and deploying the Convolutional Neural Network (CNN) model [8].Image Processing: OpenCV is used for processing and preparing facial images before analysis [14].

3. Product Recommendation System

   After the skin analysis process is completed, the system provides personalized skincare product recommendations to the user.The recommendation process is based on a rule-based approach [19], [20],where predened skincare rules are used to match the detected skin conditions with suitable products. The system considers several factors such as skin type, acne presence, pigmentation, and open pores while gen-erating recommendations. Based on these detected conditions, the system suggests appropriate skincare products including cleansers, serums, moisturizers, and sunscreens. For example, if the system detects oily skin with acne, it may recommend products containing ingredients such as salicylic acid or oil-control formulations. Similarly, if pigmentation is detected, the system may suggest products containing brightening ingredients like vitamin C or niacinamide. The recommendation rules are designed using common dermatological skincare guidelines to ensure that the suggestions are relevant and practical for users. By combining AI-based skin analysis with rule-based product selection, the system aims to provide users with simple and personalized skincare guidance. These recommendations help users choose products that are more suitable for their skin condition without requiring professional consultation for basic skincare decisions.

4. Results and Discussion

   The performance of the proposed AI-based skin analysis and product recommendation system was evaluated by testing the system with different facial images captured through the webcam or uploaded by users. The main objective of this evaluation was to examine how effectively the system can identify skin type, detect common skin concerns, and provide suitable skincare product recommendations. During testing, the system successfully analyzed facial images and classied the skin type into categories such as dry, oily, or normal skin using the trained Convolutional Neural Network (CNN) model. In addition to identifying the skin type, the system also detects visible skin concerns such as acne, pigmentation, and open pores. These detected conditions are then used as inputs for the recommendation module, which suggests suitable skincare products for the user.

   The results generated by the system are presented through a user-friendly web interface. After the analysis process is completed, the system displays the detected skin type, the identied skin concerns, and the recommended skincare products. These recommendations include commonly used

   skincare products such as cleansers, serums, moisturizers, and sunscreens. The suggestions are generated using predened rule-based logic that connects specic skin conditions with suitable skincare ingredients and products. Another important feature of the system is the ability to store user analysis results in the database. Registered users can view their previous analysis results through the dashboard history option. This allows users to keep track of their skin analysis results and the recommended products over time. Overall, the testing results show that the proposed system is capable of performing automated skin analysis and delivering useful skincare guidance in a simple and accessible manner.

   1. System Output Results

      Fig. 2. Output of the Skin Analysis System Showing Skin Type, Detected Skin Concerns and Recommended Products

      The output shown in Fig. 2 represents the result page generated by the system after analyzing the users facial image. The system rst identies the users skin type using the trained CNN model. In this example, the system predicts the skin type as oily skin. Along with the skin type classication, the system also detects visible skin concerns such as acne, pigmentation, or open pores depending on the features observed in the facial image. After identifying these conditions, the system generates a list of recommended skincare products that are suitable for the detected skin type and concerns. These recommendations include products such as cleansers, serums, moisturizers, and sunscreens that are selected using predened skincare rules. The goal of this recommendation module is to provide users with simple and personalizd skincare suggestions that can help them choose appropriate products for their skin condition. The result page is designed in a clear and structured manner so that users can easily understand the analysis outcome. By presenting the detected skin type, skin concerns, and product suggestions together in one interface, the system provides a complete overview of the users skin condition and possible skincare solutions.

   2. Model Accuracy

   Fig. 3 shows the accuracy achieved by the CNN model used for skin type classication. The model was trained using the custom dataset created specically for this project, which contains labeled images representing different skin types such

   Fig. 3. Accuracy Performance of the CNN Model for Skin Type Classication

   Fig. 4. Analyze Page of AI Based Skin Analysis and Product Recommendation System

   as dry, oily, and normal skin. The training process allowed the model to learn important visual patterns and characteristics associated with each skin type. The accuracy results indicate that the proposed model is able to classify skin types with a high level of reliability. Compared to several existing basic skin detection approaches [4], [2] that rely on simple image processing techniques, the use of a CNN improves the systems ability to extract meaningful facial skin features. This leads to better prediction performance and more accurate skin type identication. The improved accuracy of the classication model directly contributes to the effectiveness of the entire system, as the detected skin type plays a crucial role in determining the appropriate skincare product recommendations. These results demonstrate that the proposed AI-based approach can provide more dependable skin analysis compared to traditional rule-based or manual evaluation methods.

   Number of Correct Predictions

   Fig. 5. Login Page with Login and Sign In Options

   products. The system integrates image processing and machine learning techniques to analyze facial images and identify important skin characteristics.Image preprocessing and analysis are performed [16]. By using a Convolutional Neural Network (CNN) [9], [18]trained on a custom-built dataset, the system is able to classify different skin types such as dry, oily, and normal skin. In addition to skin type classication, the system is designed to detect common skin concerns including acne, pigmentation, and open pores.

   One of the key advantages of the proposed system is the integration of an intelligent recommendation module that suggests suitable skincare products based on the detected skin type and skin concerns. The recommendations are generated using rule-based logic that connects specic skin conditions with appropriate skincare ingredients and products such as cleansers, serums, moisturizers, and sunscreens. This allows users to receive simple and personalized skincare guidance without needing expert knowledge in dermatology.

   The system is implemented as a web-based platform that provides a user-friendly interface where users can capture or upload facial images for analysis. Features such as user authentication, result history, and feedback options further enhance the usability of the system. By storing previous analysis results in the database, users are able to review their past results and track their skincare recommendations over time.

   The experimental evaluation shows that the proposed ap-proach is capable of performing reliable skin type classication and generating useful skincare recommendations. The use of a custom dataset helped improve the performance of the model by allowing it to learn more relevant skin features. Overall, the results demonstrate that the integration of articial

   Accuracy =

   Total Number of Predictions

5. Conclusion

   (1)

   intelligence with a web-based application can provide an effective and accessible solution for automated skin analysis and personalized skincare product recommendation. The proposed system highlights the potential of AI-driven technologies in the eld of skincare analysis and personalized healthcare

   In this paper, an AI-based skin analysis and product recommendation system has been proposed to help users better understand their skin condition and select suitable skincare

   applications. With further improvements and additional training data, such systems can become even more accurate and helpful for users seeking simple and reliable skincare guidance.

6. Future Scope

   Although the proposed system demonstrates effective skin analysis and product recommendation capabilities, there are several opportunities for further improvement and expansion. One of the main areas for future work is improving the accuracy and robustness of the skin condition detection model by training it with a larger and more diverse dataset. A more extensive dataset containing images from different age groups, lighting conditions, and skin tones would help the model learn more variations and improve its prediction performance.

   Another possible improvement is the integration of additional skin condition detection features. In the future, the system can be extended to detect a wider range of dermatological concerns such as dark circles, wrinkles, blackheads, and skin sensitivity. Including more skin conditions would allow the system to provide even more personalized and detailed skincare recommendations.

   The recommendation module can also be enhanced by incorporating more advanced techniques such as machine learning-based recommendation systems instead of relying only on rule-based logic. This could allow the system to learn from user feedback and previous recommendations to provide more accurate and personalized product suggestions over time. Furthermore, the system can be expanded into a mobile ap-plication to make the platform more accessible and convenient for everyday users. Integration with real-time skin monitoring features and personalized skincare routines could also improve

   the overall usefulness of the system.

   With continued development and improvements, the pro-posed AI-based skin analysis system has the potential to become a more comprehensive digital skincare assistant that helps users better understand their skin and make informed skincare decisions.

   1. O. Russakovsky et al., ImageNet large scale visual recognition chal-lenge, International Journal of Computer Vision, vol. 115, no. 3, pp. 211252, 2015.

   2. A. Krizhevsky, I. Sutskever and G. Hinton, ImageNet classication with deep convolutional neural networks, in Advances in Neural Information Processing Systems, 2012.

   3. D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no. 2, pp. 91110, 2004.

   4. G. Bradski, The OpenCV Library, Dr. Dobbs Journal of Software Tools, 2000.

   5. T. Ojala, M. Pietikainen and D. Harwood, A comparative study of texture measures with classication based on featured distributions, Pattern Recognition, vol. 29, no. 1, pp. 5159, 1996.

   6. R. C. Gonzalez and R. E. Woods, Digital Image Processing. New Jersey: Prentice Hall, 2008.

   7. S. Haykin, Neural Networks and Learning Machines. New Jersey: Prentice Hall, 2009.

   8. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning. Cambridge: MIT Press, 2016.

   9. T. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.

   10. C. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006.

   11. P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proc. IEEE Conf. Computer Vision and Pattern Recogniion, 2001.

   12. N. Dalal and B. Triggs, Histograms of oriented gradients for human detection, in Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 2005.

7. References

1. J. Deng, W. Dong and R. Socher, Deep learning for dermatological image analysis: A review, IEEE Access, vol. 11, pp. 2214522160, 2023.

2. S. Patel and A. Kumar, Articial intelligence based skin disease detection using deep learning, IEEE Access, vol. 10, pp. 9052190534, 2022.

3. L. Liu, H. Chen and Y. Zhang, Skin lesion classication using convolutional neural networks, IEEE Transactions on Medical Imaging, vol. 41, no. 4, pp. 10351045, 2022.

4. R. Kumar and S. Singh, Deep learning approach for automated skin analysis and diagnosis, International Journal of Computer Applications, vol. 183, no. 12, pp. 16, 2021.

5. A. Esteva et al., A guide to deep learning in healthcare, Nature Medicine,

   vol. 25, pp. 2429, 2019.

6. F. Chollet, Deep Learning with Python. New York: Manning Publications, 2018.

7. A. Esteva et al., Dermatologist-level classication of skin cancer with deep neural networks, Nature, vol. 542, pp. 115118, 2017.

8. M. Abadi et al., TensorFlow: A system for large-scale machine learning, in Proc. 12th USENIX Symposium on Operating Systems Design and Implementation, 2016.

9. Y. LeCun, Y. Bengio and G. Hinton, Deep learning, Nature, vol. 521,

   pp. 436444, 2015.

10. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, in Proc. Int. Conf. Learning Representa-tions (ICLR), 2015.