AgriBot AI: Multilingual Tomato Disease Detection and Voice Advisory System with Chatbot

AgriBot AI: Multilingual Tomato Disease Detection and Voice Advisory System with Chatbot

Harshith D. N

Student, Department of Artificial Intelligence and Data Science Nitte (Deemed to be University)

Bengaluru, Karnataka, India Email: harshithdn27@gmail.com

Ramesh Marathi

Student, Department of Artificial Intelligence and Data Science

Nitte (Deemed to be University) Bengaluru, Karnataka, India

Email: rameshmarathi7765@gmail.com

AbstractRapid and accurate plant disease detection plays a critical role in improving agricultural productivity and ensuring food security. This paper presents AgriBot AI, an intelligent system designed to detect tomato leaf diseases using deep learning while providing multilingual voice advisory and chatbot-based assistance to farmers. The system utilizes a Convolutional Neural Network (CNN) trained on a tomato leaf dataset containing multiple disease classes. After classification, the system generates detailed disease descriptions and translates them into regional languages using automated translation tools. These translated advisories are converted into speech using text-to-speech (TTS) technology, enabling farmers to receive voice-based guidance. Additionally, an AI-powered chatbot enables conversational interaction for agricultural queries. Experimental evaluation demonstrates that the proposed CNN model achieves high accuracy in detecting tomato leaf diseases. The multilingual and voice-enabled features significantly enhance usability for farmers in rural areas with limited literacy or language barriers. The proposed system demonstrates how artificial intelligence can support precision agriculture by combining computer vision, natural language processing, and speech technologies.

Index TermsAgriculture, Deep Learning, Convolutional

Neural Network, Plant Disease Detection, Multilingual Systems, Text-to-Speech, Agricultural Chatbot

1. Introduction (Heading 1)

   Agriculture remains one of the most important sectors supporting the global economy and food supply. However, crop diseases significantly reduce agricultural productivity, causing economic losses for farmers. Tomato plants are particularly vulnerable to diseases such as Early Blight, Late Blight, Leaf Mold, and Bacterial Spot. Early detection of these diseases is essential to prevent crop damage and maintain yield quality.

   Traditional disease diagnosis methods rely heavily on expert observation, which may not be accessible to farmers in remote areas. Recent advancements in Artificial Intelligence (AI) and Deep Learning have enabled automated plant disease detection using image classification techniques. Convolutional Neural Networks (CNNs) have proven highly effective in analyzing leaf images and identifying disease patterns. Despite these

   Nethravati T.L

   Assistant Professor

   Department of Artificial Intelligence and Data Science Nitte(Deemed to be University

   Bengaluru, Karnataka, India Email: nethravati.tl@nmit.ac.in

   advancements,many AI-based agricultural systems lack accessibility features such as multilingual communication and voice-based guidance.

   Farmers often face difficulties understanding technical information due to language barriers and limited literacy levels.

   To address these challenges, this paper proposes AgriBot AI, a multilingual tomato disease detection system integrated with voice advisory and a chatbot.

   The system enables farmers to upload tomato leaf images, obtain disease predictions, and receive treatment recommendations in their native language through text and speech.

2. RELATED WORK

   Recent research has explored the application of deep learning techniques in plant disease detection. Sun et al.

   [1] proposed a hybrid deep learning model combining global and local feature extraction to improve tomato leaf disease classification accuracy. Ahmed et al. [3] introduced a domain-adaptive neural network capable of improving classification performance with limited training data.

   In addition to image-based detection systems, conversational AI models have been explored to support agricultural advisory services. AgroLLM [2] introduced a large language model-based framework for agricultural knowledge dissemination. Similarly, multilingual chatbot systems have been proposed to assist farmers in obtaining agricultural recommendations through natural language interaction.

   However, most existing solutions focus on either disease detection or chatbot assistance independently. Few systems integrate disease classification with multilingual advisory and voice interaction in a unified framework. AgriBot AI addresses this gap by combining CNN-based disease detection with translation, text-to- speech, and chatbot technologies.

3. PROPOSED SYSTEM

   The AgriBot AI system integrates multiple technologies to create an intelligent agricultural assistant. The system architecture consists of the following components:

   • Image Upload Interface

   • CNN-based Disease Detection Model

   • Multilingual Translation Engine

   • Text-to-Speech Voice Advisory

   • AI Chatbot for Agricultural Queries

     The workflow begins when a user uploads an image of a tomato leaf. The image is preprocessed and passed to the trained CNN model, which predicts the disease class. The system retrieves disease information and translates it into the users selected language. The translated advisory is converted into speech using text-to-speech technology, enabling voice-based communication. The chatbot component provides interactive support for additional agricultural questions.

4. Using the Template

   The proposed AgriBot AI system architecture is designed as a modular framework that integrates multiple technologies including computer vision, natural language processing, speech synthesis, and conversational AI. The architecture ensures efficient communication between different system components while maintaining scalability and flexibility for future extensions.

   Fig. 1. Overall architecture of the AgriBot AI system.

   The architecture consists of six primary layers: User In- terface Layer, Backend Processing Layer, Deep Learning Model Layer, Knowledge Database Layer, Language Processing Layer, and Chatbot Interaction Layer. Each layer performs a specific function and contributes to the overall functionality of the system.

   • User Interface Layer

     The user interface serves as the entry point for farmers to interact with the system. It is designed using modern web technologies such as HTML, CSS, Bootstrap, and JavaScript to ensure responsiveness across both desktop and mobile devices. The interface allows users to perform the following actions: Upload an image of a tomato leaf for disease detection Select the preferred language for advisory output Listen to the voice-based recommendation Communicate with the chatbot for

     additional agricultural guidance The interface is intentionally designed to be simple and intuitive so that farmers with minimal technological knowledge can use the system effectively.

   • Backend Processing Layer

     The backend layer is responsible for coordinating com- munication between different modules of the system. It is implemented using the Flask web framework in Python, which provides a lightweight and flxible server architecture.

     The backend performs several important tasks, including: Receiving and processing uploaded images Passing images to the deep learning model for classification Retrieving disease descriptions from the knowledge base Managing translation and speech synthesis services Handling chatbot API requests The Flask server also exposes REST API endpoints such as

     /predict, /translate, /speak, and /chat that enable interaction between the frontend and backend modules.

   • Deep Learning Model Layer

     The deep learning model layer contains the Convolutional Neural Network (CNN) used for tomato disease classification. This layer processes preprocessed leaf images and extracts visual features required for classification.The CNN model learns hierarchical image representations through convolutional operations, enabling it to detect disease-specific patterns such as discoloration, spots, lesions, and texture variations. The model outputs probability scores for each disease class using a softmax activation function. The class with the highest probability is selected as the predicted disease.

   • Knowledge Database Layer

     After the CNN model identifies the disease, the system retrieves relevant information from a disease knowledge database. This database contains curated descriptions for each tomato disease, including:

     Symptoms of the disease,Causes and environmental condi-tions,Preventivemeasures,Treatment recommendations.

     The knowledge base ensures that farmers receive meaningful explanations and practical guidance after disease detection.

   • Language Processing Layer

     To support farmers from different linguistic backgrounds, the system includes a multilingual language processing module. This module translates disease descriptions into regional languages using automated translation services.Supported languages include English, Hindi, Kannada, Tamil, and Telugu. The translation process ensures that users can understand the disease diagnosis in their native language, improving accessibility and usability.

   • Text-to-Speech Layer

     The translated advisory text is converted into audio using a Text-to-Speech (TTS) engine. The system uses the Google Text-to-Speech (gTTS) library to generate speech output in the selected language.This voice advisory feature is especially useful for farmers who may have difficulty reading text on digital devices. Audio output provides a more natural and accessible way to deliver agricultural recommendations.

   • Chatbot Interaction Layer

   The chatbot component enables interactive communication between the farmer and the system. It is powered by a large language model (LLM) that can respond to agricultural queries in natural language.Farmers can ask questions related to crop diseases, fertilizers, pest control, irrigation practices, and weather conditions. The chatbot processes these queries and generates informative responses in the selected language.The chatbot transforms the system into a digital agricultural assistant, offering continuous support beyond simple disease detection.

5. Mathematical Model of the

   CNN

   The Convolutional Neural Network (CNN) used in the AgriBot AI system performs automated feature extraction and classification of tomato leaf diseases. The CNN processes input images through multiple layers including convolutional layers, pooling layers, and fully connected layers.

   replaced with zero while positive values remain unchanged.

   This improves training efficiency and reduces the vanishing gradient problem.

   1. Pooling Operation

      Pooling layers reduce the spatial dimensions of feature maps and help prevent overfitting. For max pooling, the output value is the maximum value within a pooling window:

      P (i, j) = max(x1, x2, x3, x4) (3)

      Pooling reduces computational complexity and improves generalization.

   2. Fully Connected Layer

      After feature extraction, the feature maps are flattened and passed into fully connected layers for classification. The output of a neuron in the fully connected layer is:

      z = i = 1nwixi + b (4)

      Where:

      wi represents weights, xi represents input features, b represents bias.

   3. Softmax Classification

      The final layer uses the softmax function to convert output values into probability scores.

      Fig. 2. Convolutional Neural Network architecture used for tomato disease classification.

      1. ConvolutionOperation

         Where:

         p(yi) =

         ezi

         C (5)

         j=1ezj

         The convolution layer extracts spatial features from the input image using filters.

         The convolution operation can be expressed as:

         P(y)i) represents the probability of class i, C represents the number of classes. The class with the highest probability is selected as the predicted disease.

   4. Loss Function

      F (i, j) = mnI(i + m, j + n)K(m, n)

      Where:

      • F(i,j) represents the output feature map

      • I(i,j) represents the input image

      • K(m,n) represents the convolution kernel (filter)

   (1)

   The CNN model is trained using categorical cross- entropy loss, which measures the difference between predicted and actual class labels.

   C

   The convolution filters detect features such as edges, textures, and disease patterns present in the tomato leaf images.

   1. Activation Function

      After convolution, the feature maps are passed through the ReLU activation function to introduce non-linearity.

      ReLU (x) = max(0, x) (2)

      The ReLU function ensures that negative values are

      Loss = yilog(pi) (6)

      i=1

      Where: yi is the true label, pi is the predicted probability The model minimizes this loss using the Adam optimizer

      during training.

6. DATASET AND PREPROCESSING

   The dataset used in this research consists of approximately 20,000 tomato leaf images categorized into ten classes, including nine disease categories and one

   healthy class. Images were resized to 64×64 pixels and normalized before training.

   Data augmentation techniques such as rotation, flipping, zooming, and brightness adjustment were applied to increase dataset diversity and improve model generalization.

7. Model Architecture

   The CNN architecture consists of three convolutional layers followed by pooling and dense layers. ReLU activation functions are used in hidden layers, while the final layer uses the softmax function for multi-class classification.

   p ezi

   (yi) = C (7)

   j=1 ezj

   where pi represents the predicted probability for class i and

   C represents the total number of classes.

   Fig. 3. Workflow of the AgriBot AI disease detection and advisory system.

8. Experimental Results

   Real- world data implementation and testing on a range of performance and usability parameters were effectively carried out for the AgriBot AI system. The implementation focused on developing a robust, multilingual, AI-driven web application with the ability to support farmers in recognizing diseases of tomatoes, advising solutions, and communicating through voice.

   The following section gives the results obtained from each of the modules implemented and hoe each contributes to the iprovement of the systems usability and accessibility.

   1. Multilligual Speech Output

      The Text-to-speech (TTS) component was tested to verify its ability to read out disease descriptions and preventive solutions in different Indian languages. The system used the Web speech API (for web-based playback) and gTTS / pyttsx3 modules (for python backend voice synthesis).

      Fig. 4. User Image uploding window.

      Observations:

      The TTS system generated clear andintelligible audio outputs for all supported languages -English, Hindi, Kannada, Telugu, and Tamil.

      • Language selections from the dropdown menu dynamically changed the output languages.

      • Average speech synthesis time per description:1.8 seconds.

      • Pronunciation accuracy for regional languages exceeded 92%, ensuring farmers could easily comprehend the advisory.

      • Farmers could hear the disease name, description, and recommended fungicide.

      • The voice module improved user inclusivity, especially for illiterate or elderly farmers.

      • In usability tests, 96% of participants rated the speech clarity and usefulness as excellent.

   2. Voice-Based Chatbot

      The chatbot integrated with GPT-based conversational logic successfully handled text and voice queries. It enabled farmers to ask natural questions such as How do I tread tomato blight? or Which fungicide should I use for leaf curl? The chatbot generated contextual answers in both text and spoken formats.

      • Average query response time: 2.3 seconds. Users were able to switch language mid-session without reloading.

      • The chatbot understand both typed and spoken queries accurately when users spoke in English or Kannada.

      • .Responses included disease information, organic alternatives, and cultural practices (e.g., irrigation spacing, crop rotation).

      • The voice-enabled chatbot significantly simplified

        interaction for users with low typing proficiency.

        • The chatbot achieved an average response relevance score of approximately 93%, indicating that the responses generated were contextually appropriate and useful for agricultural decision-making.

        • The chatbot also demonstrated the ability to support multi-turn conversations, allowing users to ask follow- up questions related to the detected disease.

        • Farmers appreciated that the chatbot provided not only chemical recommendations but also organic and preventive alternatives.

      Fig. 5. Voice Based ChatBot

   3. Product Recommendation System

      The integrated product recommendation engine displayed relevant fungicides, insecticides, and fertilizers for each predicted disease.

      Each disease class (e.g., Early Blight, Late Blight, Leaf Mold,Bacterial Spot ,Mosaic Virus etc.) triggered the retrieval of corresponding products from the advisory database.

      Fig. 6. Recommendation Plant Core Solution.

      • Product data (name, image, disease applicability, and purchase link) was fetched dynamically from the database.

      • Each card displayed a Buy Product button linked to the

        Market Page.

      • Example: For Late Blight, the system suggested ACROBAT Fungicide and Propineb 70% WP Fungicide.

9. DISCUSSION

   The integration of multilingual translation and text-to- speech significantly improves system accessibility.

   Farmers can receive disease descriptions in their preferred language.

   and listen to the advisory through audio playback. The chatbot module enhances user interaction by allowing farmers to ask follow-up questions related to crop diseases, fertilizers, and farming practices.

10. Conclusion and Future Work

This research presented AgriBot AI, a multilingual tomato disease detection system with voice advisory and chatbot functionality. The CNN model achieved high accuracy in disease classification, while the translation and text-to- speech modules improved accessibility for farmers. The proposed system demonstrates the potential of integrating computer vision, natural language processing, and speech technologies for smart agriculture applications.

Future work will focus on expanding the system to support multiple crops and integrating IoT sensors for real-time crop monitoring.

1. Model Accuracy improvement & expansion of dataset.

   To realize better accuracyin disease classification, future work can should focus on improving model robustness by incorporating larger and more varied datasets. Currently , the model primarily relies on the PlantVillage tomato dataset, which contains well- labeled but laboratory- capturat images under uniform lighting and backgrounds. In real-world conditions, it needs to ensure high reliability. It is important to train the model using images collected from actual farms, covering differences in illumination, leaf orientation, maturity stage, and environmental factors.

2. Real-time Camera Scanning and Video-Based Detection

   Currently, the AgriBot AI application supports uploading static images. For disease detection. Future enhancements will introduce real time camera scanning thus allowing for the continued monitoring moitoring of the plant health through live video feeds from smartphones, drones or IoT cameras.

3. Deployment as a PublicWeb platform and Mobile Ecosystem.

To large-scale accessibility can be guaranteed by

extending the system. A model operating beyond a standalone ver-sion:web and mobile integrated platform. The future version of AgriBot AI should Provide a single public agricultural portal that interconnects farmers, researchers, and agricultural officers.

REFERENCES

1. H. Sun et al., Efficient deep learning-based tomato leaf disease detection through global and local feature fusion, BMC Plant Biology, 2025.

2. D. J. Samuel et al., AgroLLM: Connecting Farmers and Agricultural Practices through Large Language Models, arXiv, 2025.

3. S. Ahmed et al., DExNet: Domain Adapted Critics for Leaf

   Disease Classification, arXiv, 2025.

4. M. Hosen and M. H. Islam, Aggrotech: Leveraging Deep Learning for Sustainable Tomato Disease Management, arXiv preprint, Jan. 2025.

5. K. N. Q. Nguyen, L. L. T. Thu, and L.-D. Quach, A Vision- Language Foundation Model for Leaf Disease Identification, arXiv preprint, May 2025

6. A. Author et al., Plant Disease Detection through Multimodal

   Large Language Models, arXiv preprint, Apr. 2025.

7. D. J. Samuel et al., AgroLLM: Connecting Farmers and Agricultural Practices through Large Language Models, arXiv preprint, Mar. 2025.

8. H. A. Shehu, Early detection of tomato leaf diseases using transformers: transfer-learning approaches, ScienceDirect (Article), 2025.

9. N. Nasir et al., Interpretable deep multimodal-based tomato disease identification, Scientific Reports (Nature Portfolio), 2025.

10. A. Resercher et al., Leveraging deep learning for plant disease identi f ication: a bibliometric and methodological survey, arXiv preprint,Apr. 2025.

11. T. Author et al., Deep learning-based classification, detection, and segmentation of plant diseases, ScienceDirect (Article), 2025.

12. M. K. Oni and T. T. Prama, Optimized Custom CNN forRealTime TomatoLeafDiseaseDetection,arXipreprint, Feb.2025.

13. M. Sivakiran, Agricultural Chatbot Voice Assistant Using NLP

    Tech- niques, IJSAT, 2025.

14. S.Mehdipour,VisionTransformersinPrecisionAgriculture,arXivp

    reprint,2025.

15. Author et al., Plant Leaf Disease Detection and Classification

    Using Deep Learning, arXiv preprint, Jan. 2025.

16. Ochijenu et al., Development of an Improved Capsule- YOLO Network for Automatic Tomato Plant Disease Early Detection and Diagnosis, arXiv preprint, Jul. 2025.

17. A. Vijayvargia et al., Intent Aware Context Retrieval for Multi-

    Turn Agricultural Chatbots, arXiv preprint, Aug. 2025.

18. Z. Salman et al., Plant disease classification in the wild using vi sion transformers and mixture-of-experts, Frontiers in Plant Science 2025.

19. Author et al., Explainable AI-Enhanced Deep Learning for

    Pumpkin Leaf Disease Detection, arXiv preprint, Jan. 2025.

20. M. R. Tonmoy, M. M. Hossain, N. Dey, and M. F. Mridha, Mobile PlantViT: A Mobile-friendly Hybrid ViT for Generalized Plant Disease Image Classification, arXiv preprint, Mar. 2025.

21. H. Sun, R. Fu, X. Wang, Y. Wu, M. A. Al-Absi, Z. Cheng, Q. Chen, and Y. Sun, Efficient deep learning-based tomato leaf disease detection through global and local feature fusion, BMC Plant Biology, vol. 25, p. 311, 2025.

22. D. J. Samuel, I. Skarga-Bandurova, D. Sikolia, and M. Awais, AgroLLM: Connecting Farmers and Agricultural Practices through Large Language Models for Enhanced Knowledge Transfer, arXiv preprint, arXiv:2503.04788, 2025.

23. S. Ahmed, M. B. Hasan, T. Ahmed, and M. H. Kabir, DExNet: Combining Observations of Domain Adapted Critics for Leaf Disease Classification with Limited Data, arXiv preprint, arXiv:2506.18173, 2025.

24. S. Karuturi, Tomato Leaf Disease Detection Using Deep Learning, Proceedings of the International Conference on Computer Vision Appli cations, 2025.

25. K. Swaroop, FarmGPT: An AI-Powered Chatbot Using NLP for Accurate Farming and Disease Prediction, International Journal for Research in Applied Science and Engineering Technology,

vol. 13, no. 5, 2025