AI-Ml Customer Support Chatbot using FFNN-Feed Forward Neural Network Preprocessing Technique

AI-Ml Customer Support Chatbot using FFNN-Feed Forward Neural Network Preprocessing Technique

Nandhini S

School of Information Science Presidency University Bangalore, India

M Nandini

School of Information Science Presidency University Bangalore, India.

Sandhya S

School of Information Science Presidency University Bangalore, India

Umme Imon

School of Information Science Presidency University Bangalore, India

Abstract In real world, Chatbots are essential for solving complex problems and effective user interaction. This research develops a customer support chatbot employing state of the art machine learning (ML) artificial intelligence technologies and natural language processing (NLP) techniques. The neural network architecture can model linear as well as nonlinear relationships apart from the diverse nature of data. When a customer enters a query in the client libraries, that input processing occurs using natural language processing services and then given to Softmax for classification to ensure equality in the distribution of scores among categories. Most of the earlier research approaches have failed to achieve tangible gains in scalability, dynamic integration, and complicated customer intent understanding in most studies. Earlier research lack in preprocessing techniques like tokenization, which, resulting in low accuracy and high latency. This research devotes its objectives to filling these critical gaps by preprocessing techniques such as tokenizing, stemming, and vectorization methods, using tools like Flask, Pandas, and TensorFlow. The intents file consists of intent data labeled and is to be used for training the model for a match closer to the real-world interactions. This processes to reduce redundancy within the Natural Language Toolkit (NLTK) based preprocessing has been done to deliver overall performance. Compared to previous traditional methods, it gives higher accuracy levels than other traditional methods. The proposed research demonstrates a modular method to making use of machine learning based answers to customer support packages, with an emphasis on scalability and real-international applications.

Keywords Chatbots, Feed-forward Neural Network (FFNN), Machine Learning, Natural language processing (NLP), Flask and TensorFlow.

1. INTRODUCTION

   The explosive and record-level growth in machine learning and artificial intelligence has considerably changed customer service operations in industrialized countries. Customer service in the past has been based on human representatives, which resulted in high operational costs, delayed responses, and failure to respond to multiple queries concurrently [1], [2].In addition of AI-based chatbots has offered companies cost- effective and scalable solutions, offering automated, real-time, and personalized customer communications [3], [4]. These Deep Learning and Natural Language Processing chatbots are transforming the landscape of customer support by increasing user interaction, minimizing turnaround time, and enhancing availability of service [5], [6].

   This research addresses the development, design, and deployment of AI chatbots using advanced machine learning technology and NLP strategies. The main goal is to transcend the limitations of traditional chatbot platforms by enhancing their capability to detect various user commands and provide context-dependent and correct responses [7], [8]. With the incorporation of deep learning architecture like feed-forward neural networks, AI chatbots can easily categorize customer inputs, classify categories of intent, and generate precise, human-language responses, thereby improving user experience significantly [9], [10].

   Structural background of the suggested chatbot model is a Feedforward Neural Network (FFNN), and it is supervised with logistic regression methods on JSON-structured intent datasets as well. This information has varying question and answer patterns and allow the chatbot to identify various intents and give contextually appropriate responses [11], [12]. In this

   organized manner, AI chatbots can manage complex, multi-turn conversation, enhance response accuracy, and mimic human- like decision-making [13], [14].Their overall application performance in commercial, banking, health care, and technical support contexts indicates their ability to be implemented across different business applications [15], [16]. Moreover, real-time learning, sentiment, and ethics rules for AI enhance chatbot capabilities to provide greater customer satisfaction, security, and alignment with the norms of AI ethics [17], [18]. The paper also discusses innovative AI chatbot technology like multimodal AI, hybrid AI architecture, and reinforcement learning-based optimization techniques that are revolutionizing customer service automation [19], [20]. All of these developments vindicate the revolutionary potential of AI- driven customer service systems as precious resources for modern organizations that have to optimize the quality of experience and effectiveness of operations.

2. RELATED WORKS

   Artificial intelligence (AI) and deep learning have significantly improved chatbot systems, and they are more effective in customer service applications. A number of studies have explored various parameters of chatbot deployment, including neural network architecture, performance metrics, and real- implementations.

   Doe & Smith carried out research on the application of AI chatbots in customer service, focusing on the application of deep neural networks to enhance user experience. The research demonstrates the shift from rule-based chatbots to smart virtual assistants that are capable of managing human questions more effectively [3]. Similarly, Johnson & Lee reiterated AI methods in customer service chatbots, considering their effect on response accuracy and customer satisfaction [4].

   Several researchers have investigated the application of neural networks in chatbot platforms. Davis & Brown contrasted neural network design in customer service chatbots, showing how deep learning enhances conversational accuracy and contextual awareness [5]. Martinez & Nguyen also investigated AI-driven chatbots in e-commerce and concluded that neural networks improve customer interaction and sales conversion rates [6].

   One of the most prominent fields of research focus in building chatbots has been applying deep learning to improve response generation. Wilson & Taylor emphasized real-time adaptation and learning to make AI chatbots optimize user experience in real time [7]. Harris & White examined performance indicators like user satisfaction, response appropriateness, and conversation fluency in customer support chatbot applications. Enhancing customer engagement and natural language understanding is one of the largest challenges for developing chatbots. Clark & Lewis studied deep learning techniques used in customer interaction, specifically NLP and sentiment analysis, being implemented in AI chatbots for better understanding emotional tone and intent of users [9].

   The application of AI chatbots has also stretched to finance and healthcare. Robinson & Walker wrote about money chatbots, illustrating how it is possible through neural networks for

   personalized finance advisory and query options specific to the user [10]. In the medical field, Perez & Hall authored on AI chatbot-supported patient care, showing how chatbots powered by neural networks are involved in medical consultation, symptom diagnosis, and health information communicaion [11].

   In spite of the gigantic progress in AI chatbots in customer care, finance, and healthcare, contextual comprehension, ethical AI, and measures of evaluation are still to be seriously tackled. Young & King examined ethical aspects, personalization, and emotional intelligence in chatbot development, hinting towards fairness and transparency [12]. Multimodal AI, advanced NLP capabilities, and AI fairness of decision-making will be issues addressed by future studies, providing a responsive and intelligent atmosphere for chatbots [13], [14].

3. PROPOSED FRAMEWORK

   The proposed framework uses machine learning, Customer support chatbot follows the systematic approach adopted by the data collection and preprocessing stages, model selection, implementation, and database integration.

   1. Data gathering and processing:

      An intents Json file was created, which carried sample queries from the users and responses to those with predetermined intentions tagged on them. It was a base on which the training of the chatbot started. To build the robustness of the application, even more datasets were prepared including varied phrasing and different scenarios to prepare the chatbot to respond robustly to different forms of inputs from the user. Preprocess Input data were standardized. Tokenization, stemming, and removal of stop words are based on the usage of Natural Language Toolkit (NLTK) library, the preprocessing done on the data was converted into numerical format for use in machine learning models by using the Bag-of-Words (BoW) model.

   2. Tokenization and Preprocessing

      It simply tokenizes the sentences, eliminating unnecessary items such as punctuation and stop words, and does stemming or lemmatization that normalizes forms.

      Algorithm 1: Tokenization and Preprocessing
      Input Initialization	:	Feed unrefined data string S = s1, s2, s3sn where si represents a single string (e.x paragraph)
      Output	:	Return token refined string as Srefined, appropriate for prediction and training model
      Step 1	:	Tokenization: Break each clause Si into smaller words (codes) Scodes = x1, x2, x3xn.
      Step 4	:	Standardization: change all codes to lowercase Sp =
      Step 5	:	Halt word removal: detect and omit general stop word, e.g., as this that from catalog Scodes = Scodes Stop Words.

      Step 6 Stemming/Lemmatization Expand words to their lemma forms, e.g., “hopping” “hop”

      1. BoW Model: Converts sentences to fixed-length vector representations based on word frequency.
      2. Feedforward Neural Network (FFNN): Processes vectorized text data. Apply weights and activation functions to uncover patterns
      3. Optimization with Stochastic Gradient Descent: Calculate the gradient of the loss. Updates weights iteratively using a predefined learning rate. Learns over different epochs till convergence

         Formula: Softmax(zi)=ezij=1nezjSoftmax(z_i)=e^{z_i}/_{j=1}^n e^{z_j}i

   3. Architecture of Proposed Model
      1. System Architecture Diagram

         The design of tell-me-about-it configuration entirely modularizes the architecture of customer support chatbots due to efficient handling of all its functionalities.

         figure 1: System Architecture

         System modules consist of four major such types: Data Preprocessing, Model Training, Response Generation, and Frontend Integration. Comprising the data preprocessing is the first level, meaning it converts the ungracious user input into a form ready for machine learning. It does tokenization, stemming, stop-word removal, and so on. All in all, the operation standardizes and normalizes input for the future training process. The Model Training Module is that one into which there is embankment of the core intelligence of the chatbot. Based on TensorFlow and Keras as implement using feedforward neural network architecture, this module is going to be trained to classify intents in user queries given their respective processed input data. For its training phase, it would tap the labeled datasets stored within intents. Json and use Stochastic Gradient Descent (SGD) and so forth to optimize model weights; thereby, the high-accuracy recognition of diverging intents and queries. The Response Generation Module is termed a decision layer since it actually maps a user’s

         inputs to the most appropriate identified intents defined by the trained model. Once the intent is picked, it fetches from the predefined response stored in the dataset so that correct and contextual responses are guaranteed. Finally, the Frontend Integration Module integrates the backend functionality with the user interface

         .

      2. Flow of Data Preprocessing

         figure 2: Data processing flow diagram

         The given pipeline is the Data Preprocessing Pipeline which is the major process of preparing raw text input by users into a model that could be employed by the machine learning that a chatbot uses. Starting from the Input Query, that is, raw text written by the user. The Input will be tokenized, split into individual words or tokens; to explain further “How are you?” would be tokenized to “How”, “are” and “you”. After tokenization is Stopword Removal which takes away common words like (“is”, “the”, “and”) which have no effect in identifying an intention from meaning, reducing data noise and hence more precise processing. Thus, stemming is the next stage, where words are brought back to their root forms to avoid repetition, so terms such as “running,” “runs”, “runner” will be stemmed into one base word as “run.” Then, the filtered text is passed through the Bag-of-Words Conversion for vectorization, or converted to numerical values using this method. It provides a specific vector for the tokenized text through either the presence or absence of words among vocabulary terms.

         Consequently, increased usefulness in machine learning applications. The same pre-processed data is used for training or intent classification in the chatbot system.

      3. Model Architecture

         figure 3: Model Architecture

   4. IMPLEMENTATION

   The proposed work consists of several steps the most important among which is data collection. In-between pre-activation, training integration, and deployment must be carried out in order for the chatbot to be trained well and answer customer questions wisely.

   Algorithm 2: Development of chatbot

   The chatbot framework employs a feedforward neural network for the purpose of intent detection, processing numerical features extracted from user input queries. The architecture of the model consists of three elements: input layer, hidden layers, and output layer, for providing one-way information flow for computational stability.

   1. Input Layer: Converts user queries into numerical vectors through Bag-of-Words (BoW), where every word is mapped by frequency and presence in a pre-defined vocabulary.
   2. Hidden Layers: Comprised of neurons that carry out weighted sums of the inputs, applying the ReLU activation function for non-linearity and complex pattern recognition.
   3. Output Layer: Uses a softmax activation function for determining intents with probabilistic output values summing to 1. The model is trained with cross-entropy loss iteratiely, optimizing correct prediction weights.

      This structured architecture offers robust intent classification with feedforward one-way links depending on it and no

      Input Initialization Output

      Step 1

      Step 4

      Step 5

      Step 6

      Step 7

      : End user string enquires and intents

      : Operation chatbot ability to predict intents and response

      : Collect String: determine users intent and segment them into precise identifiers (e.g,. enquires, regards)

      : Processing Text: Tokenize the given code into smaller clause (e.g,. Hello, whats up

      [“hello”, “whats”, “up”]).

      : Training Model: define FFNN, encode string, hidden layer extraction

      : Deploy Model: save in .p form, implement backend to handle end user queries

      : Interaction: design UI, generate communication among frontend and

      backend for user response

      feedback loops, enhancing stability in the model.

      1. FRONTEND TO BACKEND INTERACTION

      User queries are input via an HTML/CSS interface, which triggers an HTTP POST request to the Flask server. The query is preprocessed into Bag-of-Words (BoW) vectors and passed through a pre-trained TensorFlow feedforward neural network for intent classification. The model passes the input through its layers (input, hidden, output) and returns the most probable intent. The response is passed back to Flask, and it updates the frontend in real time, making communication between the server, user interface, and machine learning model very easy.

      figure 4: Frontend to Backend Interaction Diagram

      Step 8 : Generation: preprocess received string, trained model predicts intent of input, response generates from flask

4. EXPERIMENT AND OUTCOMES
   1. 1. Experimental Setup

         The chatbot was trained using a dataset of customer interactions, pre-processed with tokenization, stopword removal, and word embeddings (TF-IDF, Word2Vec). The experiments were conducted on a system with Intel Core i7, 16GB RAM, and NVIDIA GPU, utilizing TensorFlow and Scikit-learn for model training.

      2. Model Training and Evaluation

         The Feedforward Neural Network (FFNN) was trained using an 80-20% train-test split, optimized with Adam optimizer (learning rate: 0.001), and regularized with dropout (0.3) to prevent overfitting. Performance was benchmarked against Multi-layer Perceptron (MLP), Recurrent Neural Networks (RNN), and Convolutional Neural Networks (CNN).

      3. Performance Metrics

         Evaluation was based on accuracy, precision, recall, and F1- score. The FFNN model achieved 94.12% accuracy, outperforming MLP (91.37%), RNN (88.23%), and CNN (73.89%). The confusion matrix analysis confirmed minimal misclassifications in FFNN compared to other models.

      4. Key Findings

         The results demonstrate that FFNN provides superior accuracy and reliability in chatbot query classification, making it an efficient choice for automated customer support systems. The models low false-negative rate and high true-positive rate further validate its robustness.

      5. Outcomes

   Figure 5: Outcomes of chatbot model

5. RESULTS AND DISCUSSION

The comparative assessment of the performance of the chatbot under various machine learning models is demonstrated in this section. The suitability of the Feedforward Neural Network (FFNN) model is evaluated compared to other models such as Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN). It has been assessed considering accuracy, precision, recall, F1- score, and the impact on datasets to achieve an all- encompassing performance of the chatbot’s efficiency.

1. Comparative Performance Analysis of Classifier Models

   For further confirmation of the efficacy of the chatbot, precision, recall, F1-score were calculated for all models. These values speak about the reliability of the model in classifying user intents accurately. The comparison results are illustrated in table.

   Model	Accuracy
   Feedforward Neural Network (FFNN)	94.12
   Multi-layer Perceptron (MLP)	91.37
   Recurrent Neural Networks (RNN)	88.23
   Convolutional Neural Networks (CNN)	73.89

   Table 1: Accuracy Comparison of Different Models

   The model FFNN works better than all the other architectures with an accuracy of 94.12%, and CNN has the worst accuracy (73.89%). The MLP model, being a fully connected neural network, works relatively high (91.37%), though less efficiently than FFNN because of inefficiencies in optimizations. Similarly, RNN works 88.23% accuracy, with an inability to process long- term dependencies in sequential data processing that leads to misclassifications. Figure 6 illustrates accuracy distribution across models.

   Figure 6: Prediction of Accuracy

2. Evaluation Metrics Analysis

   To further authenticate the effectiveness of the chatbot, precision, recall, F1-score were computed for all models. The values provide a sense of how reliable the model is in identifying user intent correctly. Table II depicts the comparison results.

   Model	Precision	Recall	F1-Score
   FFNN	0.93	0.91	0.92
   MLP	0.89	0.87	0.88
   RNN	0.85	0.82	0.83
   CNN	0.78	0.76	0.77

   Table 2: Precision, Recall, and F1-score Comparison

   • FFNN achieves the maximum accuracy (0.93), meaning minimum false-positive predictions.
   • Recall is optimized to 0.91, which shows the optimal recall of relevant responses by the chatbot.
   • CNN has the poorest F1-score (0.77), which verifies its poor performance in processing textual information.
   • MLP and RNN are satisfactory but fail to comprehend intricate contexts.
3. Evaluation Metrics

A number of experiments were directed at the evaluation of recall and F1 score of the chatbot responses against various other models. The results confirmed feed-forward neural network superiority to the other models. F1 score: The F1 score achieved 0.92, indicating the end or success of a chatbot towards all these measurements of precision and recall.

Figure 7: F1 Scores and evaluation

C. Impact of Training Dataset Size on Model Performance

To verify the impact of training dataset size on chatbot accuracy, experiments were performed for different dataset ratios (50%, 75%, 80%, and 90%). The findings, as observed in Figure 8, are that performance is better with larger dataset sizes but overfitting happens above 80%.

Observations:

1. 50% dataset size (Underfitting)
2. 75%-80% dataset size (Optimal Performance)
3. 90% dataset size (Overfitting)

Figure 8: Training Size vs accuracy analysis

D. Comparative Confusion Matrix Analysis

For measuring the classification accuracy of the chatbot, confusion matrices for FFNN, MLP, RNN, and CNN were created and presented in Figure 9.

1. For measuring the classification accuracy of the chatbot, confusion matrices or FFNN, MLP, RNN, and CNN were created and presented in Figure 4.
2. FFNN had the maximum True Positives (TP) = 120 and minimum False Negatives (FN) = 7, thus providing minimal misclassification.
3. MLP had 112 TP but with maximum FN = 15, decreasing accuracy.
4. RNN obtained 105 TP and FN = 20, resulting in lower recall.
5. CNN was worst with 95 TP and FN = 35, which reflects high misclassification.

   Figure 9: Comparative Confusion matrix

   [9]	L. H. M. Perez, “Integrating AI Chatbots into Healthcare: A Neural Network Approach to Patient Support,” IEEE Journal of Biomedical and Health Informatics, vol. 18 , 2023 .
   [10]	B. K. H. Young, “The Role of AI Chatbots in Enhancing Customer Experience: A Neural Network Perspective,” IEEE Transactions on Engineering Management, vol. 30 , 2022 .
   [11]	P. L. R. Zhang, “Conversational AI Chatbots: Advances in Deep Learning-Based NLP Models,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35 , 2024.
   [12]	C. G. A. Thomas, “Preprocessing Techniques for Chatbot NLP Pipelines: An Empirical Study,” IEEE Transactions on Artificial Intelligence, vol. 9 , 2023 .
   [13]	D. W. M. Kim, “Enhancing Customer Experience with AI Chatbots in Retail Services,” IEEE Transactions on Services Computing, vol. 16 , 2024.
   [14]	L. H. J. Brown, “Chatbot Response Optimization Using Reinforcement Learning,” IEEE Transactions on Computational Social Systems, vol. 14 , 2023.
   [15]	S. W. P. Liu, “Hybrid AI Chatbots: Combining Rule-Based and Deep Learning Models,” IEEE Transactions on Human-Machine Systems, vol. 11 , 2022.
   [16]	E. G. O. Patel, “Sentiment Analysis in AI Chatbots for Improved Customer Engagement,” IEEE Transactions on Computational Social Systems, vol. 15 , 2023.
   [17]	N. G. K. Singh, “Real-Time Language Translation in AI Chatbots Using Transformer Models,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33 , 2024.
   [18]	Y. C. M. Ahmed, “A Comparative Study of Neural Network Architectures for Chatbot Development,” IEEE Transactions on Artificial Intelligence, vol. 10 , 2023.
   [19]	H. W. T. Jackson, “Chatbot Security: Addressing Privacy Concerns in AI-Powered Customer Service,” IEEE Transactions on Services Computing, vol. 21 , 2023 .
   [20]	V. K. S. Reynolds, “Optimizing AI Chatbot Response Time Using Lightweight Deep Learning Models,” IEEE Transactions on Neural Networks and Learning Systems, vol. 9 , 2024.

    

   1. CONCLUSION

      The customer support all-purpose chatbot is an Artificial Intelligent M-Learning, Natural Language Processing application. It uses feedforward neural network and intent- based models to interpret user queries and provide answers to diverse questions. The development was modeled using TensorFlow, NLTK, and Flask among frameworks so as to make it scalable, efficient, and easily deployable. Accuracies, precisions, recalls, and turn-around time for response have been designed to meet both subjective and objective approaches, thus demonstrating reliability and efficiency in solving most- established common queries. A user index database module is additional to this system, which saves user details and query histories. Besides tracking interaction, it would customize answers, and analyze patterns of user behavior, thus enhancing their engagement and satisfaction levels. This combination of highly AI-powered capabilities with serious data occupancy enables a tailored and insightful example of user experience. However, some limitations exist: the current bot is underwhelming in multi-intent detection as it does not track complex queries having overlapping intents. It can be considered quite good in identifying context for simple, solitary interactions, although improvement could certainly be made with more complex conversational flows. Future developments may include the use of advanced transformer-based models like BERT or GPT to enhance multi-intent and context capture and then enable adaptable interaction from the purpose in the future. This work serves the purpose of bringing forth intelligent customer service systems that simplify interactions, reduce response time, and enhance user satisfaction. Ongoing progress in this would lead to development into a very highly scalable and efficient highly friendly solution for customer service.

   2. REFERENCES