LETTER OF DECLARATION I hereby certify that my graduation project, "Sentiment analysis of customer feedback for KFC restaurant using machine learning " is my work and has not been previo
INTRODUCTION
Research background
In recent years, the rapid development of the Internet has led customers to share their comments and feedback about products and services on social media, influencing the purchasing decisions of future buyers As a result, websites are now designed to facilitate user experiences, allowing individuals to read reviews before making purchases From a behavioral psychology standpoint, customers consider not only product information but also peer feedback, which helps them select reliable services and products To refine their business strategies, companies gather customer feedback, but managing the influx of information can be challenging Therefore, it is essential for businesses and stakeholders to implement automated systems that analyze and summarize user feedback, enabling quick and informed decision-making.
Analyzing and evaluating student feedback is crucial during the learning process However, much of the feedback tends to be superficial, focusing on general surveys rather than delving deeply into the underlying issues.
Many systems analyzing user feedback on websites primarily rely on rating scales for products and services However, these rating scales fail to objectively capture user satisfaction as effectively as written comments and feedback sections.
Some systems have been developed to analyze user comments effectively For instance, a user review stating, "The food is really good, the service is good, but the downside is that it is a bit hard to find," would be interpreted as positive by a sentiment analysis system.
In the restaurant industry, customers often rely on feedback from previous diners to make informed decisions about where to eat or host events They focus on specific factors such as food and beverage quality, service, ambiance, and pricing, rather than just the general reputation of the restaurant This attention to detail helps them choose establishments that meet their expectations and preferences.
To enhance our understanding of user evaluations, it is crucial to conduct aspect-based opinion analysis within the restaurant sector By developing a system that effectively analyzes user comments, we can extract valuable insights that reflect the detailed status of various aspects.
User -based opinion analysis is gaining traction across various fields, including education, sociological surveys, and particularly in the service and business sectors Many algorithms and corpora have been developed and tested in multiple languages Recently, the VinAI Research Center introduced the phoBert dataset for Vietnamese text on November 6, 2023 Additionally, VinAI announced two large language model variants, PhoGPT-7B5 and PhoGPT-7B5-Instruct, which are pre-trained with 7.5 billion parameters using 41GB of Vietnamese text data, now available on GitHub and Hugging Face for Vietnamese users.
Sentiment analysis is a crucial natural language processing technique that examines the emotions expressed in text, classifying it as positive, negative, or neutral For businesses like KFC, this analysis is vital for interpreting customer feedback, improving service quality, and bolstering brand perception.
Objectives of the research
This study analyzes online reviews to extract and interpret customer opinions, identifying crucial factors that impact customer satisfaction, including food quality, service experience, pricing, and restaurant atmosphere.
This study evaluates various models, such as Logistic Regression, Multinomial Naive Bayes, Support Vector Classifier (SVC), and PhoBERT, to identify the most effective approach for sentiment classification It also tackles challenges like imbalanced datasets and aims to improve the accuracy of sentiment analysis Ultimately, the research offers valuable insights into customer preferences and identifies areas for improvement, assisting KFC in enhancing service quality and overall customer experience.
Scope and limitations of the research
This research investigates sentiment analysis of customer reviews for KFC restaurants in Hanoi, Vietnam It leverages user-generated comments and feedback from online platforms, capturing customer opinions about their dining experiences.
The analysis focuses on 11 different aspects, including food quality, service experience, ambiance, and pricing The initial data was sourced from Google reviews, translated into Vietnamese, and subsequently categorized into specific labels such as "Positive."
“Neutral”, and “Negative” based on the star rating To perform the analysis, the study employs some models as Logistic Regression, Multinomial Naive Bayes, Support Vector Classifier (SVC), and PhoBERT
This study analyzes Vietnamese text data preprocessing through techniques such as tokenization, stop-word removal, and TF-IDF vectorization It addresses data imbalance issues using methods like SMOTE to enhance result reliability and accuracy Although the research is confined to Vietnamese reviews within a specific geographic area, the findings are intended to assist business owners in understanding customer preferences, common complaints, and overall sentiment trends By leveraging these insights, businesses can adapt their strategies to boost customer satisfaction and increase revenue.
This study on sentiment analysis of customer feedback for KFC using machine learning techniques has notable limitations The raw data, sourced from Google reviews, may not accurately represent KFC's diverse clientele, as it tends to favor more vocal customers who express strong opinions This bias could lead to an imbalance in sentiment representation, ultimately affecting the overall accuracy of the model.
This study's sentiment analysis is limited as it excludes non-textual elements like star ratings, emojis, and multimedia content, which could offer valuable context and deepen the understanding of sentiment Incorporating these factors is essential, as they can significantly impact customer perceptions and contribute to a more thorough analysis.
The research emphasizes overall sentiment trends but does not explore specific elements of the customer experience, such as taste, service, or cleanliness While understanding general sentiment is important, it may miss vital insights that could guide targeted improvements for KFC Future studies should broaden their scope to include these aspects, offering a more detailed understanding of customer feedback.
Time constraints can significantly impact sentiment trends, which may evolve due to new marketing strategies, product launches, or external factors like economic conditions This study focuses on data collected from 2021 to 2024, highlighting the need for future research to utilize a more extensive dataset over a longer timeframe Incorporating diverse machine learning techniques could further strengthen the findings and provide deeper insights.
The machine learning models in this study have limitations that could affect their effectiveness, particularly due to the risks of overfitting or underfitting, especially in cases of imbalanced or non-diverse datasets This may result in models that perform poorly on unseen data or fail to accurately capture customer sentiment complexities Moreover, the generalizability of these models is restricted to KFC, as they may not produce similar outcomes in different industries or restaurant environments due to varying customer expectations and language nuances Future research should focus on validating these models in a wider array of settings to improve their applicability and robustness.
Structure of the report
This research has been divided into 4 main chapters
Chapter 1: Introduction includes background; objectives; scopes; and limitations
Chapter 2: Literature overview includes an overview of SA; SA model; research situation in Vietnam and globe; and the applications of SA in the field of fast-food
Chapter 3: Methodology consists data collection; imbalanced data handling; splitting train and test; vectorization; machine learning models and evaluation metrics used in this research
Chapter 4: Result and discussion includes a comparison of the result performance and sentiment distribution of customer feedback
LITERATURE REVIEW
Overview of sentiment analysis
In recent years, sentiment analysis, also known as opinion mining, has gained significant attention from the NLP research community worldwide This process involves detecting and analyzing text to categorize emotions such as positive, negative, or neutral, as well as specific feelings like happiness, sadness, anger, or disgust The primary goal is to assess people's attitudes toward a particular subject or entity.
Sentiment analysis (SA) is a crucial aspect of natural language processing (NLP) that plays a significant role in both academic research and various industries It is vital for understanding customer behavior and attitudes towards products and services SA systems enable business owners to gain valuable insights into real-time customer sentiment, enhance customer experience, and monitor brand reputation In the service sector, particularly within the restaurant and hotel industries, SA automates the analysis of user opinions, often sourced from social media and customer feedback platforms regarding quality and satisfaction.
Sentiment Analysis (SA) not only identifies the subject and opinion holder of a text but also determines its polarity, indicating the levels of positivity and negativity This technique is applicable for analyzing various textual components, such as paragraphs, sentences, subsentences, and complete documents.
Sentiment analysis is primarily divided into three levels: sentence level, document level, and aspect level The sentence level involves classifying sentences as objective or subjective and then labeling subjective sentences as positive or negative At the document level, the overall sentiment of a review is assessed, categorizing it as positive, negative, or neutral, particularly when a single author discusses a specific subject The aspect level, on the other hand, identifies features mentioned in a review and evaluates the sentiment towards those features This study will concentrate on sentiment classification specifically at the sentence level.
Figure 2.1: Types of Sentiment Analysis [1]
Figure 2.1: Task of Sentiment Analysis [2]
Sentiment analysis model
The sentiment analysis model has been summarized to this diagram as below:
The dataset for sentiment analysis is sourced from open-sourced Google Reviews, extracted from social media platforms and the Internet The preprocessing phase prepares the data by removing noise and irrelevant features that could hinder classifier performance, including unnecessary repetitions, emoticons, and extraneous characters Following this, feature selection is conducted to identify relevant characteristics from the cleaned data, which helps in reducing data dimensions and further eliminating noise Finally, the sentiment analysis process involves several key steps: tokenization, case transformation, removal of stop-words, creation of a bag of words, training the model using selected machine learning algorithms, and evaluating the model's performance.
This structured approach enables effective SA, leveraging machine learning techniques to derive meaningful insights from textual data.
Research situation
2.3.1 Research situation in the globe
Sentiment analysis, or opinion mining, originated in the early 2000s as researchers sought ways to extract subjective information from text The term "sentiment analysis" was first introduced by Nasukawa and Yi, while "opinion mining" was coined by Dave, Lawrence, and Pennock.
The pioneering study by Pang et al is recognized as the foundational research for opinion analysis, sparking increased attention and development in this field.
In the early development of sentiment analysis, researchers utilized machine learning and natural language processing (NLP) to categorize text as positive, negative, or neutral The rise of social media and online reviews significantly boosted this field, providing a wealth of user-generated content for analysis With advancements in deep learning, particularly through neural networks and transformer models like BERT, sentiment classification has become more accurate and context-sensitive Today, the Internet's rapid expansion encourages users to share their feedback on products and services across social media, e-commerce platforms, and online forums.
Sentiment analysis technology has emerged as a crucial component of artificial intelligence, significantly impacting various fields such as marketing, customer service, and public opinion analysis.
Global research on sentiment analysis is progressing swiftly, showcasing innovative methods and practical applications across diverse sectors Key elements of the current research landscape highlight the continuous evolution and significance of sentiment analysis in understanding public opinion and enhancing decision-making processes.
Large language models, including BERT, RoBERTa, GPT, and ChatGPT, have greatly improved sentiment analysis (SA) by enhancing accuracy and contextual comprehension These advanced models enable more sophisticated techniques, such as aspect-based sentiment analysis, moving beyond simple classification methods.
The increasing global demand for sentiment analysis has led researchers to develop multilingual models that can effectively analyze sentiments across various languages, even those with limited training data Notable examples of this trend include XLM-RoBERTa and similar models, which showcase advancements in multilingual sentiment analysis capabilities.
Aspect-based sentiment analysis (ABSA) is a key area of focus that allows systems to detect and evaluate sentiments concerning specific aspects of content, such as service quality, pricing, or ambiance, particularly in the restaurant industry.
The extensive data from social media platforms like Twitter, Facebook, Amazon, and Yelp has fueled research aimed at leveraging user sentiments for e-commerce, enhancing customer service, and conducting trend analysis.
+) Transfer learning: Techniques like fine-tuning large language models on domain-specific datasets are improving performance for various industries and contexts, making sentiment analysis more adaptable
+) Multimodel sentiment analysis: Researchers are exploring the integration of text with images, videos, and audio to understand overall sentiments from diverse data sources, expanding the scope of analysis
Sentiment analysis has become increasingly important in Vietnam, driven by the rise of user-generated content on social media, e-commerce sites, and review platforms Researchers and practitioners are concentrating on analyzing Vietnamese text data to derive valuable insights into user opinions across diverse areas, such as customer feedback, political discourse, and product evaluations.
The field of sentiment analysis in Vietnam is still evolving compared to global standards, focusing primarily on processing the Vietnamese language Researchers are dedicated to developing sentiment analysis systems that cater to the distinct features of Vietnamese, which include its tonal nature and complex word segmentation Additionally, the widespread use of informal writing styles on online platforms poses significant challenges for natural language processing (NLP) Overcoming these linguistic hurdles is essential for building effective sentiment analysis models.
The adoption of pretrained language models like PhoBERT and ViBERT has significantly enhanced sentiment analysis in Vietnam PhoBERT distinguishes itself with its SentencePiece tokenization, effectively managing the intricate word segmentation of the Vietnamese language without relying on predefined linguistic rules, making it adept at processing informal writing and slang Fine-tuned for various NLP tasks, PhoBERT showcases state-of-the-art performance in Vietnamese text processing, while ViBERT serves as a valuable alternative for researchers seeking different pretrained representations or model comparisons Both models have greatly advanced sentiment analysis, providing context-aware text representations that improve the accuracy and efficiency of sentiment evaluation in Vietnamese texts By leveraging these advanced models, researchers have successfully overcome many limitations of traditional rule-based and machine learning approaches.
Aspect-Based Sentiment Analysis (ABSA) is increasingly recognized in Vietnam, mirroring global advancements in the field This analytical approach emphasizes the detection of sentiments related to specific features of a product or service, including factors like food quality, customer service, and pricing.
18 approach is particularly relevant in domains like e-commerce, tourism, and restaurants, where customer feedback often targets multiple aspects of the experience
In Vietnam, the rise of popular social media platforms such as Facebook, Zalo, and TikTok has opened up new avenues for sentiment analysis Businesses are increasingly leveraging this analysis to gauge customer opinions, manage brand reputation, and extract valuable insights from user-generated content This approach has become especially beneficial for enhancing customer service, refining marketing strategies, and guiding product development.
Vietnamese sentiment analysis faces ongoing challenges, including the informal communication style prevalent online, which often includes slang, abbreviations, and code-switching between Vietnamese and English, making accurate sentiment detection difficult Furthermore, regional dialect variations affect word choice and sentiment, complicating natural language processing (NLP) systems A major hurdle is the scarcity of high-quality annotated datasets for Vietnamese, which hampers the development and training of advanced models.
Applications of sentiment analysis in the fast-food industry
Sentiment analysis serves as a valuable tool for understanding customer opinions and behaviors, providing actionable insights that can enhance business performance In the case of KFC, the applications of sentiment analysis are extensive and impactful.
KFC utilizes sentiment analysis to gain valuable insights from customer feedback on social media, delivery apps, and surveys This approach helps the company identify trends in customer satisfaction, allowing it to target specific areas for improvement in its menu, services, and overall experience, ultimately ensuring that customer needs are effectively addressed.
Brand reputation management is essential in the fast-food sector, where maintaining a positive image is vital Through sentiment analysis, KFC can effectively monitor online mentions and gauge public sentiment By promptly addressing negative feedback, the company can reduce potential reputation risks and assess the effectiveness of its marketing campaigns and public relations strategies.
Customer reviews play a crucial role in refining product development and innovation at KFC By utilizing sentiment analysis, the company can pinpoint popular menu items and tackle frequent complaints related to pricing and portion sizes This valuable feedback guides the creation of new menu offerings and improves the overall product lineup, ensuring it aligns with customer preferences.
Competitor analysis is essential for KFC to maintain its market position, as it enables the brand to evaluate its performance against rivals By utilizing sentiment analysis to assess customer reviews, KFC can pinpoint its strengths and weaknesses, which informs strategic decisions regarding pricing, promotions, and menu diversification This approach ultimately provides KFC with a competitive advantage in the industry.
KFC can enhance its marketing efforts by implementing personalized strategies based on sentiment segmentation, categorizing customers as positive, neutral, or negative By targeting dissatisfied customers with tailored promotions and offers, KFC can effectively re-engage them, while also rewarding loyal customers to sustain their positive feelings and encourage ongoing patronage.
Crisis management is crucial for KFC during service failures, product recalls, or public controversies, and sentiment analysis provides real-time feedback to the company This enables KFC to respond quickly, effectively address customer concerns, and rebuild trust, thereby minimizing potential damage to its reputation.
Regional preferences and cultural differences significantly influence customer expectations, making sentiment analysis essential for KFC By understanding local trends, KFC can customize its menu and marketing strategies to cater to the distinct needs of customers in various locations.
Customer feedback plays a crucial role in evaluating employee performance at KFC, particularly for frontline staff By utilizing sentiment analysis, KFC can pinpoint common themes in service experiences, whether positive or negative This valuable insight not only informs employee training programs but also helps recognize and reward outstanding staff members.
By leveraging sentiment analysis in these areas, KFC can maintain a strong customer focus, adapt to market trends, and enhance its competitive standing in the global fast- food industry
METHODOLOGY
Data collection
This research utilizes a dataset sourced from Google Reviews and Google Maps, encompassing comments from 23 KFC restaurants in Hanoi, collected between 2022 and October 2024 The raw data was gathered using the tool "Instant Data Scraper" to facilitate efficient data collection from the Internet.
The dataset comprises over 10,140 entries, featuring six key attributes: reviewer name, comments, service rating, food rating, atmosphere rating, and an average rating Additionally, the dataset is presented in Vietnamese.
Figure 3.1: The table of raw dataset
Preprocessing is a crucial initial step in data analysis, especially for Vietnamese datasets, which present unique challenges such as tonal variations, word segmentation, and informal language in online content Proper preprocessing is essential for enabling machine learning models to accurately understand and classify sentiments.
To effectively prepare data for sentiment analysis, it's essential to follow specific steps Utilizing NLTK, a popular library for text preprocessing, enables tasks such as word segmentation, normalization, stop-word removal, tokenization, and lemmatization Prior to these processes, it's crucial to check the dataset for any null or duplicate values.
Any values that are not valid will be removed from the dataset The dataset consists of three attributes: service, food, and atmosphere ratings The label for the data will be determined by calculating the average of these three rating values, and it will be temporarily classified accordingly.
In sentiment analysis, labels are assigned based on average values, with "Negative" for scores below 3, "Positive" for scores above 4, and "Neutral" for scores in between Vietnamese text requires careful word segmentation due to its lack of spaces between words, making tools like VnCoreNLP, RDRSegmenter, and PyVi essential for breaking phrases into meaningful tokens For example, "Tôiyêuthíchchươngtrìnhnày" is segmented into "Tôi yêu thích chương trình này." Additionally, data normalization is performed to standardize different forms of expressions, including converting abbreviations and addressing slang terms, such as changing ‘ko’/’khum’ to ‘không’ and ‘so cute’ to ’dễ thương’ Furthermore, less significant words in the dataset are removed to enhance analysis efficiency.
During the preprocessing stage, text is converted to lowercase to ensure uniformity, while tokenization aids models in understanding sentence structure Additionally, lemmatization reduces words to their base forms, enhancing analysis efficiency To prevent interference with analysis, noise elements such as emojis, hashtags, and punctuation are removed during this crucial step.
Figure 3.2: The code for data normalization
Figure 3.3: The code for emoji removing
Figure 3.4: The code for data preprocessing (1)
Figure 3.5: The code of data preprocessing (2)
Figure 3.6: The result of data after preprocessing
The raw data with 10143 data rows, after preprocessing process, has been reduced to
9985 data rows and it is ready to transfer to the next step in the data analysis
Exploratory Data Analysis (EDA) involves using statistical and visualization techniques to describe data, highlighting key aspects for further examination Essential libraries such as Sklearn, NumPy, and Pandas facilitate data loading and transformation, while Seaborn and Matplotlib are employed for plotting and visualizing the data Although the semantic meaning may sometimes be secondary, this approach remains an effective strategy for data analysis.
Figure 3.7: The image of statistical results depicting data fields
Figure 3.8: The image of counting words in the attribute "Comment"
Figure 3.9: Word cloud of data before preprocessing
Figure 3.10: Word cloud after preprocessing
Figure 3.11: The pie chart shows the number of labels in the dataset
Vectorization
TF-IDF (Term Frequency-Inverse Document Frequency) is a key technique in natural language processing that evaluates the significance of words in a document By transforming textual data into numerical vectors, it measures word importance within a document compared to an entire corpus This method effectively identifies essential terms that aid in understanding the sentiment of reviews or comments.
This vectorization measures the number of occurrences of a given word t in document d.: tf(t,d) = count of t in d / number of words in d
Inverse Document Frequency (IDF) is a metric that emphasizes words that are distinctive to a specific document by reducing the weight of terms that appear frequently across multiple documents It is calculated using the formula: idf(t) = log(N/df(t)), where N represents the total number of documents and df(t) indicates the number of documents that contain the term t This approach helps in identifying unique keywords that enhance the relevance of a document in search engine optimization (SEO).
TF-IDF is a key metric for evaluating the significance of a term within a text or corpus It assigns a weight to each word based on its term frequency (tf) and the reciprocal of its document frequency (idf) Words with higher TF-IDF scores are deemed more important, reflecting their relevance in the context The TF-IDF score is calculated using the formula: tf-idf(t, d) = tf(t, d) * idf(t).
Imbalanced data
Imbalanced data is a prevalent issue in machine learning that can negatively impact feature correlation, class distinction, model evaluation, and overall performance Most datasets experience this imbalance, which often arises during the collection and surveying of user information.
Imbalanced datasets pose significant challenges in machine learning, primarily due to the model's bias towards the majority class, which often leads to neglect of the minority class and poor performance for underrepresented categories Misleading metrics, such as accuracy, can create a false impression of model effectiveness, as a model may achieve high accuracy by simply predicting the majority class Furthermore, inadequate learning of patterns for minority classes results in poor generalization, causing suboptimal performance on unseen data Addressing class imbalance is crucial for ensuring fair and accurate predictions in machine learning models.
To effectively address imbalanced data, it is crucial to implement techniques that enhance model performance across all classes Popular data-level manipulation strategies, such as undersampling and oversampling, aim to achieve a balanced dataset Additionally, the SMOTE technique generates a more balanced dataset by creating synthetic samples rather than merely duplicating existing ones.
Algorithm-level adjustments, such as assigning class weights, enhance the learning process by prioritizing errors from minority classes, leading to improved predictions for underrepresented categories Ensemble methods like Random Forest and XGBoost incorporate built-in mechanisms to address class imbalance, making them effective choices for imbalanced datasets By strategically combining these techniques, practitioners can improve model fairness and accuracy, ensuring better generalization across all classes.
In this research, SMOTE has been used to deal with the problem of imbalanced data The dataset has 3 labels: 2 (Positive), 1 (Neutral), and 0 (Negative) as below:
Figure 3.12: Checking the number for each label
In this dataset the minor labels are 0 and 1 with the total number for each label are
2756 and 1217, respectively [5], [6] The pair of points x_i and x_j is closest to the minimum class x_j is sampled from the k-neighbors of x_i Then, a new instance x = t
* x_i + (1-t) * x_j is created The process continues to iterate until the number of samples reaches the oversampling rate threshold (hyperparameter)
SMOTE is a powerful technique for addressing imbalanced datasets, offering benefits such as preventing overfitting by generating synthetic samples that promote diverse learning patterns By balancing class distribution, SMOTE enhances model performance for minority classes, allowing better understanding of underrepresented categories Its ease of implementation makes it a popular choice in preprocessing pipelines However, SMOTE has drawbacks, including the risk of class overlap, which can lead to misclassification, and it may be computationally demanding for large or high-dimensional datasets Additionally, its effectiveness diminishes with categorical or mixed data types, necessitating careful tuning for optimal application.
After using SMOTE in this research dataset, the number of minor labels (label 1 and label 0) has been increased equally to the largest label (label 2)
Figure 3.13: A number of each label before SMOTE
Figure 3.14: A number of each label after SMOTE
Splitting data
The training and evaluation process involves preprocessing the dataset, splitting it into training and testing sets, and assessing the model's performance in sentiment classification Proper structuring of this phase is crucial for enabling the model to accurately interpret and analyze sentiments.
Sentiment analysis datasets commonly consist of text data, such as customer reviews, categorized into classes like positive, negative, or neutral To prepare this data for model input, preprocessing tasks, including extraction techniques like TF-IDF vectorization, are utilized.
In this study, the dataset has been splited into training and testing subsets using a 75:25 ratio
Figure 3.15: The code for splitting data
During the training phase, machine learning algorithms examine training data to recognize patterns and refine parameters Various techniques, including Logistic Regression, Support Vector Classification (SVC), Multinomial Naive Bayes, and advanced models like phoBERT, can be utilized for sentiment analysis The choice of model is influenced by factors such as dataset size, linguistic characteristics, and task complexity.
After training is finished, the model's performance is evaluated using a specific testing set To ensure a balanced assessment across all sentiment categories, especially in cases of imbalanced datasets, weighted or macro-averaged metrics are typically utilized.
Machine learning techniques used
Logistic regression is a widely utilized statistical method for classification and prediction, primarily focusing on forecasting discrete categorical outcomes This technique is designed to handle outputs that are categorical or discrete, resulting in binary outcomes such as "Yes" or "No," 1 or 0, and "True" or "False."
Logistic regression is primarily classified into three types, with the first being binary logistic regression This statistical method models the relationship between a binary dependent variable—having two possible values such as 0 or 1, "Yes/No," "True/False," or "Success/Failure"—and one or more independent variables It is commonly employed to predict the likelihood of an event happening or not, based on the influencing independent variables.
Multinomial logistic regression is utilized when the dependent variable consists of three or more unordered categories This model is particularly useful for analyzing factors that affect individuals' transportation preferences, such as the choice between a car, bicycle, or public transit, taking into account variables like age, income, and commuting distance.
Lastly, ordinal logistic regression deals with dependent variables that have three or more ordered categories Examples include rankings such as "Low," "Medium," and
"High," where the sequence of values carries meaningful significance
Here is a logistic regression formula:
Figure 3.16: The formulation of Logistic Regression
Naive Bayes is a classification algorithm grounded in Bayes' Theorem, renowned for its efficiency and effectiveness, especially with large datasets It is commonly utilized in applications such as spam detection, sentiment analysis, and document classification, making it ideal for managing high-dimensional data Among the various types of Naive Bayes classifiers, Gaussian Naive Bayes is the first and most notable variant.
Naive Bayes classifiers are effective for different types of data The Gaussian Naive Bayes is ideal for continuous data, calculating feature likelihoods based on mean and variance, making it useful for predicting house prices In contrast, Bernoulli Naive Bayes is designed for binary features, treating each feature as an indicator of presence or absence, and is commonly used in text classification tasks, such as identifying spam emails based on specific keywords Lastly, Multinomial Naive Bayes is suitable for discrete data, modeling feature distribution through counts or frequencies, and is frequently applied in text classification scenarios involving word counts in documents.
This study utilizes Multinomial Naive Bayes for sentiment analysis, demonstrating its effectiveness on datasets characterized by discrete word frequencies or event counts Its capabilities make it a strong choice for a range of natural language processing (NLP) applications.
In text classification, feature vectors are commonly represented by word counts or term frequencies, often utilizing the "bag of words" method The multinomial distribution plays a crucial role in assessing the probability of specific word occurrence combinations within a document, offering valuable insights into word patterns and their impact on classification outcomes.
Multinomial Nạve Bayes predicts labels by analyzing how often specific features appear within the data The conditional probability formula used is as follows:
Figure 3.17: The formulation of conditional probability
+) P(C/X) – posterior probability: Probability that text X in the sentiment label C (positive, neutral and negative)
+) P(C) – class prior probability: Prior probability of class C (based on the frequency of the class in the training set)
+) P(X/C) - likelihood: The conditional probability of words in X being in class C, is calculated using the word frequencies in the review
+) P(X): The marginal probability of the data X in all labels It is used as a normalizing constant to ensure probabilities sum to 1
The likelihood P(X∣C) is calculated as the product of the conditional probabilities of individual words in the review:
Figure 3.18: The formulation of likelihood
The conditional probability of each word given a class is calculated as:
Figure 3.19: The formulation of conditional probability
+) Count(wi,C): Count the word wi in class C
+) Count(C): Total number of words in class C
+) α\alpha: Smoothing parameter (Laplace smoothing, usually α=1\alpha = 1α=1) to handle words not present in the training data
+) ∣V|: Size of the vocabulary (total number of unique words in the dataset)
By adding 1 to each data, the data with 0 occurrence in the dataset will give probability value and provide likelihoods for future process
The final step involves predicting the test dataset by calculating the probabilities for each class The sentiment of the review is determined by selecting the class with the highest probability, following a specific decision rule.
Multinomial Naive Bayes is an efficient and straightforward method for sentiment analysis, utilizing word frequencies and probabilistic calculations to classify customer feedback effectively Despite its assumption of word independence, which may not always apply, MNB serves as a strong baseline for sentiment analysis, providing valuable insights into customer perceptions of KFC's food and service.
The Support Vector Classifier (SVC), a key component of Support Vector Machines (SVM), excels in classification tasks by effectively managing high-dimensional data This model can transform text data into sentiment classifications such as positive, negative, or neutral It is particularly advantageous for datasets lacking clear class boundaries, as it focuses on maximizing the margin between distinct classes.
The choice of Support Vector Classifier (SVC) for sentiment analysis is primarily due to its specialization in classification tasks, offering practical benefits for analyzing KFC review data Unlike Support Vector Machines (SVM), which cater to both classification and regression, SVC is designed specifically for classification, enhancing its efficiency in this context Additionally, SVC streamlines the implementation process by focusing solely on the classification task, avoiding unnecessary complexities Its effectiveness in text classification is particularly notable, as it adeptly manages high-dimensional data typical of text inputs.
Figure 3.20: The formulation of decision rule
Support Vector Classification (SVC) utilizes numerical representations such as Bag of Words and TF-IDF to effectively analyze sentiment in textual data By employing kernel functions like RBF and Polynomial, SVC effectively handles non-linear data, which is crucial for capturing the often ambiguous emotional boundaries in sentiment analysis Optimized in libraries like scikit-learn, SVC offers straightforward deployment and resource-efficient processing, making it a practical choice over other SVM variants like SVR This focus on SVC ensures precision and relevance in classifying emotions, enhancing clarity in sentiment analysis.
Support Vector Classification (SVC) identifies a hyperplane in an n-dimensional feature space that effectively separates different classes The primary objective of Support Vector Machine (SVM) is to maximize the margin between this hyperplane and the closest data points, known as support vectors, from each class.
Next, the mathematic behind SVC is rooted in linear algebra and optimization:
Linear SVC: For linearly separable data, SVC identifies a hyperplane that separates sentiment classes (e.g., positive, negative, neutral)
Non-linear Support Vector Classification (SVC) utilizes kernels to transform data into a higher-dimensional space when linear separation is not feasible For instance, when positive and negative sentiments overlap in a two-dimensional space, the Radial Basis Function (RBF) kernel can effectively map these sentiments into a separable space.
Hyperplane equation: In a binary classification problem, the hyperplane is used to separate the data points of two classes
W*x+b=0 +) W: the weight of the hyperplane
Margins refer to the distance between the hyperplane and the closest data points from each class, where a larger margin indicates greater confidence in classification This margin can be quantified as the distance between two parallel hyperplanes, each representing a different class Mathematically, the margin is inversely proportional to the norm of the weight vector w.
Figure 3 : The hyperplane of SVM [12]
Support Vectors find the nearest data points to the hyperplane These points define the margin and are critical for classification
[13] PhoBERT is built on BERT and is a pre-trained model specifically for
Evaluation Metrics
Evaluation metrics are really necessary since they measure the effectiveness of machine learning models Below are some of the most commonly used evaluation metrics across various machine learning applications:
[14] Classification Metrics predict the class labels given input data In this research is a multiclass classification, so three labels that can be presented
Accuracy: It simply measures the classification model exactly predicts the result It can be calculated as the total true predictions divided by the total number of predictions
To effectively assess the performance of a classification problem involving three labels, it is essential to utilize a confusion matrix A 3x3 confusion matrix can be structured to represent the outcomes for classes A, B, and C, providing a clear visual of the classification results.
+) True positive (TP): is an outcome where the model correctly predicts the positive class
+) False positive (FP): is an outcome where the model incorrectly predicts the positive class
+) True negative (TN): is an outcome where the model correctly predicts the negative class
+) False negative (FN): is an outcome where the model incorrectly predicts the negative class
Precision is also called the precision of the classifier It is calculated by using the formula as below:
Recall is defined as the ratio of true positive points to those that are actually positive (TP + FN) It can be calculated by this formula:
However, using only precision or recall can not examine the model performance So, F1-score is calculated by using recall and precision:
Figure 3.24: The formulation of precision
Figure 3.25: The formulation of recall
Figure 3.26: The formulation of F1-Score
3.6.3 Measuring accuracy using Cross-validation
Cross-validation is a statistical method employed to evaluate the effectiveness of machine learning models by repeatedly dividing the dataset into training and validation sets Its main objective is to reduce overfitting and enhance the model's ability to generalize to new, unseen data.
Accuracy in Cross-validation: It measures the proportion of correctly classified class:
During cross-validation, accuracy is assessed for each fold, with the overall estimation typically derived from the average of these results The process begins by splitting the dataset into k folds, such as k=5 One fold serves as the validation set while the remaining k−1 folds are used for training Accuracy is then calculated based on the validation set Finally, after evaluating all k-folds, the individual accuracies are averaged to determine the overall cross-validation accuracy.
Figure 3.28: Formulation for Cross-validation Accuracy Figure 3.27: The formulation of accuracy in Cross-validation
RESULTS AND DISCUSSION
Model Performance
This section analyzes the performance metrics of the models used in this research, focusing on key evaluation metrics to compare their effectiveness It also highlights the results of the cross-validation process and the impact of addressing imbalanced data through techniques such as SMOTE.
Before running these models, the KFC dataset needs to preprocess, handle the issue of imbalanced data, and vectorization as mentioned in the “Chapter 3: Methodology”
Figure 4.1: The code for Logistic Regression model (1)
Figure 4.2: The code for Logistic Regression (2)
Confusion matrix of Logistic Regression:
Figure 4.3: Confusion matrix of Logistic Regression
Figure 4.4: Classification report for Logistic Regression
Figure 4.5: The code for Multinominal NB (1)
Figure 4.6: The code for Multinomial NB (2)
Confusion matrix of Multinomial Nạve Bayes:
Figure 4.7: Confusion Matrix of Multinomial NB
Figure 4.8: Classification report of Multinomial NB
Figure 4.9: The code for SVC (1)
Figure 4.10: The code for SVC (2)
Figure 4.11: The code for SVC (3)
Figure 4.12: Confusion matrix of SVC
Figure 4.13: The code for phoBERT (1)
Figure 4.14: The code for phoBERT (2)
Figure 4.15: The code for phoBERT (3)
Figure 4.16: Confusion matrix of phoBERT
Figure 4.17: Classification scores of phoBERT
Result
4.2.1 Comparision result performance between models
Figure 4.18: Table comparing the result performance between models
Figure 4.19: The chart of comparing Accuracy between models
Figure 4.20: The chart comparing the Precision between models
Figure 4.21: The chart comparing Recall between models
Figure 4.22: The chart comparing the F1-Score between models
The PhoBERT model stands out as the top algorithm for sentiment analysis on the KFC review dataset, achieving an impressive accuracy of 90.64% Although its precision of 85.43% is slightly lower than that of Logistic Regression, PhoBERT excels in balancing precision and recall with a score of 83.31%, resulting in a robust F1-score of 84.27% This performance underscores PhoBERT’s effectiveness in comprehending the intricacies of Vietnamese text, thanks to its specialized deep learning architecture.
Logistic Regression achieved an accuracy of 88.09%, ranking second in performance It excelled in precision with a score of 89.12%, indicating its reliability in identifying positive sentiments Additionally, its recall of 88.12% and F1-score of 88.1% showcased a strong balance across key metrics This makes Logistic Regression a suitable option for simpler applications that prioritize computational efficiency.
The Support Vector Classifier (SVC) achieved an accuracy of 87.20%, placing it in third position among classifiers With a precision of 88.17% and a recall of 87.25%, SVC remains competitive, albeit slightly behind Logistic Regression Its F1-score of 87.24% demonstrates its reliability, making SVC a strong choice for datasets that feature well-defined class boundaries.
Multinomial Naive Bayes (MNB) achieved a fourth-place ranking with an accuracy of 81.44% While it demonstrated decent performance in precision at 82.17% and recall at 81.42%, its F1-score of 81.54% indicates challenges in effectively capturing the subtleties of Vietnamese sentiment analysis when compared to more advanced models.
Overall, PhoBERT's superior performance underscores its suitability for complex tasks, while Logistic Regression and SVC offer reliable alternatives for simpler scenarios
4.2.2 Testing an example on each model
Figure 4.23: Case study of Logistic Regression testing
Figure 4.24: Case study of SVC testing
Sentiment Distribution of Customer Feedback
Word clouds were created for positive, neutral, and negative reviews to clarify key terms related to customer sentiments These visual representations showcase the most commonly used words in each category, providing essential insights into customer opinions regarding KFC restaurants.
Figure 4.25: Word cloud of positive reviews
Positive reviews frequently highlight key terms such as "service," "friendly," "clean," and "delicious," showcasing that customers value high-quality service, a welcoming atmosphere, cleanliness, and great food taste These words represent the primary factors contributing to customer satisfaction.
Figure 4.26: Word cloud of neutral reviews
The neutral review word cloud is dominated by terms such as "không" (not), "được"
(received), and "bình thường" (normal) These words suggest that customers in this category had average or mixed experiences, neither particularly positive nor negative
Figure 4.27: Word cloud of negative reviews
For negative reviews, prominent words include "nhân viên" (staff), "thái độ" (attitude),
The terms "food" and "customer" highlight dissatisfaction related to staff behavior, food quality, and customer service The recurring mention of "disappointed" emphasizes a prevailing negative sentiment among patrons.
Analyzing word clouds reveals key strengths and weaknesses in customer experiences at KFC Positive feedback highlights strengths in food and service, whereas negative and neutral reviews point out areas needing improvement, such as staff attitudes and food quality These insights are crucial for KFC to enhance customer satisfaction and effectively address common complaints.
This study explored sentiment analysis of customer reviews for KFC restaurants in Hanoi, Vietnam, with the goal of improving customer satisfaction and business strategies Various models, including Logistic Regression, Multinomial Naive Bayes, Support Vector Classifier (SVC), and PhoBERT, were evaluated for their performance Notably, PhoBERT outperformed the other models, achieving an impressive accuracy of 90.64%, demonstrating its effectiveness in processing Vietnamese text data.
The analysis of customer sentiment distribution identified significant trends in positive, neutral, and negative reviews Positive feedback emphasized food quality, friendly staff, and cleanliness, whereas negative reviews often focused on staff attitudes, service quality, and particular elements of the dining experience Additionally, word clouds visually represented these sentiment trends, enhancing the understanding of factors that shape customer opinions.
To tackle issues like imbalanced data, the study employed SMOTE (Synthetic Minority Oversampling Technique), enhancing the model's accuracy in classifying rare sentiment categories Furthermore, it highlighted the significance of utilizing pre-trained models such as PhoBERT and advanced feature extraction methods like TF-IDF to improve overall effectiveness.
The study highlights the significance of sentiment analysis in converting customer feedback into actionable insights, allowing KFC and similar companies to enhance service quality, tackle customer issues, and fine-tune strategic choices However, it also recognizes limitations such as dataset specificity, computational complexity, and a sole focus on textual feedback, which future research should seek to address.
Implications for KFC and the Fast-Food Industry
The findings of this thesis have important implications for KFC and the fast-food industry by introducing a user-friendly interface powered by the PhoBERT model This system allows customers to provide feedback, enabling real-time sentiment analysis to categorize comments as positive, neutral, or negative By efficiently processing large volumes of feedback, KFC can quickly identify sentiment trends and address negative comments promptly, enhancing customer satisfaction Furthermore, the insights gained from sentiment predictions can inform strategic decisions, helping the company focus on key areas for improvement.
KFC is frequently praised by customers for its food quality and effective resolution of service-related issues This scalable approach can be applied to other fast-food chains, enhancing customer experience management across the industry and improving competitive positioning By utilizing advanced sentiment analysis, KFC can convert customer feedback into actionable insights, which fosters stronger customer relationships and drives business growth.
The research focuses on utilizing the PhoBERT model to create an intuitive interface for sentiment analysis By incorporating PhoBERT into web or mobile platforms, fast-food chains like KFC can allow customers to submit feedback directly The system will analyze the input and provide real-time sentiment predictions—categorizing comments as positive, neutral, or negative—thus offering immediate insights into customer opinions.
The interface of front-end web (1)
The interface of front-end web (2)
This study provides valuable insights into sentiment analysis of customer feedback in the fast-food industry, particularly focusing on KFC reviews in Hanoi, Vietnam However, its findings may not be generalizable to other regions or industries due to the specificity of the dataset Additionally, the research primarily relies on text-based feedback, overlooking other forms of data, such as images or videos, that could offer further context regarding customer opinions Furthermore, while the study emphasizes sentiment prediction, it does not explore the underlying causes of customer satisfaction or dissatisfaction, which could yield deeper insights for enhancing business strategies.
Future research should focus on addressing current challenges in sentiment analysis to improve its scope and applicability By expanding datasets to include reviews from various regions, industries, and languages, researchers can develop more generalized models Additionally, exploring multimodal sentiment analysis through the integration of images, videos, and audio recordings with textual data will enhance the understanding of customer sentiment Simplifying advanced models like PhoBERT to lower computational requirements will facilitate their use in real-time and large-scale scenarios Furthermore, incorporating aspect-level sentiment analysis with customer behavior prediction can yield actionable insights in areas such as service quality, pricing, and menu preferences, ultimately enhancing the effectiveness of sentiment analysis across diverse contexts.
[1] Marouane Birjali, Mohammed Kasri, Abderrahim Beni-Hssane, “A comprehensive survey on sentiment analysis: Approaches, challenges and trends” Available at: https://www.sciencedirect.com/science/article/abs/pii/S095070512100397X
[2] Mayur Wankhade, Annavarapu Chandra Sekhara Rao, “A survey on sentiment analysis methods, applications, and challenges” Available at: https://link.springer.com/article/10.1007/s10462-022-10144-1
[3] Erick Odhiambo Omuya, George Okeyo, Michael Kimwele, “Sentiment analysis on social media tweets using dimensionality reduction and natural language processing” Available at: https://onlinelibrary.wiley.com/doi/10.1002/eng2.12579
[4] “Understanding TF-IDF (Term Frequency-Inverse Document Frequency)” Available at: https://www.geeksforgeeks.org/understanding-tf-idf-term-frequency-inverse-document- frequency/
[5] Nitesh V Chawla, Kevin Bowyer, “SMOTE: Synthetic Minority Over-sampling technique” Available at: https://www.researchgate.net/publication/220543125_SMOTE_Synthetic_Minority_Ov er-sampling_Technique
[6] Turintech, “What is imbalanced data and how to handle it?” https://www.turintech.ai/what-is-imbalanced-data-and-how-to-handle-it/
I don't know!
I don't know!
[9] GreeksforGreeks, “Multinomial Naive Bayes” https://www.geeksforgeeks.org/multinomial-naive-bayes/
[10] Dishant Salunke, “SVC-Support Vector Classifier” Available at: