Development of a generic NIRS calibration pipeline using deep learning and model ensembling application to some reference datasets Phát triển một đường ống hiệu chuẩn NIRS chung sử dụng ứng dụng học sâu và mô hình tổng hợp cho một số tập dữ liệu tham chiếu
Introduction
In this chapter, we will provide an overview of the study framework of the IFI (International Francophone Institute) and discuss the hosting structure for this internship, which is based at the AGAP research unit (Genetic Improvement and Adaptation of Mediterranean and Tropical Plants) of CIRAD (Centre for International Cooperation in Agricultural Research for Development).
Présentation du cadre d’étude : IFI
Objectifs
Étant une école de renommée en informatique, l’IFI a pour principaux objectifs de :
Equip students with knowledge in artificial intelligence, machine learning, deep learning, image and video processing, computer vision, modeling and simulation of complex systems, and data mining.
— former les étudiants aux méthodes et à la pensée de la recherche scientifique,pour être capables de résoudre indépendamment les problèmes techniques.
Conditions d’accès et mode de recrutement
Licence (BAC +3) en informatique ou dans une spécialité proche (Mathématiques, Phy- sique).
To meet the language requirements, candidates must demonstrate proficiency in French at least at the DELF B2 level, according to the Common European Framework of Reference for Languages, or possess an equivalent qualification such as a TCF score of 400 Exceptions apply for candidates from countries where French is the primary or official language, as well as for those who already hold a university degree in French.
— Mode de recrutement : évaluation du dossier et entretien de recrutement.
Organigramme de l’IFI
L’Institut Francophone International est constitué d’une direction et de divisions.
La direction s’appuie sur une Direction, un conseil scientifique, des services, des labo- ratoires et presse et des centres comme l’indique plus clairement l’organigramme1.2 ci-dessous.
Formations
The IFI offers four specialized programs for its training: Intelligent Systems and Multimedia, Communication Systems and Networks, a Master's in Banking, Finance, and Fintech, and a Master's in Information and Communication with a focus on Digital and Editorial Communication.
1.2.4.1 Parcours systèmes intelligents et multimédia
The Master's program in Computer Science with a focus on Intelligent Systems and Multimedia integrates knowledge from various fields, including modeling and simulation, artificial intelligence, data mining, human-computer interaction, and software engineering This program aims to design and develop intelligent decision-support systems that leverage multimedia information It is tailored to enhance students' scientific and intellectual capabilities, preparing them to meet the growing innovations in science and technology, particularly in the context of the Fourth Industrial Revolution.
1.2.4.2 Parcours systèmes et réseaux communicants
The Master's program in Computer Science with a focus on Communication Networks and Systems integrates knowledge from various fields, including advanced networking, cloud storage, virtualization, network security, and wireless and mobile networks The curriculum is designed to develop and build sustainable communication platforms that provide seamless and continuous service It aims to enhance students' scientific, intellectual, and visionary potential in response to ongoing innovations in science and technology, particularly in the context of the Fourth Industrial Revolution.
1.2.4.3 Master en Banque, Finance et Fintech
This program aims to equip students with in-depth and up-to-date knowledge in the banking and financial sector in the context of Industry 4.0 The Fintech program, a collaboration between IFI and EM Normandie, is the first of its kind in Vietnam and the second in Asia It features contributions from international educators and experts with extensive training and consulting experience from leading global financial markets and Fintech centers.
1.2.4.4 Master en Information - Communication, Spécialité Communication digi- tale et éditoriale
The Master's program in Information and Communication, specializing in Digital and Editorial Communication, is a collaborative initiative between the University of Toulon and the Francophone International Institute (Vietnam National University in Hanoi), funded by the Agence Universitaire de la Francophonie (AUF) Its primary goal is to train specialists in communication, utilizing advanced information technology techniques.
Assist French-speaking graduates in Humanities and Social Sciences who aim to work in communication by providing them with essential knowledge in digital and editorial communication, as well as the technical skills needed in this field.
- aider les diplômés en sciences plus précisément en sciences technologiques à acqué- rir des connaissances complémentaires de communication ;
- aider les diplômés en Journalisme-Éditation, les correspondants et les éditeurs à mo- derniser leur procédure professionnelle [8].
Structure d’accueil
Organisation du CIRAD
Le Cirad (Centre de coopération Internationale en Recherche Agronomique pour le Développement) is a public scientific establishment under the dual supervision of the Ministry of Higher Education, Research and Innovation, and the Ministry of Europe and Foreign Affairs Its work focuses on both life sciences and social sciences.
2 https://www.cirad.fr/qui-sommes-nous/le-cirad-en-bref et des sciences de l’ingénieur appliquées à l’agriculture, à l’alimentation, à l’environ- nement et à la gestion des territoires Il travaille autour de grandes thématiques telles que la sécurité alimentaire, le changement climatique, la gestion des ressources natu- relles, la réduction des inégalités et la lutte contre la pauvreté Le Cirad emploie 1650 personnes, dont 800 chercheurs Il comprend 3 départements scientifiques et 33 unités de recherche La figure1.3ci-dessous illustre l’organigramme du CIRAD.
Mission du CIRAD
The Cirad collaborates with partners in the Global South to generate and disseminate new knowledge that fosters agricultural innovation and development It leverages its scientific and institutional expertise to support public policies in these countries and engage in international discussions on major agricultural challenges Additionally, Cirad contributes to France's scientific diplomacy efforts.
Le développement par la recherche
Le Cirad aims to build a sustainable agriculture system that can adapt to climate change and feed 10 billion people by 2050 while protecting the environment It emphasizes the importance of societal participation in knowledge production to develop long-term strategies and appropriate public policies The success of this research-driven development relies on countries establishing effective higher education and research systems, supported by public authorities that allow for real autonomy Through sustainable partnerships, Le Cirad contributes to agriculture that benefits all, especially small farmers who make up the majority of producers This initiative addresses global challenges related to food security, climate change, and aligns with the 17 Sustainable Development Goals (SDGs) of the UN and the Paris Agreement on climate change.
La formation et le partage des connaissances
The Cirad's mission encompasses not only research but also the training, dissemination of information, and sharing of knowledge and innovations, empowering partners and development actors to make informed choices In the Global South, both degree and professional training are vital, relying on the commitment of each researcher Additionally, Cirad plays a significant role in fostering dialogue between Europe and Africa, actively participating in numerous European and international networks to enhance access for Southern partners to community programs and facilitate their integration into international scientific cooperation networks.
Partenariats
Cirad has established a network of partners across three continents, collaborating with over 100 countries through its regional offices Its long-term partnerships are organized through research and teaching initiatives involving 200 institutions in the Global South, with 200 researchers allocated to these projects—50% in Africa, 25% in Asia, and 25% in South America In France, Cirad provides significant research and training resources primarily based in Montpellier and French overseas territories Additionally, Cirad is a member of French consortiums such as Agreenium/IAVFF, Montpellier University of Excellence (I-site Muse), and national research alliances like AllEnvi, Avieasan, and Ancre.
Projet RTFoods
The RTBfoods project, launched in Buea, Cameroon from January 23 to 28, 2018, aims to identify quality traits that influence the adoption of new varieties of roots, tubers, and cooking bananas (RTB) across five African countries: Benin, Cameroon, Côte d'Ivoire, Nigeria, and Uganda With a funding of €11.5 million over five years, the project employs an innovative and participatory methodology that engages consumers, processors, and researchers.
Contexte et problématique du projet
Contexte du sujet
This internship focuses on the RTB Foods project, which aims to facilitate the adoption of new varieties of roots, tubers, and cooking bananas in Africa while ensuring product quality To assess this quality, Cirad has chosen a cost-effective, high-precision, and high-throughput technique known as Near-Infrared Spectroscopy (NIRS) This method predicts product content by combining spectral information with laboratory data through an appropriate predictive model based on chemometric methods Chemometrics employs statistical and mathematical techniques to interpret near-infrared spectra, involving multiple preprocessing steps (such as multi-scattering correction) and model calibration However, the complexity of this technique lies in handling a large number of explanatory variables, its sensitivity to the physical characteristics of the sample (like flour particle size and moisture), and the redundancy of information.
Aussi, le nombre de pré-traitements de spectres et de types de modèles disponibles augmente constamment et diffère entre les études et les analytes d’une même étude
The selection of pre-treatment methods and sample presentation, such as fresh, dried, or ground samples, is becoming increasingly complex, despite existing knowledge on starch, carotenoids, and cyanide.
In many high-performing data analysis methods, model ensemble techniques yield better results than the best individual model identified (van de Laar et al 2007) The idea of combining predictive models to enhance collective performance has gained significant traction Prominent approaches include bagging, boosting, and stacking, which rely on applying either (i) the same learning algorithm to various data variations (e.g., weighting observations) to create a sufficiently heterogeneous pool of models or (ii) a diversity of algorithms to generate the base models Model ensemble techniques, such as bagging and boosting, involve combining multiple "weak learners" to form a "strong learner," effectively creating a more efficient and satisfactory meta-model from algorithms with lower individual performance This process entails the successive application of these weak learners to estimate a target variable.
3 https://blog.octo.com/les-methodes-ensemblistes-pour-algorithmes-de-machine-learning/ modèles résultants de l’assemblage permettent l’obtention d’une meilleure précision. Dans le cas du stacking par contre, on recherche des learners (modèles) performant mais prộsentant des diversitộs que l’on assemble De faỗon plus tangible, l’assemblage de modèle dans le cadre de ce stage permettra d’avoir :
— une diversité des combinaisons de pré-traitements, des modèles et des analytes ;
— d’avoir un modèle fiable en termes de robustesse quel que soit le type et la taille du jeu de donnée.
There are limited metrics available to determine the base models that contribute to diversity in the meta-model The overall effectiveness relies on the individual performance and heterogeneity of these models, akin to the concept of decorrelation in random forests Multi-Agent Systems (MAS) can identify various combinations of base models to create a high-performing meta-model This multi-agent modeling approach allows for the distribution of skills and knowledge among autonomous entities known as agents, facilitating communication and interaction between them Additionally, it helps define recomposition rules through organizational development MAS can effectively meet the needs of a hybrid and heterogeneous information system that is dynamic and evolving, offering two main advantages through agent communication.
— une meilleure orientation du choix des combinaisons pré-traitement/modèle à calculer vu qu’il est impossible de toutes les essayer ;
To enhance the selection of base models utilized in the meta-model, it is essential to focus on the most effective options, as retaining all models is impractical Additionally, developing a generic NIRS calibration pipeline that employs model stacking, with an optimization strategy driven by a multi-agent system, is crucial Ultimately, this approach aims to improve the overall performance and efficiency of the calibration process.
Questions de la recherche
ce sujet de stage est né des questions suivantes :
— Quels sont les meilleurs algorithmes pour prédire des données de calibration NIRS ?
— la combinaison de modèles offrirait-il de meilleurs résultats qu’un modèle pris de faỗon individuel ?
— les SMA peuvent-ils contribuer à l’optimisation du choix des modèles de base devant fournir le modèle d’ensemble.
Problématique du sujet
The traditional approach to problem-solving raises several questions, as the number of tested combinations is often limited and based on empirical methods This constraint, coupled with the wide variety of preprocessing techniques and models available, significantly reduces the likelihood of identifying the optimal combination Furthermore, relying on a single predictive model often underutilizes prototyping efforts and frequently results in a suboptimal solution compared to an ensemble of models.
This study aims to develop a generic NIRS calibration pipeline that leverages recent advancements in data science and computational capabilities By integrating traditional methods with modern machine learning techniques, the research seeks to establish a standardized approach for identifying solutions that may vary across different studies.
To avoid pitfalls related to the interaction between matter and light, an All Possibilities Approach (APA) is implemented for pre-processing spectra This approach utilizes increasing computational capabilities and improvements in heuristic search algorithms to explore a broader research space for spectrum pre-processing, including various types and combinations of models The generated pre-processings will be combined with a diverse array of models to create foundational models, which can then be integrated into a meta-model To leverage this diversity, model assembly will be prioritized to gather complementary information from multiple base models into an enhanced meta-model However, selecting which base models to compute and retain within the meta-model remains a poorly documented and challenging issue, as the potential number of models is infinite and their selection involves a trade-off between performance and diversity This work aims to define a multi-agent system architecture that ultimately facilitates better selection of base models for effective model assembly.
Objectifs du stage
The primary goal of this work is to develop a generic NIRS calibration pipeline that utilizes model stacking, with the selection strategy optimized through a multi-agent system.
— Choisir les modèles de base ;
— Définir un méta-modèle et implémenter son calcul via le stacking ;
— Développer une architecture de SMA qui permette de :
— mieux orienter le choix des combinaisons pré-traitement/modèle à calcu- ler ;
— mieux choisir les modèles de bases utilisés dans le méta-modèle ;
— Établir des mécanismes de coopération entre les modèles de base.
Hypothèse de travail
Cooperation among multiple learners, through the sharing of experiences and the enhancement of hyperparameters, can significantly optimize the predictions of base models Additionally, employing an ensemble technique to combine individual solutions into a cohesive global solution can further improve predictive outcomes.
Conclusion
The International Francophone Institute (IFI) is a part of the National University of Vietnam, dedicated to training mostly international students in master's programs It offers four specialized fields: intelligent systems and multimedia, communication systems and networks, banking, finance, and fintech, as well as digital and editorial communication Additionally, IFI occasionally hosts interns who are nearing the completion of their training.
The CIRAD is a leading French research center specializing in agronomy, providing research internships for students eager to advance their studies I completed my final training internship with the PAM team at the UMR AGAP, which focuses on the genetic improvement and adaptation of Mediterranean and tropical plants.
Introduction
This chapter focuses on exploring various methods and techniques that will help us achieve our objectives The goals of this internship are diverse, and for each, we have multiple potential approaches We will first assess the existing methods for model assembly techniques in machine learning Following that, we will examine basic algorithms (regressors) to prototype stacking Finally, we will discuss Multi-Agent Systems (MAS) and the related research efforts.
Quelques travaux sur les techniques d’assemblages de modèles du ma-
Many learning algorithms create a single model for making predictions, but various decisions, such as model initialization parameters, significantly impact the classifier's performance.
Selecting the best classifier from a set of available options may not yield optimal results due to variations in new data distributions Testing multiple models before choosing one can lead to the loss of valuable insights from all classifiers In contrast, Multiple Classifier Systems (MCS), also known as ensemble models, typically offer more robust solutions by leveraging all available information from base models Researchers define MCS as the aggregation of multiple classification algorithms, whose individual predictions are combined to generate a final prediction MCS can be constructed using similar or different classifiers, and the design should incorporate mutually complementary and diverse models Although there is no precise definition, ensemble diversity refers to the uncorrelated errors among different classifiers The combination of individual solutions is achieved through information fusion, with one popular technique being the simple averaging of individual results This method has proven effective, especially in large and complex datasets, provided that the base classifiers exhibit strong, or ideally no, correlation to achieve improvements.
Différentes techniques d’assemblages de modèles du machine learning 15
Stacking [17]
Stacking, also known as blending, is a technique that involves applying a meta-model to base models generated by other algorithms This approach aims to predict the best base models and assign them appropriate weights The advantage of stacking lies in its ability to combine diverse models, significantly enhancing the quality of the final prediction, as demonstrated by the $1 million Netflix challenge.
The concept is straightforward: instead of using simple functions like hard voting to combine predictions from all predictors in an ensemble, why not train a model that utilizes the predictions of previous models? As illustrated in the figure, this ensemble performs a regression task on a new instance Each of the last three predictions yields different values (3.1, 2.7, and 2.9), and the final prediction, referred to as the "blender" or "meta learner," takes these predictions as input to generate the final output (3.0).
FIGURE2.2 – Agrégation des prédictions à l’aide du stacking [17]
To train the mixer, a common approach involves using a training set, which is divided into two subsets The first subset is utilized to train the predictors of the first layer.
FIGURE2.3 – Formation de la première couche [17]
In the stacking process, the first layer of predictors is utilized to make predictions on a second dataset, ensuring that the predictions remain "clean" as these cases were not encountered during training For each instance in the holdout set, three predicted values are generated These predicted values can then be used to create a new training set, where they serve as input features while retaining the target values Consequently, the stacking model is trained on this new dataset, learning to forecast the target value based on the predictions from the first layer.
It is indeed possible to create various different mixers in this way, such as one using linear regression and another utilizing random forests, resulting in an entire layer of mixers The key is to divide the training set into three subsets.
— le premier sert à former la première couche,
— le second sert à créer l’ensemble de formation utilisé pour former la deuxième couche (en utilisant les prédictions faites par les algorithmes de la première),
— et le troisième est utilisé pour créer l’ensemble de formation (en utilisant les pré- dictions faites par les prédicteurs de la deuxième couche).
Une fois que cela est fait, nous pouvons faire une prédiction pour une nouvelle ins- tance en passant par chaque couche de manière séquentielle, comme le montre 2.5 ci-dessous
FIGURE2.5 – Prédictions dans un ensemble d’empilage multicouches [17]
Quelques algorithmes de régression utilisées dans les méthodes assem-
Forêt aléatoire
The random forest algorithm is a machine learning technique used to build meta-models As an ensemble model, it combines outputs from multiple different models to generate a response This involves creating several decision trees, which work together to improve accuracy and reliability.
1 https://www.kaggle.com/saadzarook/random-forest-classifer-on-iris-df réponse est calculée sur la base du résultat de tous les arbres de décision Dans le cas de la classification, la classe la plus prédite par les arbres est celle qui est attribuée à cet objet.
Decision trees are predictive models that utilize a series of binary rules to compute a target value There are two main types of decision trees: classification trees and regression trees Classification trees are designed for categorical datasets, such as land cover classification, while regression trees are used for continuous datasets, like biomass and forest cover percentage.
FIGURE2.6 – Fonctionnement de l’algorithme forêt aléatoire [23]
Les “Machines à Vecteurs de Support”
Also known as "SVM" (Support Vector Machine), this algorithm is primarily used for classification problems, although it has been extended to address regression issues as well (Drucker et al., 1996).
Let's revisit our example of ideal vacation destinations For simplicity, we will consider only two variables to describe each city: temperature and population density This allows us to represent the cities in a two-dimensional space.
2 https://datakeen.co/8-machine-learning-algorithms-explained-in-human-language
We represent cities you loved visiting with circles and those you liked less with squares When considering new cities, you want to know which group the city most closely resembles.
Comme nous le voyons sur le graphique de droite, il existe de nombreux plans (des droites lorsque nous n’avons que 2 dimensions) qui sépare les deux groupes.
We will select the line that maintains the maximum distance between the two groups To construct this line, it is unnecessary to consider all points; instead, we focus on the points at the boundary of their respective groups, known as support vectors The planes that pass through these support vectors are referred to as supporting planes The separation plane will be the one that is equidistant from the two supporting planes.
La programmation à base d’agent
Agent : Définitions et particularités
Numerous studies have explored the concept of an agent, leading to diverse and rich definitions To summarize the various explanations found in the literature, we highlight the definition by Jennings et al (1996), which describes an agent as an autonomous entity, and the definition by Florez (1999), which characterizes an agent as an interactive entity.
An agent is an autonomous program that can manage its actions based on its perception of the environment, actively pursuing one or more objectives.
The definition of an agent as an interactive entity, established by Florez in 1999, is comprehensive and clear, drawing on previously established definitions by Woodridge et al in 1995.
An agent is an interactive entity that exists within a shared environment alongside other agents It is a conceptual entity that perceives and acts either proactively or reactively in an environment where other agents interact based on shared knowledge, communication, and representation.
In his 1995 work, Ferber provides a comprehensive definition of the concept of an agent, synthesizing previous definitions into a cohesive understanding.
— Un agent est une entité physique ou virtuelle :
— qui est capable d’agir dans un environnement,
— qui peut communiquer directement avec d’autres agents,
— qui est mue par un ensemble de tendances (sous la forme d’objectifs individuels ou d’une fonction de satisfaction, voire de survie, qu’elle cherche à optimiser),
— qui possède des ressources propres,
— qui est capable de percevoir (mais de manière limitée) son environnement,
— qui ne dispose que d’une représentation partielle de cet environnement (et éven- tuellement aucune),
— qui possède des compétences et des offres de services,
The individual may reproduce behaviors that align with their goals, taking into account the resources and skills available to them This process is influenced by their perceptions, representations, and the communications they receive.
Un agent est particulièrement caractérisé par son autonomie, sa réactivité, sa proac- tivité et sa sociabilité :
Autonomy refers to an agent's ability to take initiative and act independently without human intervention An autonomous agent must possess a significant level of control over its actions and internal states Reactivity, on the other hand, is the agent's capability to perceive its environment and respond automatically to any changes it detects.
— la proactivité : Un agent proactif ne se contente pas de répondre aux événements.
In addition to its attributes and methods, it has internal processes that enable it to take initiatives to achieve its own goals and objectives, which it is capable of setting for itself.
Sociability refers to an agent's ability to willingly interact with other agents to achieve their respective goals Additional characteristics highlighted in the literature include continuity, which denotes an agent's capacity to remain active and maintain identity over time, adaptability, which reflects the ability to learn and adjust to environmental changes while improving through experience, and collaboration, which involves working alongside other agents to accomplish a shared objective.
Les agents peuvent être classés selon différentes propriétés, par exemple : la granu- larité (degré d’intelligence), le rôle, la mobilité, la capacité de coopérer, etc.
The classification of agents is based on their granularity and intelligence, which is determined by their ability to reason, learn, understand, and plan This intelligence is closely linked to the agent's autonomy and flexibility in dynamic environments Generally, three levels of intelligence are identified, ranked from least to most significant.
— les agents réactifs : ce sont des agents passifs qui réagissent seulement à un sti- mulus Ce type d’agent ne dispose pas de module de raisonnement interne ;
Proactive agents are dynamic individuals who take initiative to achieve their goals Equipped with specific attributes and methods, they possess internal processes that empower them to act decisively Consequently, a proactive agent is goal-oriented and driven to succeed.
Cognitive agents reason before taking action and are often linked to the feedback loop of perception, reasoning, and action In addition to their goals, these agents encompass psychological concepts that can be articulated through mental attitudes such as beliefs, intentions, and desires.
The cognitive approach equips agents with an abstraction tool to characterize them as intentional systems Notable models in this field include the BDI model, which stands for Believe, Desire, and Intention, and the BUC model, representing Believe, Uncertainty, and Choice.
B Classification des agents selon la mobilité Un agent peut être stationnaire ou mobile :
— Un agent stationnaire est dépourvu de mobilité Cet agent agit localement, pen- dant tout son cycle de vie, dans la machine là ó il a été implanté initialement ;
Unlike a stationary agent, a mobile agent can move through a network from one node to another, acting independently or at the request of another agent This paradigm is increasingly utilized in the context of widely distributed networks.
C Classification des agents selon la fonction ou le rôle Une telle classification se base sur le rôle joué par l’agent dans le système Les agents les plus connus sont :
Information agents, also known as Internet agents, play a crucial role in managing vast amounts of information on the web through search engines These agents can be either stationary, like email management systems or virtual assistants, or mobile, enabling them to navigate the internet to find and deliver information to their intended destinations.
Les Systèmes Multi-Agents
A Multi-Agent System (MAS) consists of a group of entities, either software agents or humans, that interact within a shared environment to tackle problems that exceed the capabilities or knowledge of individual agents These interactions can involve cooperation or competition, and the environment serves as a common space for all agents, influenced by the specific application context, which may include the physical world, problem data, or a collection of agents A more detailed definition of a MAS includes an environment, a set of passive objects, and a group of active agents The objects are interconnected through defined relationships and are manipulated by agents that perceive them, allowing for creation, modification, and destruction The agents' actions on the environment and the environment's responses are governed by operators representing universal laws, which can be perceived, manipulated, transformed, consumed, and produced by the agents through a range of operations or skills.
In a Multi-Agent System (MAS), the environment represents the realm in which agents operate Generally, the environment serves as a field of interactions, encompassing signals and traces, and may be governed by physical laws This environment can embody various representations.
— Un lieu ó des actions individuelles ou collectives sont réalisées et ó des réac- tions sont perỗues ;
— Un espace de déplacement : grille, position des agents, etc ;
— Un moyen de structuration des agents : relations de proximité, définition des to- pologies spatiales ou temporelles, etc ;
— Une source de données pour le système ;
— Un lieu ó des ressources sont disponibles.
B Capacités d’un agent dans un environnement Dans un environnement, un agent dispose des deux capacités de perception et d’action.
— Un lieu ó des actions individuelles ou collectives sont réalisées et ó des réac- tions sont perỗues ;
— Un espace de déplacement : grille, position des agents, etc ;
— Un moyen de structuration des agents : relations de proximité, définition des to- pologies spatiales ou temporelles, etc ;
— Une source de données pour le système ;
— Un lieu ó des ressources sont disponibles.
Perception capacity refers to the ability to recognize objects, including their positions and relationships with one another Meanwhile, action capacity enables the transformation of a system's state by altering the positions and relationships between these objects.
An environment can be classified as static or dynamic, deterministic or non-deterministic, discrete or continuous, and accessible or inaccessible A static environment remains unchanged, relying solely on its previous states and the actions taken by the system In contrast, the physical environment of a social agent is typically dynamic, influenced by the actions of other agents, making it unpredictable A deterministic environment ensures that any action taken by the system produces a unique and certain effect Conversely, in a non-deterministic environment, such as that of virtual actors, simultaneous actions may lead to unpredictable outcomes An environment is considered discrete if it has a finite set of perceptions and possible actions, while the real world is continuous Lastly, an environment is accessible if the system can perceive it completely and accurately at all times; for example, a robot's environment is not fully accessible due to the limitations of its sensors in terms of precision and range.
Interaction is a crucial aspect of a multi-agent system (MAS), characterized by a dynamic relationship established among various agents through their combined and reciprocal actions This interaction can take multiple forms, including cooperation, coordination, negotiation, and collaboration However, it is primarily represented by coordination, which encompasses all processes involved in decision-making and the overall behavior of a group of agents seeking new alternatives.
Interaction and communication are essential for agents to collaborate effectively While agents can sometimes infer others' plans without direct communication, the exchange of knowledge and skills is crucial This communication can occur through various means, including signal dissemination and request exchanges It may be selective, involving only a few agents, or encompass all agents in the system Initially, communication occurred indirectly via shared data structures known as blackboards, where agents alternately deposited information to be shared or retrieved data they required Overall, communication can be categorized as either direct or indirect.
Indirect communication occurs through mediating agents who interact with their environment An agent wishing to convey information alters their surroundings, allowing other agents to perceive these changes These modifications are interpreted by others to extract the necessary information.
Direct communication occurs through message transmission, aligning with the current communication model in Social Media Applications (SMA) The sender encodes the message using language before sending it to the receiver, who then decodes the received message to understand its content.
In the realm of direct communication, standards such as languages and protocols have been established by organizations like FIPA (Foundation of Intelligent Physical Agents) to enhance interoperability among multi-agent platforms FIPA has proposed and specified the Agent Communication Language (ACL) as a communication standard, while another widely used language is the Knowledge Query and Manipulation Language (KQML), developed through the efforts of DARPA (Defense Advanced Research Projects Agency).
Conclusion
In this chapter, we reviewed various approaches that can help us achieve the objectives of this internship Initially, we discussed several studies on model ensemble techniques in machine learning, followed by an overview of algorithms for model assembly, particularly focusing on stacking Finally, we introduced Multi-Agent Systems (MAS), highlighting the agents and their unique characteristics In the upcoming chapter, we will present our proposed solution and the approaches selected for accomplishing each objective.
Introduction
In this chapter, we will introduce the proposed solution to achieve the project's objectives, specifically the construction of a generic calibration pipeline for NIRS using model stacking The selection strategy for this stacking will be optimized through a multi-agent system (MAS) To reach this goal, we will present an algorithm and design the UML.
Algorithme proposé
The proposed algorithm enables stacking by utilizing the prediction results from validation folds of various classifiers Initially, a dataset will be provided to the classifiers for training, with the entire learning process and the formation of the meta-model conducted through a multi-agent system (MAS) The use of MAS aims to create a dynamic system where classifier agents exchange information and data, allowing for pauses and resumptions in learning without losing data or execution state, ensuring system persistence Additionally, these agents can adapt and improve by incorporating new data Initially, different filters will be applied to the dataset, followed by training the classifiers on these subsets, which will undergo cross-validation The prediction results from the classifiers will then be fed into the stacking model, taking into account specific parameters.
— Combiner en tenant compte des meilleures prédictions par Fold au niveau de T=0
FIGURE3.1 – Diagramme représentant l’algorithme proposée
It is important to note that the various base algorithms will have an equal participation rate in the formation of the meta-model, as the ensemble method used is stacking In contrast, other ensemble methods such as Bagging (Bootstrap Aggregating) and Boosting differ in their approach.
Conception de l’AUML du SMA
Analyse du Système
This phase focuses on understanding the system's objectives and documenting the environment for its deployment It involves a set of roles that can be utilized to achieve the system's goals and consists of three key steps: capturing the essential requirements for the proposed solution, identifying the system's stakeholders (agents), and determining the necessary roles within the system To facilitate these steps, we employ goal, organizational, and role models.
The objective model for MAS is illustrated in the figures below, where each goal is denoted by the keyword "Objective." The objectives GatherData3.3, PredClassifier3.3, and BuildEnsemble3.4 are further divided into sub-goals Some of these objectives have parameters, such as LoadDataInCSVformat, with "D" representing the training dataset loaded by the user The processing order of the objectives is specified along with the approaches to be used Additionally, the notation "ô prộcộdộ ằ" is employed to indicate that one target is executed upon the completion of another.
FIGURE3.3 – Modèle d’objectif du système mise en place
FIGURE3.4 – Modèle d’objectif du stacking
To achieve the BuildEnsemble objective, the GatherData goal must first be met, along with the PredClassifiers goal The prediction of new data is accomplished by executing the sub-goals in the following order: selecting the classifiers.
In the Chooseclassifier method, information is exchanged between classifiers once a predetermined number of iterations is reached, allowing them to retain their learning history Additionally, each classifier is assigned a weight based on performance metrics from each fold and results obtained using various filters.
The structure of the BuildEnsemble objective is depicted in the figure above, highlighting a trigger between the EvaluateClassifiersBy5FoldCV and CollectPerformanceMeasures targets This indicates that each time a classifier is evaluated using cross-validation with the defined number of partitions (five in this case), a performance measure (RMSE) is sent to CollectPerformanceMeasures for collection, along with a corresponding set of points Once all performance parameters have been gathered, the selection of the best classifier can commence through the SelectBestClassifier objective.
The second step involves developing the organizational model based on the Objective Model Its purpose is to identify the interactions within the system among the actors, where each actor represents an agent The various modes of interaction between these agents are modeled in the form of protocols.
FIGURE3.5 – Modèle organisationnel du système
The organizational model of the system is depicted in Figure 2.1 The GUI actor sends a dataset in CSV format to the coordinator using the submitDataInCSV protocol The coordinator receives this dataset through the Choosedata protocol via the Database agent Upon receipt, the coordinator applies various filters and forwards the data to the classifiers using the SendTrainingDatawithFilter protocol, while selecting different classifiers through GetModel The classifiers communicate with each other to share results such as RMSE and BestRmseByFilters through messages (including Message class and its subclasses XMLMessage, ActMessage, ACLMessage) using their unique AgentAddress identifiers They engage in asynchronous communication through specific primitives.
— BroadcastMessage(Community c, Groupe g, Role r, Message m) et pour accéder à ses message un agent utilise :
3.3.1.3 Stratégie d’échange entre les classifiers
This strategy is designed to optimize learning and enhance the selection of various combinations for stack formation The illustration in Figure 3.6 below demonstrates this strategy.
In the exchange strategy model, certain classifiers learn at different rates, using the results of faster classifiers as a benchmark for comparison A classifier may halt its learning process if its Root Mean Square Error (RMSE) consistently exceeds 30% of the RMSE from other classifiers However, it can resume learning upon receiving a message from another classifier that has a lower RMSE, such as 2.3 at filter 16, and continues to learn effectively with an RMSE of 1.8 at filter 56 In this scenario, the classifier will notify others to restart their training based on the successful filter.
The objective of this task is to detail the protocols identified within role and class models, focusing on the messages exchanged between agents Two protocol models are presented: GatherData and BuildEnsemble The GatherData protocol, illustrated in Figure 3.7, facilitates interaction among the graphical interface, database, and the coordinating agent, denoted by the keyword "Agent." A perpendicular line to this keyword indicates the agent's lifespan Specifically, the "chooseData" protocol enables the selection of the dataset through the graphical interface agent.
FIGURE3.7 – Diagramme de séquence du protocole GatherData
Figure 3.8 illustrates the BuildEnsemble protocol, which outlines the interaction between the coordinator and classifier agents In this protocol model, the classifier agent sends a "ready" message to the coordinator agent Upon receiving this message, the coordinator responds by transmitting the training dataset through the sendTrainingDatasetwithfilter protocol The label Parallel 1 m indicates that exchanges between the coordinator and classifiers occur simultaneously Consequently, the classifier agent concurrently shares its prediction results with both the other classifiers and the coordinator agent.
FIGURE3.8 – Diagramme de séquence du protocole BuildEnsemble
Conception du système
A plan serves as a means for agents to achieve objectives within a system, modeled using a finite state machine to specify a single control flow that dictates an agent's behavior Each plan has a starting state, represented by a black circle, and an ending state, depicted as a black circle enclosed within a red circle Transitions between states, defined by the term "state," utilize arrows with the syntax: [guard] receive(message, sender) / send(message, receiver) The guard keyword represents a boolean condition that activates the transition The receive(message, sender) denotes a message received from the sender that triggers the transition, while send(message, receiver) indicates a message sent to a recipient during the transition Messages can include various parameters.
FIGURE3.9 – Modèle de plan de l’agent GUI
Conclusion
In this chapter, we outlined the proposed solution to achieve the project's objectives and the various methods selected for each task The following chapter will focus on the implementation of these methods and the analysis of the results obtained.
Implémentation, résultats et analyse des résultats
Introduction
In this chapter, we will discuss the implementation of our solution and the tools utilized for this process Following that, we will present the proposed solution along with the results achieved Additionally, we will conduct an analysis for each step involved.
Outils utilisés
Several platforms exist for developing multi-agent systems, but we chose MadKit, an API designed for creating basic programs that highlight the key primitives of multi-agent systems MadKit, which stands for Multiagent Development Kit, is a Java-based platform that facilitates the easy construction of distributed applications and simulations using the multi-agent paradigm A key aspect of MadKit is its organization-centered approach (OCMAS), in contrast to conventional methods that focus primarily on agents As a result, MadKit is built on the organizational model of Agent/Group/Role (AGR), allowing agents of any type to assume roles within groups and thereby form artificial societies Consequently, MadKit offers unique features tailored for effective multi-agent system development.
— Création d’agents artificiels et gestion du cycle de vie,
— Grande hétérogénéité des architectures des agents : Pas de modèle d’agent pré- défini,
— Une infrastructure organisationnelle qui est utilisée pour plusieurs aspects, no- tamment pour (1) permettre la communication entre les agents et (2) structurer des modèles de simulation multi-agents,
— Outils de simulation et de création de simulateurs basés sur des agents multiples,
— Installations de création d’applications distribuées basées sur des agents mul- tiples.
Le langage python via l’outil Google Colab a été utilisé pour implémenter les modèles de base et le stacking.
Présentation et caractéristiques des données utilises au cours des expéri-
The initial experiment utilized a dataset comprising 326 calibration records and 100 validation records, focusing on 2,876 variables related to cassava from CIAT The primary aim of this dataset is to enhance the varietal quality of cassava roots, specifically targeting their beta-carotene content, a sought-after nutritional attribute.
— La deuxième expérience a été appliquée sur un jeu de données de plus gros vo- lume sur la cocạne la taille de ce jeu est la suivante : 5787 variables étudiés et
2977 observations L’analyte étudié est la benzoylecgonine (principal produit de dégradation de la cocạne).
— La troisième expérience a été appliquée sur un plus petit jeu de données de
The dataset comprises 40 calibration recordings and 20 validation recordings, analyzing a total of 1,154 variables This data is related to beer and its alcohol content.
Prộ-traitement et description du processus d’entraợnement et de forma-
L’entraợnement des modốles de base
Chacun des modốles est entraợnộ sur diffộrentes donnộes transformộes (des filtres différents) Pour chacune d’elles, on réalise les opérations suivantes :
— Entraợnement (avec cross-validation) sur l’ensemble de train,
— Prédiction de l’ensemble de test avec le meilleur modèle des différents folds pour générer le fichier predict_hidden.csv,
— Utilisation du modèle obtenu par le cross-validation pour prédire l’ensemble de test afin d’avoir le fichier predict_global.csv.
Formation du stacked model
Different models can yield better results, each with its own advantages and disadvantages The goal of stacking is to combine the strengths of all base models to achieve an improved outcome The final model will use the predictions of the base models as input data However, a common challenge arises when using multiple base models: identifying complementary models to create a superior final model Without the Smart Model Aggregation (SMA), both the learning and stacking processes occur incrementally, requiring the completion of one model's training before starting another, making the stack effective only after all models are trained One can choose to use all models or a subset by defining a parameter 'm' that represents the number of classifiers to include in the stacking process, which may not always enhance results compared to the best base model Thus, with SMA, the aim is to transform this approach into a more intelligent one, enabling inter-agent communication to automatically discover the best model combinations for stacking.
Fonctionnement avec SMA
Fonctionnement général du SMA
La figure4.1présente un résumé du fonctionnement du Système Multi-Agent.
FIGURE4.1 – Diagramme du fonctionnement général du SMA
Description du SMA
Après avoir découper les fichiers selon les besoins du système multi-agent, voici le fonctionnement du framework :
When launching a learner agent, it executes the learner.py script with the required parameters, leading to the creation of a best.csv file that contains the best RMSE values for each filter Additionally, the process generates global_fit and predict_hidden files, which are utilized by the stacker The best.csv file will be used later by the agent for inter-agent communication to share results.
In Step 2, the coordinating agent executes the stacker.py script, which will be utilized by both the intermediate and final stacks Upon completion of the script, a file named stack_rmse.csv will be created or updated.
Step 3: The GUI agent updates its results table at each defined time interval The stack_rmse file is utilized by the GUI to display various results from the combinations of stack models.
A la différence du fonctionnement initial, on utilise le systèmes multi-agent pour des objectifs suivants :
To enhance training efficiency and improve the performance of baseline models within a multi-agent system, we have implemented a specific approach detailed in section 3.3.1.3.
Instead of training on all datasets, the classifier agent can choose to halt training when it becomes unlikely to achieve good results on the remaining datasets This intelligent decision allows for the optimization of parameters in the event of poor outcomes The result of this approach is akin to pruning in tree search algorithms.
Performance optimization of the stacked model involves combining various models to yield intermediate results before achieving final outcomes This approach allows for an analysis of how different filters impact the overall model, as detailed in section 4.4.3.
fonctionnement du classifier
The classifier has been restructured into a finite-state machine to optimize time and performance To enable recovery behavior based on results shared by other classifiers, we have defined the following states for the classifiers.
— État learning initial : à cette étapes, les différentes filtres de données sont parcou- rues de faỗon incrộmentale,
— État pause : État pause c’est l’état à la sortie de chaque learning state intermé- diaire avant de passer au prochain,
— État learning resume : learning en partant du filtre du meilleur filtre donnés par les autres,
— ẫtat learning optimization : learning en repartant du meilleur filtre reỗu et avec maximisation des hyper-paramètres,
— État terminated : état à la fin du learner.
In the classifier operation diagram, during the initial learning state, if the best results from other classifiers fall below 30%, the current state will pause Two scenarios may arise: if the best result is ahead of the filters, the next state will transition to learning resume; conversely, if the best result lags behind its current filter (indicating that lower RMSE is preferable), it will likely have passed through the same filter without achieving satisfactory results, leading it directly to the learning resume opt state Following the learning resume phase, the next state must be learning resume opt, with the output from this state signifying the end of the cycle.
Communication inter-classifiers
For message exchanges, we have implemented two listening threads: listen and share_result These threads will cease operation if the classifier enters a terminated state The share_result function runs in the main loop, keeping the result-sharing thread active If there is at least one better local RMSE, it sends an update A best.csv file is refreshed each time a filter is applied, and after every five filters, the best RMSE is retrieved with the current filter Each classifier folder contains a best file, which is read by share_result to send the best filter along with its associated RMSE.
FIGURE4.3 – exemple de contenu du fichier best
Le stacking intermediate
In the previous version, a simple stacker was used that attempted combinations of classifiers but did not produce intermediate results, as it executed only after all training was completed The addition of an intermediate stack allows for real-time results from the combinations of stacks of all models, generating 10 files of hidden and global fit results Its operation is similar to the default stack, but it halts as soon as the final stack is initiated.
Il diffuse les résultats au GUI avec un type de resultat intermediate Les résultats s’affi- chant sur le GUI ont actuellement deux types : intermediate et final.
Présentation du pipeline mise en place
Présentation de l’écran d’accueil
Les figures4.4et4.5ci-dessous représentent la page d’accueil de notre pipeline Sur cette page, nous avons la possibilité de :
— choisir le jeu de données,
— définir le nombre minimal de classifier qui vont constituer le stacking (5 modèles sont définis par défaut),
— choisir les modèles de base,
FIGURE4.4 – Page d’accueil de notre solution
FIGURE4.5 – Page de choix des algorithmes
Expériences réalisées et analyse des résultats
Métrique utilisé
La métrique retenue pour mieux comprendre et caractériser la qualité et la robus- tesse de nos modèle est le RMSE 1
The RMSE (Root Mean Square Error) serves as an indicator of the dispersion or variability in prediction quality and can be linked to the model's variance However, interpreting RMSE values can be challenging, as it is often unclear whether a variance value is low or high To address this issue, normalizing RMSE as a percentage of the average observation value can provide clearer insights For instance, an RMSE of 10 may seem low when the average observation is 500, but it indicates high variance if the average is only 15 In the former scenario, the model's variance represents just 5% of the average, while in the latter, it constitutes over 65%, highlighting the importance of context in evaluating RMSE.
1 https://www.aspexit.com/comment-valider-un-modele-de-prediction/
L’exemple suivant montre des données d’évaluation qui contiennent N enregistrements :
Expériences réalisées
Cette section présente les résultats obtenus à partir des expériences menées sur différents jeux de données
La configuration pour cette expérience est la suivante :
A selection of 19 base classifiers, including lasso, Lars, NuSVR, KNeighborsRegressor, ridge, AdaBoostRegressor, LinearSVR, TheilSenRegressor, decision tree, SGDRegressor, and Pipeline, was made from a total of 33 options This choice is driven by the fast learning capabilities of these algorithms Additionally, stacking is implemented using the Random Forest Regressor.
— Le nombre de modèle pour la construction du stacking définit a cinq (5)
A Analyse des résultats Les tableau et figure ci-dessous présentent les résultats ob- tenus au cours de la première expérience.
TABLE4.1 – Récapitulatif des RMSE par classificateurs de base
Algorithmes Min RMSE Mean RMSE Max RMSE lasso 3,1176690358 3,1176690358 3,1176690358
FIGURE4.6 – Évolution de la moyenne des RMSE par algorithmes
Table 4.1 and Figure 4.6 summarize the obtained results, highlighting that the best baseline classifier is Lasso in terms of Min, Mean, and Max RMSE, followed by Lars, NuSVR, and KNeighborsRegressor Notably, out of the 19 selected classifiers, only 11 reached the threshold for inclusion in the baseline model Additionally, while SGDRegressor has the highest average RMSE, it also shows one of the best minimums, demonstrating its complementary role among classifiers and contributing to the development of optimal ensemble models.
TABLE4.2 – Résultat partiel des différents ensembles constitués
SGDRegressor_Linear_KNeighborsRegressor_lasso_Lars 3,3023873399 inter
The analysis of various regression models reveals a range of combinations involving SGDRegressor, NuSVR, KNeighborsRegressor, and lasso techniques, with performance metrics indicating their effectiveness Notably, models such as SGDRegressor combined with ridge, TheilSenRegressor, and KNeighborsRegressor exhibit strong results, with some configurations achieving scores above 3.31 Additionally, the integration of decision tree methods alongside these regressors, as seen in combinations like SGDRegressor with ridge and TheilSenRegressor, further enhances predictive accuracy The exploration of ensemble methods like AdaBoostRegressor alongside these regressors also shows promising outcomes, highlighting the versatility and potential of these machine learning techniques in regression tasks.
La configuration pour cette deuxième expérience est la suivante :
A selection of 19 base classifiers, including lasso, Lars, NuSVR, KNeighborsRegressor, ridge, AdaBoostRegressor, LinearSVR, TheilSenRegressor, decision tree, SGDRegressor, and Pipeline, was made from a total of 33 options This choice is driven by the rapid learning capabilities of these algorithms Additionally, stacking is implemented using the Random Forest Regressor.
— Le nombre de modèle pour la construction du stacking définit a cinq (5).
A Analyse des résultats Les tableau et figure ci-dessous présentent les résultats ob- tenus au cours de la deuxième expérience.
TABLE4.3 – Récapitulatif des RMSE par classificateurs de base
Algorithmes Min RMSE Mean RMSE Max RMSE
FIGURE 4.7 – Évolution de la moyenne des RMSE par algorithmes au cours de la deuxième expérience
Le tableau4.3et figure4.7ci-contre résument les résultats obtenus lors de la deuxième expérience Le meilleur classificateur de base est Ridge en terme de minimum suivi de
LinearRegression et de SGDRegressor Sur les 19 classificateurs de base choisis, seuls
13 ont pu passé le seuil de résultats à retenir dans le modèle de base LinearRegression a contribué à créer les meilleurs modèles d’ensemble4.4
TABLE4.4 – Résultat partiel des différents ensembles constitués
The performance of various regression models is highlighted, with notable combinations achieving significant results The AdaBoostRegressor, in conjunction with SGDRegressor, LinearSVR, and NuSVR, recorded an impressive score of 2.27645557 Another combination featuring SGDRegressor, LinearSVR, NuSVR, decision tree, and Lars achieved a score of 2.27655618 The TheilSenRegressor, combined with NuSVR, lasso, and decision tree, reached a score of 2.27681478 The final model, which includes AdaBoostRegressor, SGDRegressor, LinearSVR, TheilSenRegressor, and SVR, achieved a remarkable score of 2.27689220 Additionally, the combination of AdaBoostRegressor, LinearSVR, SVR, KNeighborsRegressor, and lasso scored 2.27722395, while the TheilSenRegressor, NuSVR, SVR, lasso, and decision tree reached a score of 2.27726439 Lastly, a similar combination with TheilSenRegressor, NuSVR, SVR, lasso, and decision tree achieved a score of 2.37726439, showcasing the effectiveness of these regression techniques.
SGDRegressor_LinearSVR_NuSVR_lasso_Lars 2,27792203 final
The performance of various regression models has been evaluated, with the AdaBoostRegressor and TheilSenRegressor achieving an inter score of 2.27867518 The combination of AdaBoostRegressor, TheilSenRegressor, SVR, KNeighborsRegressor, and Lasso reached a final score of 2.28054593 Additionally, the TheilSenRegressor combined with KNeighborsRegressor, Lasso, and Decision Tree yielded a final score of 2.28062380 The SGDRegressor, LinearSVR, NuSVR, KNeighborsRegressor, and Lasso collectively attained an inter score of 2.28185129, which was consistently repeated across multiple evaluations.
SGDRegressor_LinearSVR_NuSVR_lasso_Lars 2,28198745 inter
AdaBoostRegressor_SGDRegressor_LinearSVR_KNeighborsRegressor 2,28636414 inter AdaBoostRegressor_SGDRegressor_LinearSVR_TheilSenRegressor_lasso 2,29144186 inter SGDRegressor_NuSVR_KNeighborsRegressor_lasso_Lars 3,34725533 inter SGDRegressor_svr_KNeighborsRegressor_lasso_Lars 3,34771156 inter
NuSVR_svr_KNeighborsRegressor_lasso_Lars 3,34995254 inter
The performance metrics for various regression models indicate the following results: the combination of SGDRegressor, SVR, KNeighborsRegressor, and Lasso achieved an inter score of 3.35054156 Another model featuring SGDRegressor, TheilSenRegressor, SVR, and Lasso recorded an inter score of 3.35362073 Additionally, the integration of SGDRegressor, TheilSenRegressor, SVR, and Lasso yielded an inter score of 3.35368127 The model comprising TheilSenRegressor, NuSVR, SVR, KNeighborsRegressor, and Lasso also reached an inter score of 3.35441270 This score was matched by another configuration of TheilSenRegressor, NuSVR, SVR, KNeighborsRegressor, and Lasso Furthermore, the combination of TheilSenRegressor, NuSVR, SVR, KNeighborsRegressor, and Lars achieved an inter score of 3.35480968, while the model with TheilSenRegressor, SVR, KNeighborsRegressor, Lasso, and Lars recorded an inter score of 3.35517897.
La configuration pour cette troisième expérience est la même que la suivante et se décline comme suit :
We selected 11 foundational classifiers, including Lasso, Lars, NuSVR, KNeighborsRegressor, Ridge, AdaBoostRegressor, LinearSVR, TheilSenRegressor, DecisionTree, SGDRegressor, and Pipeline, from a pool of 33 options This selection is driven by the fast training capabilities of these algorithms Additionally, we utilize stacking with the Random Forest Regressor for enhanced performance.
— Le nombre de modèle retenu pour la construction du stacking définit a cinq (5).
A Analyse des résultats Les tableau et figure ci-dessous présentent les résul- tats obtenus au cours de la première expérience.
TABLE4.5 – Récapitulatif des RMSE par classificateurs de base
Algorithmes Min RMSE Mean RMSE Max RMSE
FIGURE4.8 – Évolution de la moyenne des RMSE par algorithmes
Table 4.5 and Figure 4.8 summarize the results obtained, indicating that the best baseline classifier is SVR in terms of minimum performance, while Ridge excels in average performance, followed closely by Lars NuSVR and KNeighborsRegressor Out of the 11 selected baseline classifiers, only 6 met the threshold for inclusion in the base model Additionally, LinearRegression played a significant role in developing the best ensemble models.
TABLE4.6 – Résultat partiel des différents ensembles constitués
LinearRegression_NuSVR_KNeighborsRegressor_lasso_Lars 0,96727328 final LinearRegression_ridge_NuSVR_lasso_Lars 1,12945905 inter LinearRegression_ridge_NuSVR_lasso_Lars 1,13217277 inter ridge_NuSVR_svr_lasso 1,13785115 inter
The analysis of various regression models reveals the following results: Linear Regression with Ridge, NuSVR, Lasso, and Lars achieved an inter score of 1.14138517, while a similar combination with Ridge, NuSVR, KNeighborsRegressor, Lasso, and Lars reached a final score of 1.14505311 Another configuration, Linear Regression with Ridge, NuSVR, SVR, and Lars, recorded an inter score of 1.14532893 Additionally, the combination of Ridge, NuSVR, KNeighborsRegressor, and Lars resulted in a final score of 1.14581649 The model comprising Ridge, NuSVR, Lasso, and Lars had a final score of 1.1464713 Furthermore, the inter score for Ridge, NuSVR, SVR, and Lasso, along with Lars, was noted at 1.14848454 The Ridge, NuSVR, SVR, and Lasso model scored 1.14892023 inter, while the Ridge, NuSVR, SVR, and Lasso combination achieved an inter score of 1.16991899 Finally, the Ridge, NuSVR, SVR, Lasso, and Lars configuration culminated in a final score of 1.19469976.
The analysis of various regression models reveals the following results: the final model combining LinearRegression, ridge, SVR, lasso, and Lars achieved a score of 1.19534915 Another model featuring LinearRegression, ridge, KNeighborsRegressor, lasso, and Lars reached a score of 1.19912943 Additionally, the combination of LinearRegression, NuSVR, SVR, lasso, and Lars resulted in an intermediate score of 1.2015925 The model with LinearRegression, NuSVR, SVR, KNeighborsRegressor, and lasso attained a final score of 1.20298646 Other notable scores include 1.20946069 for LinearRegression, ridge, SVR, lasso, and Lars, and 1.21326633 for NuSVR, SVR, KNeighborsRegressor, lasso, and Lars The model with LinearRegression, SVR, KNeighborsRegressor, lasso, and Lars scored 1.22016314, while the ridge, SVR, KNeighborsRegressor, lasso, and Lars combination achieved a final score of 1.22397878 Lastly, the TheilSenRegressor with SVR, KNeighborsRegressor, lasso, and Lars yielded an intermediate score of 2.3551789798.
Interprétation des résultats
During the experiments, certain models successfully passed the initial filter established during training The RMSE scores for the best baseline model and the top ensemble models are presented in the table below.
TABLE4.7 – Synthèse des résultat obtenus
Expérience Modèles de base RMSE Modèles d’ensemble RMSE
SGD Regressor 3,117408901 ridge_TheilSenRegressor_NuSVR
NuSVR_decisiontree_Lars 2,27655618 ridge 2,470964492 TheilSenRegressor_NuSVR_ lasso_decisiontree_Lars 2,27681478
KNeighborsRegressor_lasso_Lars 0,96727328 svr 2,388608831 LinearRegression_ridge_
FIGURE 4.9 – Synthèse des résultats obtenus en comparaison avec les résultats sans SMA
In the initial experiment, the reference value for the root mean square error (RMSE) regarding beta-carotene content is 3.456, leading to two significant observations.
— le premier constat concerne la nette amélioration du RMSE au niveau des classi- ficateurs de base (RMSE compris entre 3.1083 et 3.1174),
— le second apport réside au niveau du modèle d’ensemble qui certes fournit une erreur supérieure à celle des meilleurs modèles de bases mais meilleur que la valeur de référence.
In conclusion, this initial experiment clearly demonstrates that the type of cassava known as ộtudiộ is of superior quality, as the error falls below the reference value.
The second experiment serves as the most representative in terms of data set and results, demonstrating a reference error rate of 2.55 for the presence of benzoylecgonine, confirming the presence of cocaine In this experiment, the root mean square error (RMSE) ranges from 2.47 to 2.87 for baseline models, while ensemble models achieve a lower RMSE of 2.27 This highlights the robustness and improved accuracy of the established pipeline in the second experiment.
The third experiment supports the previous findings regarding the comparison of reference values to determine the alcohol content of beverages It is important to note that if the error value is equal to or less than 1.6, the beverage is classified as alcoholic Therefore, ensemble models enable us to assert this conclusion with certainty.
Based on the previous results from the ensemble models, the initial hypothesis is confirmed Additionally, the pipeline demonstrates the expected robustness and accuracy, as it enhances learning time and allows for the exploration of various ensemble combinations while executing these tasks in parallel.