INTRODUCTION
Overview
As our country advances in socio-economic development, it is transitioning from an agricultural society to one focused on industrialization and modernization, with modern technologies being widely adopted across various sectors This shift significantly boosts the service industries, particularly the food and beverage (F&B) sector, which has experienced robust growth due to economic integration policies The rise of innovative solutions, such as delivery robots seen in countries like China and Japan, enhances customer experiences and facilitates access to advanced 4.0 technology Consequently, this trend fosters the development of science and technology within the F&B industry both globally and locally.
Overview of Service Robot
The International Organization for Standardization (ISO) defines a "service robot" as a robot that carries out beneficial tasks for humans Unlike industrial robots that focus on automating production, service robots are distinct in their roles and designs, enabling them to adapt to various tasks and circumstances while providing excellent service to customers.
Service robots are designed to operate autonomously or semi-autonomously, enabling them to interact with humans to accomplish various tasks These robots are increasingly utilized across diverse sectors such as retail, hospitality, healthcare, security, warehousing, and agriculture, as well as in high-risk job environments.
Contactless delivery is quickly gaining popularity, especially as restaurants adapt to pandemic-related challenges In April, Venezia's, a pizzeria in Arizona, faced difficulties due to lockdown measures aimed at curbing the spread of COVID-19, similar to many other local eateries.
Venezia has enhanced its delivery service by introducing 11 delivery robots from Starship Technologies, which transport pizzas on covered pallets to local households This innovation has transitioned their delivery operation to a "contactless" model, aligning with the growing customer preference for such services during the pandemic.
Starship Technologies has reported a significant surge in demand for contactless delivery in recent weeks Their delivery robots are currently operating in Washington, D.C., providing food delivery services, while in Southern California, they have been successfully delivering milk tea since late March.
Self-driving car services delivering to consumers' homes within a 1-mile radius are being significantly enhanced in the US and other countries to help curb the spread of the virus.
While advancements in technology may eventually enhance robotic capabilities, it is improbable that robots will fully replace humans in the near future As the cost of developing and maintaining robotic staff decreases, the dynamics of human roles in the restaurant industry may shift significantly.
The computational management process in restaurants can be streamlined, reducing reliance on part-time staff, which often leads to instability and inconsistent service quality By integrating robotic systems, restaurant owners can proactively address service challenges, ultimately enhancing customer satisfaction and improving overall operational efficiency.
Research situation
The development of science and technology has received more and more attention from most countries in the world The 4.0 races continuously broke out to be able to
The advancement of science and technology is significantly enhancing development across nations, with diverse applications impacting daily life, from households to military operations Additionally, the growth of service industries worldwide is noteworthy, leading to innovative research and production in various sectors, including restaurants, hotels, hospitals, and educational institutions Notably, the food service industry has embraced modern technologies to improve both food preparation and customer service.
Figure 1.2 Service robots at restaurants [2]
Figure 1.3 Sacarino Service Robot in a Hotel Environment [3]
In our country today, there are also many restaurants and eateries that apply automatic technologies in the operation process The famous hot pot restaurant
Kichi Kichi offers a unique dining experience with its conveyor belt system that automatically delivers hot pot side dishes to customers Located in Ho Chi Minh City, the University of Technology and Education is renowned for its Synary coffee shop, which features a serving robot This trend of integrating science and technology into the restaurant industry is growing, with many establishments adopting innovative solutions Consequently, the use of serving robots in restaurants remains a significant area of interest for researchers, engineers, and students alike.
Reasons for choosing topic
In today's fast-paced world, people's leisure activities increasingly involve dining out and socializing at restaurants, bars, and shopping centers, especially during weekends and holidays The growing interest in modern technology is evident through the popularity of robots, which symbolize advancements in science and technology As our country progresses in various sectors, including economy and technology, there has been significant research and development of robots for diverse applications, such as massage therapy, medical assistance, and logistics support Notably, the restaurant and food service industry is embracing robotics to enhance customer experiences A delightful meal not only boosts happiness but also fosters connections with loved ones, prompting the innovation of service robots aimed at creating a more enjoyable dining atmosphere.
Objectives of the topic
The successful development of restaurant service robots involves research and design to enable self-location and optimal movement within the dining environment These robots utilize a pre-designed voice communication interface to interact with customers effectively Additionally, advancements in multi-robot distributed systems allow for self-propelled operation and data transmission among robots during service By integrating artificial intelligence, these robots enhance customer communication, reduce the workload of service staff, and ensure more accurate food ordering and serving.
Research Methods
The research method employed by the team integrates document analysis with theoretical foundations for design and fabrication, alongside experimental evaluations To enhance their theoretical knowledge, the team explored additional resources related to the food and beverage industry and service robots, both domestically and internationally They also conducted practical surveys to observe the workflow of restaurant waitstaff, aiming to design a user-friendly robot that effectively performs waiter duties.
Limit of topics
This article emphasizes the design, manufacturing, and movement control of robots to achieve optimal positioning An ideal operational environment for these robots is one with minimal pedestrian traffic and obstacles, ensuring smooth and efficient movement.
THEORETICAL BASIS
Overview
Designing a robot to function as a restaurant waiter requires a deep understanding of restaurant operations and the role of a waiter This knowledge is essential for developing robotic functions that are tailored to the specific needs of the dining environment.
Definition of service robot
A service robot is designed to assist people across a wide range of tasks, from simple to complex These robots are particularly beneficial for remote, hazardous, and dirty jobs that pose health risks to humans Additionally, service robots are utilized for repetitive tasks and everyday household chores, enhancing efficiency and safety in various environments.
The benefits of using Robots to serve humans:
• Bring more convenience to business establishments, reduce the cost of hiring human resources and management costs
• More secure and will not affect human health, as well as create the best work efficiency
• Makes it do the job faster, smarter, handle more diverse situations in the service industry.
Definition of Distributed System
A distributed control system (DCS) is a computerized control system designed for processes or plants with multiple control loops, where autonomous controllers are distributed throughout the system without a central operator overseeing the control Unlike centralized systems that rely on discrete controllers in a central control room or a central computer, the DCS enhances reliability and minimizes installation costs by localizing control functions near the process plant, while still allowing for remote monitoring and supervision.
A distributed system, often referred to as distributed computing or distributed databases, consists of independent components situated on various machines that communicate through message sharing to accomplish shared objectives.
A distributed system presents itself to the end-user as a single interface or computer, aiming to optimize resources and information collectively.
7 while preventing failures, as if one system fails, it won't affect the availability of the service
The most important functions of distributed computing are:
• Resource sharing - whether it’s the hardware, software or data that can be shared
• Openness - how open is the software designed to be developed and shared with each other
• Concurrency - multiple machines can process the same function at the same time
• Scalability - how do the computing and processing capabilities multiply when extended to many machines
• Fault tolerance - how easy and quickly can failures in parts of the system be detected and recovered
• Transparency - how much access does one node have to locate and communicate with other nodes in the system
Modern distributed systems have evolved to include autonomous processes that might run on the same physical machine, but interact by exchanging messages with each other.
Robot Knowledge and Dynamics
2.4.1 Kinematics of the Robot Platform
The kinetic model of a mobile robot utilizes a system of equations to predict its position and direction during movement, encompassing both translational and rotational motion Translational motion refers to the displacement of the robot's center, while rotational motion involves the rotation around its axis Central to this model are the wheels, which are fundamental components of any mobile robot Key kinematic parameters include the wheel radius, axle length, and center of gravity A simplified kinematic model typically features two degrees of freedom (DoF) and consists of three wheels, including two driving wheels.
1 self-selecting passive wheel (caster) The wheel configuration is completely described by 4 variables: position coordinates (x, y) on the plane relative to the fixed axis system,
The direction of rotation of the wheel, denoted as θw, is measured relative to the x-axis, representing the angle between any radial and vertical axes Additionally, r refers to the radius of the wheel.
To simplify the calculation of moving components, consider the wheel as a non-slip rotating vertical disc This can be represented by four variables in a coordinate system, indicating the position (x, y) on the plane relative to a fixed axis system, along with the wheel's rotation direction for each axis in the Oxy plane With slip control effectively managed, the vehicle speed adheres to specific constraints.
The position of the robot's center is represented by the reference axis system (x, y,
In this context, the robot's position is defined by coordinates x and y, while its orientation is indicated by the angle θ relative to the reference axes The robot operates on a plane, exhibiting both linear and angular velocities, which can be expressed as the vector q = [x, y, θ].
The robot's movement is governed by two motors that operate at different speeds, enabling it to follow a straight line or a defined arc based on the velocity of its wheels Its state is defined by its position and orientation, represented by the parameters (x, y, θ, ωl, ωr) The control programmer specifies the straight and angular velocities, which correspond to the angular speeds of the left and right wheels.
The kinematic model of a robot is based on its movement dynamics over time, allowing for the prediction of its position and direction using initial body values By analyzing the wheel speed and direction, the robot's position and motion can be accurately determined within a 2D Cartesian coordinate system The robot's location is defined on a global reference axis (X, Y) through the center of the wheel hub and an angle θ A rotation matrix facilitates the plotting of the robot's movement trajectory in the global system by enabling rotation around the z-axis.
The robot's velocity can be predicted using the known rotational speeds of the left and right wheels, the wheel radius, and the distance from the robot's center to the wheels In a differential wheel setup, the fixed drive wheels enable positive rotation to result in positive displacement around the axis The constraints of rolling and sliding for standard wheels ensure that all rotations occur through the wheels, allowing for simultaneous translation and rotation of the robot To maintain pure rotational motion and avoid slippage, the robot must rotate around a point on the line connecting the two wheels, known as the instantaneous center of rotation (ICR) As the speeds of the wheels change, the ICR shifts, resulting in varied trajectories.
The long speed of the platform is established as the average speed of its two wheels, while the robot's actions are influenced by the control variables v and ω.
The action of the robot depends on the control variables v and cos sin 2 cos sin 2 l r v l x v l y
(2.6) Robot movement can be described with two modes, pure rotation and pure translation Newspaper travel paths consist of circular arcs with distinct radii and
To ensure smooth operation and prevent abrupt stops in robotic movement, 10 rotation angles are utilized alongside straight tangent lines The integration of curves and straight lines facilitates continuous motion The fuzzy controller is employed to calculate the linear and rotational velocities, denoted as v and θ, to achieve a controlled trajectory.
In general, we can describe the position of the robot based on its velocity in each instantaneous time interval Ꝋt: x(t) = ∫ 𝑉(𝑡)𝑐𝑜𝑠Ꝋ(𝑡)𝑑𝑡 0 𝑡 (2.8) y(t) = ∫ 𝑉(𝑡)𝑠𝑖𝑛Ꝋ(𝑡)𝑑𝑡 0 𝑡 (2.9)
Given the special case of the differential wheel robot as done, the formula becomes: x(t) = 1
𝑙∫ [𝑣 0 𝑡 𝑟 (𝑡) − 𝑣 𝑙 (𝑡)]𝑑𝑡 (2.13) From there we can calculate the position of the robot through the instantaneous velocity of each wheel
To effectively control a robot's movement, it is essential to create a precise path that the motion controller must adhere to The robot features independently controlled drive wheels powered by motors, while a self-aligning wheel maintains balance on flat surfaces For the purposes of dynamic calculations, the movement of the self-aligning wheel can be disregarded The mobile robot's model is illustrated in Figure 2.2, where 'l' denotes the distance between the two wheels and 'r' indicates the wheel's radius.
We have 2 kinematics: no horizontal and vertical sliding between the wheel and the platform According to these two constraints, the robot can be described as follows:
The coordinates of the robot center are represented as (x C, y C), while θ denotes the angle between the robot's direction and the x-axis Additionally, φ l and φ r indicate the rotation angles of the left and right wheels, respectively The kinetic equation governing the motion of the wheeled mobile robot can be expressed accordingly.
It is suitable to represent the dynamic equation of motion with internal velocity When the robot is the goal of nonholonomic control then:
DESIGN OF MECHANICAL SYSTEM
Overview
In today's rapidly advancing robotics landscape, the development of mobile robots for various applications, including restaurants, requires a well-structured mechanical design process that aligns with initial objectives and facilitates the integration of electrical components and control systems A key challenge in this project is ensuring that the robots can navigate around tables, chairs, and other obstacles seamlessly By focusing on precise mechanical design, we can enhance the overall performance and effectiveness of restaurant robots.
Design option
Determining the problem is that we need to design a device with the following functions as analyzed in Chapter 1:
The self-propelled mobile robot is designed for efficient movement on flat surfaces, making it ideal for environments such as restaurants, cafeterias, and homes Its functionality ensures smooth navigation without the need for leveling, allowing it to operate seamlessly in various indoor settings.
Function 2: The restaurant service robot interacts through the touch screen with a menu-shaped graphic interface, possibly accompanied by audio instructions Interactive screens must be able for customers - staff / users to easily observe and communicate with activities such as: presenting menus, being able to give analysis and advice on dishes to customers, communicating customer information to order inside
Function 3: The mobile robot is capable of transporting the ordered dishes, besides it can transport drinks and other desserts
Transportation of food and water
Function 4: The robot must operate independently of the managed and monitored energy source.
Requirements in mechanical design
To achieve a distinctive design for the serving robot, it's essential to integrate features like menu display and user communication while adhering to size constraints Proper arrangement of supporting peripherals, including touch screens, microphones, and sensors, is crucial for meeting design requirements and ensuring functionality A robust mechanical structure is vital for supporting the robot's weight, enhancing both flexibility and accuracy in its operations.
The design of the robot must translate the requirements from the problem definition into functionality, featuring an aesthetically pleasing and user-friendly shape with a height between 1.2 to 1.6 meters and an interactive screen positioned for optimal viewing Weighing under 60 kg, the robot should be easily transportable to various locations The outer frame, constructed from aluminum, ensures firmness, compactness, and ease of disassembly for repairs It operates on an independent energy source, with a battery capable of lasting 8-10 hours Additionally, the dimensions and functional blocks must align with Vietnamese anthropometry, as the robot's primary function is to efficiently order and deliver food to customers' tables.
Matrix of ideas to determine the design option
Based on functional analysis, we have developed various design options for each sub-function, as outlined in Table 3.1 The moving platform allows for multiple self-propelled motion methods, including wheel, leg, and flying movements, leading to four adjustable screen orientations Additionally, the robot must be capable of transporting food and water, necessitating at least one compartment for proper storage We considered four storage options: mounting the container on the pedestal, integrating it within the pedestal, allowing it to move with the pedestal, or using drawer storage Ultimately, for design simplicity and maximizing space, the team opted for a container positioned above the pedestal.
The movable platform ensures that food and water remain independent and unaffected by the movement of equipment It features a charging function with four options available, allowing users to choose the most suitable option based on their specific needs, as detailed in Table 3.1.
Table 3.1 Matrix of Design Ideas Matrix of Design Ideas
Moving platform differential transmission sprocket-wheel Multi-directional scroll wheel
Permanent turn Rotate and translate
Attached to the pedestal Drag the system Drag from the pedestal
Charger Charging through the station
Wireless charging Wiring in operation Replace batteries
With 4 functions and 4 ideas for each function, we can combine them into 44 design options However, with the weights set in the design with the design requirements set out in the above section in order to meet the reasonable price, moderate size, flexibility,
We propose a moving platform featuring two degrees of freedom, driven by two wheels with differential active control and two self-selecting wheels To ensure stability and prevent damage, a fixed screen will be incorporated The robot is designed to interact with a diverse range of customers It will be powered by a robust battery to support prolonged operation, allowing users to recharge it at the end of each day to maintain consistent battery life While wireless charging options exist, they are costly, and the impracticality of dragging cords in large environments, along with the challenges of battery replacement, necessitate a reliable and user-friendly power solution.
Design of the movable pedestal
The design pedestal is shown in (Figure 3.2) Made of aluminum alloy to minimize the weight of the robot while ensuring the rigidity of the entire structure
The moving platform of the robot is divided into three floors (Figure 3.2):
Figure 3.2 Moving platform Floor 1: Because it is the lowest floor, it is responsible for containing the motor and actuators
Floor 2: Because the program computer as well as the central controller are quite complicated, the battery will be left on the second floor
Floor 3: Contains the central controller and the entire control circuit
A mobile robot on wheels is a mechanical device designed to navigate its environment with two fixed degrees of freedom To optimize the robot's platform size and enhance spatial efficiency, we have selected belt transmission as the preferred method of movement (see Figure 3.3).
In the initial design phase, the team outlined essential requirements for the robotic platform, opting for a differential wheel configuration featuring a two-wheel drive system with independent actuators for each wheel This efficient design utilizes only two motors—one for each steering wheel—connected directly to the motors with an internal reducer Following a thorough evaluation, the team finalized the transmission diagram and created a pedestal simulation image using specialized software.
CHAPTER 4: ENGINE SELECTION CALCULATION AND TRANSFER
Overview
To ensure stable system operation, it is essential to calculate parameters that align with functionality and affordability, catering to user needs This chapter outlines the theoretical basis for parameter calculations, facilitating the selection of suitable components based on specific requirements.
Calculation of engine selection
- Number of revolutions of the wheel:
- Force acting on one wheel:
+ Friction force on one wheel:
+ According to Newton's second law, for the car to move: Fk - Fqt - Fms > 0
=> Fk > Fqt + Fms = 1,25 + 122,625 = 123,875 (N) (4.5) Choose the required force Fk = 150 (N)
- The drive mechanism is arranged as shown below (belt drive)
We have: + 𝜂 đ = 0,96 : Belt drive efficiency
+𝜂 𝑜𝑙 = 0,99 : Transmission efficiency of 1 pair of bearings
Mechanical transmission system with belt drive we choose usb = uđ = 1
Transmission Ratio Distribution
Number of revolutions through the impeller: n1 = 𝑛 đ𝑐
Power through the working shaft:
Table 4 2 Table of calculation results
Belt transmission
+ Power on the drive shaft: Pđc = 55,56 (W)
+ Number of revolutions on the drive shaft: ndc = 2500/62,5 (rpm)
+ Transmission ratio of belt drive: ud = 1
+ Static load, slight vibration, impact
+ The belt is periodically adjusted to the belt tension
Select belt type and cross-section: Choose to fix the ladder belt with teeth A
Determine the diameter of the small pulley:
Determine the axis distance a and belt length L:
- We preliminarily choose the shaft distance a based on the gear ratio ud and the diameter of the pulley d2
Therefore uđ = 1 so choose the ratio a
We choose the standard belt length as: 𝐿 = 254(𝑚𝑚)
Belt life test according to belt laps per second: i = 𝑣
0,254 ≈ 0,28 (time/s) ≤ imax = 10 (time/s) (4.20) Calculate the exact 𝑎 - axis distance according to the standard L:
Calculate the force acting on the belt drive:
But 𝐹 𝑣 = 𝑞 𝑚 𝑣 2 (periodically adjust the tension), with 𝑞 𝑚 = 0,105(𝑘𝑔/𝑚)
- Select Kd = 1,2 (due to 2 shift working mode, light impact load)
- Force acting on the shaft:
Calculation and testing of shafts
Force from belt drive and wheel:
Inside: + Torque I shaft (working): T (N.mm)
+ Radius of belt divider: r (mm) + Vehicle mass: m (kg)
+ Gravity acceleration: g (m/s 2 ) Choose: l1 = 30 (mm) l2 = 40 (mm) l3 = 80(mm) T1 = 12606 Nmm
Figure 4.2 Force analysis on transmission Projection on the zOy axis:
Figure 4.3 Force analysis when projecting on the xOy axis
Projection on the zOx axis:
Hình 0.1 Force analysis when projecting on the zOx axis Figure 0.1 Force analysis when projecting on the zOx axis Figure 4 4 Force analysis when projecting on the zOx axis
Figure 4.5 Qy and My graph
Figure 4.6 Qx and Mx graph Diameter of dangerous cross-section B, C:
- Shaft diameter at cross-section at B and C:
So we choose the shaft diameter 𝑑 = 21(𝑚𝑚)
4.5.1 Test shaft for fatigue strength:
+ The axes of the reducer are all rotating, the bending stress changes with the period of symmetry, so according to the formula we have:
+ Because the shaft rotates in one direction, the torsional stress changes with the dynamic circuit period, so according to the formula we have:
+ Based on the shaft structure and the shaft moment chart, we can see that the following sections are dangerous sections that need to be tested for fatigue strength:
Cross-section for mounting the pulley (section 12)
+ Selection of mounting: Rolling bearings are mounted on the shaft according to k6 ; pulley, shaft connection according to k6
+ Determine the coefficients for hazardous sections according to the formula:
Axes are machined on lathes, at required dangerous cross-sections 𝑅 𝑎 = 2,5 ÷ 0,63(𝜇𝑚), so according to table (10.8), stress concentration coefficient due to surface state 𝐾 𝑥 = 1,06
No surface strengthening methods are applied, resulting in a strength coefficient of 𝐾 𝑦 = 1.6 According to Table 10.12, when utilizing end mills, the stress concentration coefficient at the keyway for a material with a yield strength of 𝜎 𝑏 = 600 MPa is 𝐾 𝜎 = 1.76 and 𝐾 𝜏 = 1.54.
According to the table (10.10), look up the dimension coefficients 𝜀 𝜎 and 𝜀 𝜏 corresponding to the diameters of the hazardous sections, thereby determining the ratio
𝜀 𝜏 at the keyway on these sections
According to the table (10.11), corresponding to the selected mounting type,
𝜎 𝑏 = 600(𝑀𝑃𝑎) and the diameter of the hazardous section, the ratio 𝐾 𝜎
𝜀 𝜏 due to tension mounting at these sections, on the base, can be found The facility uses the larger of the two values of 𝐾 𝜎
𝜀 𝜎 to calculate 𝐾 𝜎𝑑 and the larger of the two values of 𝐾 𝜏
Factor of safety considering only normal stress:
The factor of safety considering the shear stress only:
Finally calculate the factor of safety:
So the cross sections on the shaft are safe in terms of fatigue strength
To avoid excessive plastic deformation or failure from sudden overloads, such as during machine startup, it is essential to conduct a static strength test on the shaft.
Cross section at dangerous cross section d = 21 mm
Steel C45: sb = 600 MPA, sch = 360 MPa
Satisfy the condition of stability.
Select bearing
Depending on the working conditions, at the bearing positions only the ring force
Ft and the radial force Fr So at bearings 1 and 2, choose a single row ball bearing
Provided that all rolling bearings are oil-lubricated
On axis I, supports are located at B and D
At B and D: dB = dD = 21 mm
We choose 1000905 ultra-lightweight single row ball bearings with parameters:
Forces acting on the bearing:
Fy11 = 557,19 N Thus, the total reaction on each drive is:
So we test at the drive with a load greater than Fl10 = 2048,4 N
4.6.1 Test drive according to dynamic load capacity
The dynamic load capacity Cd is calculated according to the formula (11.1)
For ball bearings, calculated by the formula: Q = (X.V.Fr +Y.Fa)kt.kđ (4.56)
+ V = 1 coefficients taking into account the revolutions in rotation
+ kt =1 factor taking into account the effect of heat when working θ = 150 o C + kđ = (1÷1,2) factor taking into account the load characteristic
With: Lh life in hours
10 6 = 20,74 (4.58) m = 3 degree of fatigue curve in rolling bearing test (for ball bearings)
So according to table (P2.1), it is reasonable to choose a 306 medium-sized ball bearing
4.6.2 Test drive according to static load capacity:
To prevent residual deformation, ball bearings need to satisfy the following conditions:
With: Qt conventional static load (kN)
Calculated according to the formula: Qt = Xo.Fr + Yo.Fa (4.60)
With: + Xo = 0,6 radial load factor
We choose 1000905 ultra-lightweight single row ball bearings, which is reasonable
ELECTRICAL-CONTROL SYSTEM DESIGN
Overview
A comprehensive mechatronic system integrates mechanical, electrical, and control components This chapter focuses on the design and calculation of the electrical and control systems for the service robot, following the earlier presentation of its mechanical system The electrical system comprises a central control computer, a master microcontroller, a slave microcontroller, an actuator system, and a sensor system To enable self-propulsion, the robot processes input signals from ultrasonic sensors and encoders, utilizing peripheral devices to generate appropriate outputs for the self-propelled operation.
Electric power system
5.2.1 Communication between computer-microcontroller and microcontroller- microcontroller
Effective communication between computers and microcontrollers is crucial for control and measurement applications Central computers typically have significantly faster processing speeds than microcontrollers, allowing them to manage human interactions and relay service data to robots for tasks like navigation To facilitate this communication, it is essential to implement a serial communication standard, specifically UART, by connecting the computer and microcontroller through a COM port with matching data transmission rates (baud) Additionally, various built-in data types are available within microcontroller families to enhance functionality.
USART Universal Synchronous Asynchronous Receiver and Transmitter
2-wire data transmission (I2C: Inter-Integrated Circuit):
Synchronous serial transmission utilizes one data line and one clock line, where the master device generates clock pulses for the slave device, with data transfer rates determined by the clock frequency In contrast, asynchronous serial transmission involves a single data transmission line and a receiving line without a clock signal, requiring both the transmitter and receiver to independently generate matching clock signals, commonly referred to as the data rate or baud rate, such as 2400 baud or 4800 baud.
RS232, which stands for Recommended Standard 232, is a widely utilized serial communication standard that enables the connection of peripheral devices to computers This asynchronous format supports a maximum of two devices and allows for connection lengths ranging from 12.5 to 25.4 meters The standard typically operates at speeds of 20 kbit/s, with some specialized devices capable of reaching up to 115 kbit/s Key advantages of the RS232 serial interface include its simplicity and reliability in data transmission.
- High anti-interference ability of serial ports
- Peripherals can be removed even when the computer is powered up
- Simple circuits can receive power supply voltage through series work
- Commonly used standard data rate values: 9600, 19200, 28800, 38400 56600,
➢ RS232 is very suitable for computer communication with microcontrollers
5.2.2 Communication between actuator microcontroller and sensor
Slave microcontrollers manage two BLDC motors and communicate with the Master microcontroller and PC to relay motor control states They are linked to a sensor block that gathers environmental signals from objects like desks, chairs, and people.
The Master microcontroller has the function of communicating with the central computer block, sending data as a position string to the Slave microcontroller to perform the migration process
Just like humans have senses, the sensor system is the same as the senses in automation systems Therefore, the sensor system plays a very important role
The ultrasonic sensor operates by continuously emitting high-frequency sound waves that are inaudible to humans When these sound waves hit a solid or liquid obstacle, they create reflected wavelengths The sensor then receives and analyzes these reflections to accurately determine the distance between the sensor and the obstacle.
Figure 5.3 Working principle of ultrasonic sensor [4]
An encoder, or rotary encoder, is an electromechanical device that translates the angular position or motion of a shaft into an analog or digital output signal It is essential for detecting motor position, movement direction, and speed by counting shaft revolutions The encoder features a circular optical disc with evenly spaced small holes across 360 degrees, and its resolution is determined by the number of these grooves; a higher number of holes results in greater resolution.
The encoder operates by utilizing a rotating disc with grooves that allow an optical signal from an LED to shine through When light passes through a groove, it indicates a "yes" signal, while the absence of a groove results in a "no" signal This on-and-off pattern activates the light receiver sensor, generating square pulse signals that are sent to the central processor These pulse signals are crucial for measuring and determining the position and speed of the motor.
To ensure stable operation and precise speed control of the motors, we utilized two drivers, one for each motor The control circuit processes data from the computer to calculate and output the required PWM pulse values.
In restaurant environments, service robots face challenges such as navigating around fixed and movable obstacles like people, tables, and chairs To enhance safety and service quality, it is crucial for these robots to have obstacle detection capabilities to prevent collisions that could damage the robot or harm individuals Various detection methods exist, including lidar, cameras, and radar sensors; however, our team opted for ultrasonic sensors due to their resilience to environmental interference, moderate accuracy, and ease of programming on microcontrollers To maximize coverage and minimize interference, we strategically positioned three sensors at the front of the robot, each with a 15-degree angle, and added three more sensors to extend the detection range This configuration allows the robot to detect obstacles effectively, prompting it to stop temporarily when an obstacle is encountered.
5.2.7 Design of the center console
The robot features three controllers that alternate roles within the system, with the Slave microcontroller managing the sensor structure Additionally, the robot is equipped to build an interface that collects data from customers and transmits it to the restaurant's data center Interaction occurs via the robot's screen, microphone, and speaker, facilitating seamless communication.
Figure 5.5 Structure of control system
DISTRIBUTED SERVICE ROBOT SYSTEM
Overview
To develop a fully automated working environment, our team is investigating distributed control algorithms This system features three robots, each with distinct functions, that collaborate and share data effectively This chapter will outline the control algorithms implemented in the system.
Proposed Distributed Service Robot System (3 robots)
We propose a distributed control system in restaurant service including a PC server and 3 mobile robots with the following tasks and functions:
Robot 1 (Kiri): welcome customers at the entrance At the same time, display information about available tables on the screen
Robot 2 (Isarine): act as a waiter to help customers order food
Robot 3 (Bella): act as a waiter with the role of bringing food to customers.
Operation procedure of Distributed Service Robot System
Figure 6.1 Distributed service robot system flowchart First, customers choose the available table position displayed on the screen of robot
1 (Kiri) After the customer is directed to the table Robot 2 (Isarine) receives the command from robot 1 (Kiri) At this time, the process of calculating the optimal path
36 will be created based on the A* algorithm with the current position, the position of table that the customer has selected as inputs
Robot 2, known as Isarine, presents the menu screen to facilitate the ordering process for customers This process involves voice interactions between the customer and Isarine Once the customer completes their order, Isarine compiles the details of the selected dish and transmits this information to the server PC, also referred to as the Reception PC.
Once the chef finalizes the dish, the reception staff selects the table for serving, and robot 3, named Bella, utilizes the A* algorithm to navigate to the designated table This process is repeated for each new customer Additionally, to prevent collisions during movement, robots 1 and 2 continuously share their position information, enabling them to avoid unwanted encounters while navigating.
Data transmission methods between robots
Bluetooth is a crucial short-distance communication technology found in a wide range of devices, including computers and smartphones It is poised to play a pivotal role in the development of IoT products, enabling seamless communication between devices and smartphones, which are considered a dominant force in today's tech landscape.
Bluetooth technology is primarily optimized for small data transfers rather than large file transfers, making it ideal for use with smartphones, which are essential devices for most users today According to Bluetooth SIG, over 90% of smartphones now come equipped with Bluetooth capabilities across iOS, Android, and Windows platforms, with expectations for "Smart Ready" functionality by 2018.
A socket is a network application programming interface that facilitates the transmission and reception of data over the internet It establishes a two-way communication link between two programs running on a network, allowing them to communicate effectively The endpoint of this communication link, known as a socket, encompasses the Internet Protocol (IP) address along with the host or port used by the computers for data transmission All applications involved in the data exchange utilize the socket to send and receive information.
A TCP/IP socket serves to identify a computer or network server by its unique IP address and port number, which directs the receiving computer on where to send requested data This combination of the IP address and port number forms the basis of a socket.
The purpose of the TCP/IP socket is to identify the IP address for a computer or server
A port address refers to the specific number assigned to a port for sending and receiving data Each computer device utilizes multiple ports for communication, allowing for efficient data transfer For instance, devices like printers require designated ports to receive information from the computer In internet communication, this can involve an Ethernet port linked to a specific device or the port address of a server that connects multiple computers.
A computer socket, such as a TCP/IP socket, enables communication between two devices by directing where the data should be sent The IP address and port number within the TCP/IP socket inform the receiving computer of the destination for the requested data, while also establishing a common language for interaction between the machines.
When a socket is established, both the sender and receiver verify the communication port to ensure data transmission is successful The sender initiates the process by sending a request that identifies itself, to which the receiver replies with a confirmation of the sender's identity and its own As long as the communication is successful and the socket is operational, additional data requests can proceed uninterrupted.
Sockets are established whenever a communication protocol connects multiple devices, and they are closed once the communication concludes If the requested port is unavailable, the socket creation fails, resulting in an error message for both the sender and receiver indicating the failure.
Applications utilized for communication, such as web browsers, rely on TCP/IP sockets to transmit and receive data While a socket is active, all communications are linked to the designated port address associated with that socket As data is exchanged, applications seek the specific port address or host of the socket.
In this report, we use the TCP Socket data transfer method because of the following advantages:
• Is reliable: Packets dropped in the network are detected and retransmitted by the sender
• Has in-order data delivery: Data is read by your application in the order it was written by the sender
• Built a client and server that can handle multiple connections simultaneously
• Economically, we don't need to pay for a connection module circuit like Bluetooth.
Localize robot in 3D space
The team's robot positioning idea will be briefly summarized as follows:
We train the YOLO model using custom data of robot images to accurately detect pixel coordinates of the robot These coordinates are then translated into the robot's operating environment However, signal stability from the camera can be compromised, particularly in low-light conditions, leading to potential signal loss during operation This instability impacts the efficiency of the YOLO algorithm To mitigate this issue, we incorporate additional kinematic equations through a technique called Dead Reckoning, allowing us to continuously update the robot's position in real-time for effective control.
Object detection has emerged as a prominent topic in deep learning due to its wide applicability, ease of data preparation, and impressive results Advanced algorithms like YOLO and SSD offer rapid and accurate performance, enabling real-time tasks that can surpass human capabilities without sacrificing precision This technology supports a variety of applications, including object counting, automated payment systems, timekeeping, and fire detection, demonstrating its versatility across numerous fields.
The diverse and readily available sources of image data provide a significant advantage for training object detection models This high applicability simplifies data preparation and training processes In this context, we will explore the state-of-the-art object detection algorithm known as YOLO.
YOLO in object detection means “You only look once” That is, we only need to look once to be able to detect the object
While YOLO may not be the most accurate object detection algorithm, it stands out as the fastest option in its class It delivers near real-time performance, striking a balance between speed and accuracy when compared to leading models.
YOLO is an advanced object detection algorithm designed to not only predict labels for various objects but also accurately identify their locations within an image This capability allows YOLO to detect multiple labeled objects simultaneously, rather than limiting itself to classifying a single label for the entire image.
YOLO network architecture includes: base networks are convolution networks that perform feature extraction The following are the Extra Layers applied to object detection on the feature map
YOLO's Base network using only convolutional layers and fully connected layers YOLO architectures are also quite diverse and can be customized into versions for many different input shapes
The YOLO network architecture utilizes the Darknet Architecture as its base network, which is responsible for feature extraction This base network generates a 7x7x1024 feature map that serves as input for additional layers tasked with predicting object labels and bounding box coordinates.
The size of the feature map will depend on the input For input 416x416, the feature map has dimensions of 13x13, 26x26 and 52x52 And when input is 608x608 will generate 19x19, 38x38, 72x72 feature map
The output of the YOLO model is a vector that will include the main components
• 𝑝0 is t he predicted probability of the object appearing in the bounding box
• 〈𝑡𝑥,𝑡𝑦,𝑡𝑤,𝑡ℎ〉 helps define the bounding box In which 𝑡𝑥,𝑡𝑦 is the coordinates of the center and 𝑡𝑤,𝑡ℎ are the width and length dimensions of the bounding box
The predictive probability distribution vector of classes, denoted as 〈𝑝1, 𝑝2, … , 𝑝𝑐〉, is crucial for configuring the correct parameters during model training using open-source platforms like Darknet The output is determined by the formula (n_class + 5), meaning that for two classes, the output will be 7 Furthermore, when utilizing three anchors per cell, the total number of output parameters becomes (n_class + 5) x 3, resulting in 21 parameters for two classes.
The YOLO network architecture utilizes a 13x13 feature map, where each cell is assigned three anchor boxes of varying sizes—Box 1, Box 2, and Box 3—with their centers aligned to the cell The final output of the YOLO model is a concatenated vector comprising these three bounding boxes.
YOLO v3 utilizes multiple feature maps for object detection, where smaller initial feature maps enhance the prediction of larger objects In contrast, subsequent feature maps are larger in size but maintain a fixed anchor box, facilitating the detection of smaller objects.
Figure 6.4 Feature map With an input shape of 416 x 416, the output is 3 feature maps with dimensions of 13x13, 26x26 and 52x52 respectively
In a YOLO model, each cell of the feature maps utilizes three anchor boxes for feature prediction, resulting in a total of nine distinct anchor boxes (3 feature maps x 3 anchor boxes).
Also on a square feature map S x S, the YOLO v3 model generates a number of anchor boxes: S x S x 3 Thus, the number of anchor boxes on an image will be: (13 x
The training of the YOLO model is significantly slow due to the necessity of simultaneously predicting labels and bounding boxes for a substantial total of 10,647 bounding boxes.
To accurately determine the bounding box for an object, the YOLO model utilizes predefined anchor boxes that closely encompass the object Subsequently, a regression bounding box algorithm refines these anchor boxes to generate a precise predicted bounding box for the object.
In the training image, each object is associated with an anchor box When multiple anchor boxes encompass the same object, we identify the anchor box that exhibits the highest Intersection over Union (IoU) with the ground truth bounding box.
Figure 6.5 Defines the anchor box for an object
In Cell I, three green-bordered anchor boxes are identified, all intersecting with the object's bounding box Among them, the anchor box with the thickest blue border is chosen as the optimal anchor box due to its highest Intersection over Union (IoU) with the ground truth bounding box.
• Each object in the training image is distributed to a cell on the feature map that contains the mid point of the object
Thus, when determining an output object, it will be necessary to identify 2 components associated with it (cell, anchor box) Not just the cell or just the anchor box
YOLO's loss function is divided into two parts: (localization) measures the error of the bounding box and (confidence loss) measures the error of the probability distribution of classes
• 𝑙 𝑖 𝑜𝑏𝑗 : The indicator function has a value of 0.1 to determine if cell 𝒾 contains an object or not Equals 1 if it contains the object and 0 if it doesn't
• 𝑙 𝑖𝑗 𝑜𝑏𝑗 : Indicates whether the jth bounding box of cell 𝒾 is the bounding box of the predicted object
• 𝐶 𝑖𝑗 : Cell confidence score 𝒾, P(contain object) * IoU
• 𝑝𝑖(𝑐): Conditional probability, whether or not cell 𝒾 contains an object of class
We can simply understand their purpose as
• ℒloc is the loss function of the predicted versus actual bounding box
The loss function ℒ𝑐𝑙𝑠 quantifies the probability distribution, comprising two main components: the first sum evaluates the accuracy of predicting the presence of objects within a cell, while the second sum assesses the loss of the probability distribution when an object is indeed present in the cell.
Path Planning
It is possible to classify ways to solve the problem of finding a way for a robot according to two fields fit as follows:
Case 1: Robot does not know the information of the map in advance
Some popular route detection algorithms are as follows [5]:
The wall-following algorithm is effective in navigating a seamless, unbroken maze, allowing a robot to reach its destination by continuously following either the left or right wall.
Figure 6.12 Algorithm according to the right-following rule [5]
Pledge Algorithm: The implementation method of this algorithm is a combination of suitable for sticking to the wall but taking into account the rotation angle Hence,
A robot is capable of navigating complex maps to successfully escape from a maze, even when it starts from a designated position that is isolated from the rest of the layout.
Figure 6.13 Pledge Algorithm get rid of complicated obstacles
Various algorithms, including stochastic, Tremaux, and dead-end filling algorithms, can be utilized for path detection; however, these methods often present several disadvantages.
• Pathfinding is fully automatic without manual case
• The path found is the most accurate and shortest
Case 2: The robot already knows the map information in advance After obtaining information about the map, the automatic pathfinding algorithms can be applied to find the way Automatic pathfinding algorithms are algorithms that help find the way automatically based on known data This automatic route finding is based on divide the map into nodes, then apply algorithms on these nodes to find the path Therefore, in order to apply the automatic pathfinding algorithm, the data about the map must be known, from which information about the nodes can be obtained, for example such as which node belongs to the goable path and which node does not belong to the goable path Many automatic pathfinding algorithms have been developed such as Grass Fire, Greedy Best First Search, Dijkstra or A* [5]:
- Grass Fire Algorithm: In the Grass Fire algorithm, nodes are neighbors to the under consideration will all be selected as the next node under consideration, if the
The algorithm processes neighboring nodes to identify obstacles, which are subsequently ignored It continues to iterate until the destination node is among those being evaluated While this method is straightforward and capable of determining the shortest path, its speed is hindered by the rapidly increasing number of nodes considered as the overall node count rises.
Dijkstra's algorithm is a selective search method that prioritizes neighboring nodes based on the lowest cost value from the currently considered node While it is more complex than the Grass Fire algorithm, Dijkstra's approach is more efficient in finding the shortest path due to the reduced number of nodes it evaluates.
The Greedy Best First Search algorithm is a selective search method that prioritizes estimated costs between neighboring nodes and the destination node, rather than focusing solely on the cost values of neighboring nodes While this approach allows for faster performance compared to Dijkstra's algorithm, it has the drawback of potentially not finding the shortest path.
Algorithm A* is an advanced pathfinding algorithm that combines elements of Dijkstra's algorithm and Greedy Best First Search It evaluates nodes based on both the cost from the current node to neighboring nodes and an estimated cost from those neighbors to the destination This dual evaluation enables A* to efficiently find the shortest path, often yielding results similar to Dijkstra's algorithm but with improved speed.
Figure 6.14 Route finding results by A* algorithm (left) and Dijkstra's algorithm
This report presents a comprehensive overview of the map, with all information considered fully known in advance The robot's pathfinding is efficiently executed using the A* algorithm, which offers several key advantages.
• Pathfinding is fully automatic without manual case
• The robot does not need to track the route
• The path found is the most accurate and shortest
The A* Algorithm is a widely used graph traversal and path planning method that enhances efficiency by directing its search towards the most promising states, similar to Dijkstra’s algorithm This strategic approach allows A* to potentially reduce computation time significantly while effectively seeking near-optimal solutions based on the available dataset or nodes.
The A* algorithm is predominantly utilized in static environments, though it also finds applications in dynamic settings Its base function can be customized to meet specific application requirements Similar to Dijkstra's algorithm, A* operates by identifying the lowest cost path from the starting point to the destination The algorithm employs the least costly path and expands it using the function f(n) = g(n) + h(n), where g(n) represents the actual cost from node n to the starting node, and h(n) denotes the estimated cost of the optimal path from the target node to n.
The A* algorithm, renowned in the gaming industry, has been enhanced through advancements in artificial intelligence, making it applicable in various fields such as robot path planning, urban intelligent transportation, graph theory, and automatic control.
The A* algorithm is an efficient and less computationally-intensive path planning method, making it suitable for constrained and embedded systems It utilizes heuristic information to identify the optimal path by searching for nodes on the map and applying suitable heuristic functions, such as Euclidean, Manhattan, or Diagonal distance The algorithm's efficiency is influenced by two key factors: the resources required for task performance and the computation time needed to execute the task.
The A* algorithm presents a trade-off between speed and accuracy, allowing for reduced time complexity at the cost of increased memory usage, or lower memory consumption resulting in slower execution times Regardless of the approach, the algorithm effectively identifies the shortest path A practical application of the A* algorithm is in navigating to an available parking space in a crowded lot.
The A* algorithm is known for its computational efficiency, making it ideal for use in static environments Its speed and effectiveness, along with its variants, are largely influenced by the precision of the heuristic function employed.
6.6.2 Most Common Types of Heuristic Functions Used In Path Planning Algorithms
Diagonal distance h(n) = D * (dx + dy) + (D2 - 2 * D) * min(dx, dy)
Figure 6.17 Output of A* algorithm with Manhattan distance [22]
Figure 6.18 Output of A* algorithm with Diagonal distance [22]
Figure 6.19 Output of A* algorithm with Euclidean distance [22]
Since Euclidean distance is shorter than Manhattan or diagonal distance, we still get shortest paths, but A* take longer to run This is a worth trade off
6.6.3 Application of A* algorithm in path planning
After selecting the destination node (destination) and the starting node (starting position of the robot), the A* algorithm is basically implemented as follows:
- Select the starting node as the node under consideration and add it to the list of considered nodes
- Update the cost value compared to the starting node g(n) of the nodes neighboring to the node under consideration
Human interactive system
Speech is a fundamental human skill essential for daily communication Over time, our voice has evolved to facilitate the exchange of information, entertainment, and learning For robots to establish a sense of closeness and natural interaction with users, they must possess a voice that enables effective communication and information sharing A robust database is crucial for this process, akin to human development, as it enhances the robot's ability to learn and grow smarter Additionally, speech recognition allows robots to receive and comprehend requests, ensuring they meet the needs of their users effectively.
In this chapter, we present about the robot-human interaction by applying voice recognition to perform intelligent functions of the robot in the topic
6.7.2 Structure of the interactive system
The human-robot interaction system is designed to enhance user experience by accurately recognizing speech and interpreting user intentions By categorizing these intentions, the system enables the robot to perform specific tasks, allowing it to interact and communicate with customers in a manner similar to a real waiter, ultimately improving the overall user experience.
The human-robot interaction system, illustrated in Figure 6.20, processes user input in the form of voice commands Despite its effectiveness, the complexity of audio data poses significant challenges for processing within the system.
To address the issue, audio data is transformed into text for streamlined processing This text is then categorized to identify the user's intent, enabling the robot to respond appropriately and enhance user-robot interaction There are two primary types of text: conversational text, which involves casual dialogue that requires minimal knowledge for response generation, and order text, which pertains to the classification of the customer's desired dish.
When the input data is classified as conversational text The system starts responding based on the operating principle of a chatbot The robot system is capable of
61 answering questions about the restaurant, the food as well as information about itself: name, ability,
When the input data is identified as order text, the system initiates three key processes: confirming the customer's identity, retrieving relevant information, and classifying the ordered dish Just as humans must recognize their conversation partner, robots also need to identify the user to streamline communication by focusing on pertinent information After confirming the identity, the system retrieves vital customer details, such as their name and frequently ordered dishes This enables the system to suggest previously ordered items or other dishes tailored to the customer's preferences.
The final stage of the human-robot interaction system involves aggregating results from prior tasks This block converts written responses back into audio to facilitate ongoing communication Additionally, it compiles data on the customer's selected information and dishes, which is then transmitted to the restaurant's management center for processing.
Modern speech recognition systems predominantly utilize the Hidden Markov Model (HMM), which operates on the premise that a speech signal can be effectively modeled as a static process when analyzed over brief intervals, such as ten milliseconds This assumption allows for the statistical properties of the speech signal to be treated as constant during these short time frames.
In a standard Hidden Markov Model (HMM) for speech processing, the speech signal is segmented into 10-millisecond intervals Each segment's power spectrum, representing the signal's power across different frequencies, is transformed into a vector of real numbers known as cepstral coefficients Typically, the size of this vector is relatively small, often comprising only a few dimensions.
10, although more precise systems can have dimensions 32 or more The final output of the HMM is a sequence of these vectors
To convert speech into text, vectors are aligned with phonemes, the fundamental units of speech, which necessitates extensive training due to the variations in phoneme sounds among different speakers and their pronunciations A specialized algorithm is then utilized to identify the word or words that most likely correspond to the identified phoneme sequence.
Modern speech recognition systems often utilize neural networks to enhance efficiency by simplifying speech signals through feature conversion and dimensionality reduction techniques prior to HMM recognition Additionally, Voice Activity Detectors (VADs) are employed to filter the audio signal, isolating segments that likely contain speech, which helps the recognizer avoid analyzing irrelevant portions of the signal and improves overall processing speed.
6.7.4 Speech to Text with Google API
Numerous libraries are available for speech-to-text conversion, including Wit.ai and API.ai, which provide advanced features such as natural language processing to understand speaker intent, going beyond simple speech recognition In contrast, Google Cloud Speech primarily focuses on accurate speech-to-text conversion.
However, we use the SpeechRecognition library because of its outstanding ease of use
Speech recognition relies on audio input, and the SpeechRecognition library simplifies the process of retrieving this input, allowing users to get started quickly without the need to write complex scripts for microphone access or audio file processing Acting as a wrapper for various popular speech APIs, including Google's Web Speech API, it offers significant flexibility and ease of use, making it an ideal choice for diverse projects However, users should be aware that the features available from each API may not always be guaranteed.
Word embeddings are learned representations of text that ensure words with similar meanings have analogous representations This method of representing words and documents is regarded as a significant advancement in deep learning, particularly in addressing complex natural language processing challenges.
Word embeddings represent words as real-valued vectors within a defined vector space, mapping each word to a unique vector The values of these vectors are learned through methods similar to those used in neural networks, placing this technique within the realm of deep learning.
Key to the approach is the idea of using a dense distributed representation for each word
Words are represented by real-valued vectors with typically tens or hundreds of dimensions, which is a more efficient approach compared to sparse word representations like one-hot encoding that require thousands or even millions of dimensions.
